2024-04-02T14:18:25.194 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-02T14:18:25.194 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} --name pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645 -- /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645/orig.config.yaml 2024-04-02T14:18:25.298 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:18:25.415 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645 branch: wip-batrick-testing-20240402.004512 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} email: pdonnell@redhat.com first_in_suite: false job_id: '7635645' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240402.004512 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync rbytes: false syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240402.004512 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2763 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 sleep_before_teardown: 0 subset: 318/512 suite: fs suite_branch: wip-batrick-testing-20240402.004512 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHX+vuiFpHaCuctNjO+X9/cOuGDm1+16BZqDUdWfrbPaHTF0FCJlawslQwUZPL26zMPphDbRjn7DyK623T3crlM= smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBHea/tAzbc/det7dLVqiIpOXfZaZrD6vzQc4SU8xJUfIIuFR/TZrB3p+rxVCqfHWxUwYCmfNx2mBHS2W4ykynM= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAHQ9umABXGvlkXBtgq3+xzBXvq2nBLey/ECKPtEu6tdho/a9bqhetZfH4kePNVWWrnZP8yK9awtNzk5q95fAm0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-04-02_11:52:43 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-02T14:18:25.415 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa; will attempt to use it 2024-04-02T14:18:25.416 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks 2024-04-02T14:18:25.416 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-02T14:18:25.539 INFO:teuthology.task.internal:Checking packages... 2024-04-02T14:18:25.564 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '191d963ac9de7322da8aff20b567b3fa458e2ef6' 2024-04-02T14:18:25.564 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T14:18:25.564 INFO:teuthology.packaging:ref: None 2024-04-02T14:18:25.564 INFO:teuthology.packaging:tag: None 2024-04-02T14:18:25.564 INFO:teuthology.packaging:branch: wip-batrick-testing-20240402.004512 2024-04-02T14:18:25.564 INFO:teuthology.packaging:sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:18:25.564 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240402.004512 2024-04-02T14:18:25.714 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2609.g191d963a 2024-04-02T14:18:25.715 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-02T14:18:25.722 INFO:teuthology.task.internal:no buildpackages task found 2024-04-02T14:18:25.722 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-02T14:18:25.770 INFO:teuthology.task.internal:Saving configuration 2024-04-02T14:18:25.792 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-02T14:18:25.798 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:18:25.823 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 14:12:55.520513', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHX+vuiFpHaCuctNjO+X9/cOuGDm1+16BZqDUdWfrbPaHTF0FCJlawslQwUZPL26zMPphDbRjn7DyK623T3crlM='} 2024-04-02T14:18:25.847 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 14:12:55.524050', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBHea/tAzbc/det7dLVqiIpOXfZaZrD6vzQc4SU8xJUfIIuFR/TZrB3p+rxVCqfHWxUwYCmfNx2mBHS2W4ykynM='} 2024-04-02T14:18:25.873 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 14:12:55.522505', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAHQ9umABXGvlkXBtgq3+xzBXvq2nBLey/ECKPtEu6tdho/a9bqhetZfH4kePNVWWrnZP8yK9awtNzk5q95fAm0='} 2024-04-02T14:18:25.874 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-02T14:18:25.880 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-02T14:18:25.880 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-02T14:18:25.880 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-02T14:18:25.880 INFO:teuthology.run_tasks:Running task console_log... 2024-04-02T14:18:25.981 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4ee2961160>, signals=[15]) 2024-04-02T14:18:25.981 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-02T14:18:25.987 INFO:teuthology.task.internal:Opening connections... 2024-04-02T14:18:25.987 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-04-02T14:18:25.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:18:26.096 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-04-02T14:18:26.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:18:26.191 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T14:18:26.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:18:26.281 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-02T14:18:26.287 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-04-02T14:18:26.306 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-04-02T14:18:26.306 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-04-02T14:18:26.364 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-04-02T14:18:26.364 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-04-02T14:18:26.364 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-04-02T14:18:26.364 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-04-02T14:18:26.364 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-04-02T14:18:26.364 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-04-02T14:18:26.364 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T14:18:26.364 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-04-02T14:18:26.364 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-04-02T14:18:26.365 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T14:18:26.365 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-04-02T14:18:26.365 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T14:18:26.365 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T14:18:26.365 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T14:18:26.365 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-04-02T14:18:26.388 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-04-02T14:18:26.406 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-04-02T14:18:26.406 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-04-02T14:18:26.464 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="CentOS Stream" 2024-04-02T14:18:26.464 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="9" 2024-04-02T14:18:26.464 INFO:teuthology.orchestra.run.smithi113.stdout:ID="centos" 2024-04-02T14:18:26.464 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="rhel fedora" 2024-04-02T14:18:26.464 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="9" 2024-04-02T14:18:26.464 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el9" 2024-04-02T14:18:26.464 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T14:18:26.464 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31" 2024-04-02T14:18:26.464 INFO:teuthology.orchestra.run.smithi113.stdout:LOGO="fedora-logo-icon" 2024-04-02T14:18:26.464 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T14:18:26.465 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://centos.org/" 2024-04-02T14:18:26.465 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T14:18:26.465 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T14:18:26.465 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T14:18:26.465 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-04-02T14:18:26.488 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-04-02T14:18:26.505 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-04-02T14:18:26.505 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-04-02T14:18:26.562 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T14:18:26.563 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T14:18:26.564 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-04-02T14:18:26.579 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-02T14:18:26.590 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-02T14:18:26.602 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-02T14:18:26.603 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-04-02T14:18:26.605 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-04-02T14:18:26.607 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-04-02T14:18:26.623 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-02T14:18:26.628 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-02T14:18:26.629 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-04-02T14:18:26.666 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-04-02T14:18:26.669 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-04-02T14:18:26.758 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-02T14:18:26.828 INFO:teuthology.run_tasks:Running task kernel... 2024-04-02T14:18:26.842 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-02T14:18:26.842 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-02T14:18:26.842 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-02T14:18:26.842 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T14:18:26.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:18:26.860 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-02T14:18:27.241 DEBUG:teuthology.task.kernel:sha1 for client.0 is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-02T14:18:27.241 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-04-02T14:18:27.241 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-04-02T14:18:27.259 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-361.el9.x86_64 2024-04-02T14:18:27.259 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-02T14:18:27.260 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-02T14:18:27.260 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum upgrade -y linux-firmware 2024-04-02T14:18:29.889 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - AppStream 7.2 MB/s | 6.7 MB 00:00 2024-04-02T14:18:31.422 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2024-04-02T14:18:33.638 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-04-02T14:18:35.497 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-02T14:18:40.739 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.1 MB 00:01 2024-04-02T14:18:42.223 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-02T14:18:43.991 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-02T14:18:49.241 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-04-02T14:18:51.704 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-02T14:18:51.717 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-02T14:18:51.718 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-04-02T14:18:51.718 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-02T14:18:51.718 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-04-02T14:18:51.718 INFO:teuthology.orchestra.run.smithi062.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-02T14:18:51.718 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:18:51.718 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-04-02T14:18:51.718 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-02T14:18:51.718 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 1 Package 2024-04-02T14:18:51.718 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:18:51.719 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 375 M 2024-04-02T14:18:51.719 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-04-02T14:19:01.742 INFO:teuthology.orchestra.run.smithi062.stdout:linux-firmware-20240219-143.el9.noarch.rpm 37 MB/s | 375 MB 00:10 2024-04-02T14:19:01.743 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-04-02T14:19:01.743 INFO:teuthology.orchestra.run.smithi062.stdout:Total 37 MB/s | 375 MB 00:10 2024-04-02T14:19:01.744 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-04-02T14:19:01.818 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-04-02T14:19:01.818 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-04-02T14:19:04.302 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-04-02T14:19:04.303 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-04-02T14:19:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-04-02T14:19:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-02T14:19:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-02T14:19:12.548 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-02T14:19:12.548 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-02T14:19:12.944 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-02T14:19:12.944 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:19:12.944 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-04-02T14:19:12.944 INFO:teuthology.orchestra.run.smithi062.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-02T14:19:12.944 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:19:12.944 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-02T14:19:13.219 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-04-02T14:19:13.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-02T14:19:13.375 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-04-02T14:19:13.375 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-02T14:19:13.378 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi062.front.sepia.ceph.com to complete... 2024-04-02T14:19:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02 14:19:25 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-02T14:19:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:FINISHED --2024-04-02 14:19:25-- 2024-04-02T14:19:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:Total wall clock time: 12s 2024-04-02T14:19:25.329 INFO:teuthology.orchestra.run.smithi062.stderr:Downloaded: 1 files, 321M in 12s (27.5 MB/s) 2024-04-02T14:19:25.331 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-04-02T14:19:25.332 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-02T14:19:27.364 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying... ######################################## 2024-04-02T14:19:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing... ######################################## 2024-04-02T14:19:27.503 INFO:teuthology.orchestra.run.smithi062.stdout:Updating / installing... 2024-04-02T14:19:35.491 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-04-02T14:19:51.570 INFO:teuthology.orchestra.run.smithi062.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T14:20:14.865 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-02T14:20:14.865 DEBUG:teuthology.orchestra.run.smithi062:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-02T14:20:14.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-02T14:20:14.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-04-02T14:20:14.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-02T14:20:14.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-04-02T14:20:14.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-04-02T14:20:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-04-02T14:20:14.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-04-02T14:20:14.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-02T14:20:14.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-04-02T14:20:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-04-02T14:20:14.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-04-02T14:20:14.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-02T14:20:14.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-02T14:20:14.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-04-02T14:20:14.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-04-02T14:20:14.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-04-02T14:20:14.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-04-02T14:20:14.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-04-02T14:20:14.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-04-02T14:20:14.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-04-02T14:20:14.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-04-02T14:20:14.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-04-02T14:20:14.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-04-02T14:20:14.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-02T14:20:14.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-02T14:20:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-04-02T14:20:14.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-04-02T14:20:14.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-04-02T14:20:14.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-04-02T14:20:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-04-02T14:20:14.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-04-02T14:20:14.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-04-02T14:20:14.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-04-02T14:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-04-02T14:20:14.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-04-02T14:20:14.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-04-02T14:20:14.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-04-02T14:20:14.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-04-02T14:20:14.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-04-02T14:20:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-02T14:20:14.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-04-02T14:20:14.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-04-02T14:20:14.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-04-02T14:20:14.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-02T14:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-04-02T14:20:14.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-04-02T14:20:14.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-02T14:20:14.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-04-02T14:20:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-04-02T14:20:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-02T14:20:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-04-02T14:20:14.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-04-02T14:20:14.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-04-02T14:20:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-04-02T14:20:14.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-02T14:20:14.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-02T14:20:14.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-02T14:20:14.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-04-02T14:20:14.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-02T14:20:14.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-02T14:20:14.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-04-02T14:20:14.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-02T14:20:14.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-02T14:20:14.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-04-02T14:20:14.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-04-02T14:20:14.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-04-02T14:20:14.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-02T14:20:14.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-04-02T14:20:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-04-02T14:20:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-04-02T14:20:14.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-04-02T14:20:15.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-04-02T14:20:15.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-04-02T14:20:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-04-02T14:20:15.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-04-02T14:20:15.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-02T14:20:15.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-02T14:20:15.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-04-02T14:20:15.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-02T14:20:15.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-02T14:20:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-04-02T14:20:15.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-04-02T14:20:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-02T14:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-04-02T14:20:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-02T14:20:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-04-02T14:20:15.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-04-02T14:20:15.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-04-02T14:20:15.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-02T14:20:15.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-04-02T14:20:15.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-04-02T14:20:15.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-04-02T14:20:15.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-02T14:20:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-04-02T14:20:15.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-02T14:20:15.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-04-02T14:20:15.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-02T14:20:15.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-02T14:20:15.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-04-02T14:20:15.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-04-02T14:20:15.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-02T14:20:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-02T14:20:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-04-02T14:20:15.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-04-02T14:20:15.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-04-02T14:20:15.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-04-02T14:20:15.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-04-02T14:20:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-04-02T14:20:15.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-04-02T14:20:15.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-04-02T14:20:15.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-04-02T14:20:15.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-02T14:20:15.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-04-02T14:20:15.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-04-02T14:20:15.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-04-02T14:20:15.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-04-02T14:20:15.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-04-02T14:20:15.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-02T14:20:15.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-02T14:20:15.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-04-02T14:20:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-02T14:20:15.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-02T14:20:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-02T14:20:15.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-04-02T14:20:15.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-04-02T14:20:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-02T14:20:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-02T14:20:15.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-04-02T14:20:15.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-02T14:20:15.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-02T14:20:15.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-02T14:20:15.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-02T14:20:15.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-04-02T14:20:15.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-02T14:20:15.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-04-02T14:20:15.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-04-02T14:20:15.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-04-02T14:20:15.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-04-02T14:20:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-04-02T14:20:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-04-02T14:20:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-04-02T14:20:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-04-02T14:20:15.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-04-02T14:20:15.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-02T14:20:15.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-04-02T14:20:15.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-04-02T14:20:15.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-04-02T14:20:15.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-04-02T14:20:15.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-04-02T14:20:15.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-04-02T14:20:15.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-04-02T14:20:15.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-04-02T14:20:15.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-04-02T14:20:15.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-04-02T14:20:15.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-04-02T14:20:15.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-04-02T14:20:15.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-02T14:20:15.079 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-04-02T14:20:15.079 DEBUG:teuthology.orchestra.run.smithi062:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-02T14:20:15.101 INFO:teuthology.orchestra.run.smithi062.stdout:postinstall scriptlet (using /bin/sh): 2024-04-02T14:20:15.101 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-02T14:20:15.101 INFO:teuthology.orchestra.run.smithi062.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout:for file in vmlinuz System.map config; do 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout: fi 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout:done 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout:postuninstall scriptlet (using /bin/sh): 2024-04-02T14:20:15.102 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/update-bootloader ]; then 2024-04-02T14:20:15.103 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-04-02T14:20:15.103 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2024-04-02T14:20:15.103 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2024-04-02T14:20:15.185 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2024-04-02T14:20:15.185 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2024-04-02T14:20:15.185 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.06 2024-04-02T14:20:15.185 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 68.el9 2024-04-02T14:20:15.185 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2024-04-02T14:20:15.185 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T14:20:15.185 INFO:teuthology.orchestra.run.smithi062.stdout:Group : Unspecified 2024-04-02T14:20:15.185 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 8264254 2024-04-02T14:20:15.185 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : builder@centos.org 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : CentOS 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T14:20:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2024-04-02T14:20:15.187 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:20:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T14:20:15.187 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T14:20:15.188 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 6.8.0-g1df0c7e8a235 2024-04-02T14:20:15.188 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T14:20:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2024-04-02T14:20:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T14:20:17.377 INFO:teuthology.orchestra.run.smithi062.stderr:done 2024-04-02T14:20:17.380 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-04-02T14:20:17.398 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.PKJNTXvQzI 2024-04-02T14:20:17.399 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PKJNTXvQzI 2024-04-02T14:20:17.471 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.PKJNTXvQzI 2024-04-02T14:20:17.614 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.PKJNTXvQzI is 6KB 2024-04-02T14:20:17.663 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.PKJNTXvQzI 2024-04-02T14:20:17.678 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T14:20:17.747 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T14:20:17.747 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T14:20:17.747 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T14:20:17.747 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-04-02T14:20:17.748 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-04-02T14:20:17.907 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2024-04-02T14:20:47.910 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-02T14:20:47.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-02T14:20:47.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:21:47.915 DEBUG:teuthology.orchestra.remote:timed out 2024-04-02T14:21:56.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-02T14:21:56.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:22:12.903 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-04-02T14:22:13.603 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-02T14:22:13.604 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g1df0c7e8a235"... 2024-04-02T14:22:13.604 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-04-02T14:22:13.643 INFO:teuthology.orchestra.run.smithi062.stdout:6.8.0-g1df0c7e8a235 2024-04-02T14:22:13.643 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-04-02T14:22:13.644 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T14:22:13.644 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T14:22:14.647 DEBUG:teuthology.parallel:result is None 2024-04-02T14:22:14.648 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-02T14:22:14.655 INFO:teuthology.task.internal:Creating test directory... 2024-04-02T14:22:14.655 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T14:22:14.661 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T14:22:14.663 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T14:22:14.701 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-02T14:22:14.708 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-02T14:22:14.715 INFO:teuthology.task.internal:Creating archive directory... 2024-04-02T14:22:14.715 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T14:22:14.719 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T14:22:14.721 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T14:22:14.820 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-02T14:22:14.827 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-02T14:22:14.827 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T14:22:14.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:22:14.844 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T14:22:14.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:22:14.861 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T14:22:14.865 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T14:22:14.887 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T14:22:14.924 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T14:22:14.937 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T14:22:14.956 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T14:22:14.963 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T14:22:15.051 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T14:22:15.100 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T14:22:15.104 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-02T14:22:15.112 INFO:teuthology.task.internal:Configuring sudo... 2024-04-02T14:22:15.112 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T14:22:15.116 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T14:22:15.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T14:22:15.197 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-02T14:22:15.207 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-02T14:22:15.207 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T14:22:15.211 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T14:22:15.213 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T14:22:15.246 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T14:22:15.357 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T14:22:15.443 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T14:22:15.513 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T14:22:15.602 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:22:15.603 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T14:22:15.675 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T14:22:15.712 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T14:22:15.777 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T14:22:15.857 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T14:22:15.906 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-02T14:22:15.906 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T14:22:15.975 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T14:22:16.013 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T14:22:16.074 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T14:22:16.151 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T14:22:16.203 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T14:22:16.203 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T14:22:16.272 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-04-02T14:22:16.280 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-04-02T14:22:16.282 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-04-02T14:22:16.319 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:22:16.351 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:22:16.407 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:22:16.792 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-02T14:22:16.800 INFO:teuthology.task.internal:Starting timer... 2024-04-02T14:22:16.800 INFO:teuthology.run_tasks:Running task pcp... 2024-04-02T14:22:16.823 INFO:teuthology.run_tasks:Running task selinux... 2024-04-02T14:22:16.831 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-02T14:22:16.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-04-02T14:22:16.960 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-04-02T14:22:16.964 DEBUG:teuthology.orchestra.run.smithi113:> sudo service auditd rotate 2024-04-02T14:22:17.030 INFO:teuthology.orchestra.run.smithi113.stdout:Rotating logs: 2024-04-02T14:22:17.033 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-04-02T14:22:17.086 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-04-02T14:22:17.089 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-02T14:22:17.089 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-04-02T14:22:17.145 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-04-02T14:22:17.146 DEBUG:teuthology.orchestra.run.smithi113:> /usr/sbin/getenforce 2024-04-02T14:22:17.165 INFO:teuthology.orchestra.run.smithi113.stdout:Permissive 2024-04-02T14:22:17.165 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-04-02T14:22:17.192 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-04-02T14:22:17.192 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi113.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-04-02T14:22:17.192 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:22:17.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:22:17.264 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:22:17.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:22:17.294 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:22:17.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:22:17.323 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-02T14:22:17.323 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-04-02T14:22:17.399 DEBUG:teuthology.orchestra.run.smithi113:> sudo /usr/sbin/setenforce permissive 2024-04-02T14:22:17.433 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-04-02T14:22:17.470 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-02T14:22:17.479 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-02T14:22:17.485 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-02T14:22:17.643 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-02T14:22:17.659 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-02T14:22:17.660 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi113.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-02T14:28:27.795 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi113.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-04-02T14:28:27.797 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-02T14:28:27.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:28:27.947 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-04-02T14:28:28.112 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-02T14:28:28.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-02T14:28:28.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:28:28.204 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-04-02T14:28:28.275 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-02T14:28:28.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-02T14:28:28.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:28:28.365 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-04-02T14:28:28.435 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-02T14:28:28.436 INFO:teuthology.run_tasks:Running task clock... 2024-04-02T14:28:28.447 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-02T14:28:28.447 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T14:28:28.448 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:28:28.455 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T14:28:28.455 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:28:28.457 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T14:28:28.457 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:28:28.490 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T14:28:28.509 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T14:28:28.510 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T14:28:28.528 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T14:28:28.539 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: ntpd: command not found 2024-04-02T14:28:28.554 INFO:teuthology.orchestra.run.smithi113.stdout:506 Cannot talk to daemon 2024-04-02T14:28:28.558 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T14:28:28.559 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-04-02T14:28:28.570 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T14:28:28.573 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-04-02T14:28:28.587 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T14:28:28.590 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T14:28:28.607 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T14:28:28.613 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T14:28:28.631 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:28:28.634 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:28:28.635 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-04-02T14:28:28.635 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:28.635 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:28.635 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:28.635 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:28.651 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:28:28.654 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:28:28.654 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-04-02T14:28:28.654 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:28.654 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:28.654 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:28.654 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-04-02T14:28:28.761 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-04-02T14:28:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T14:28:28.874 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T14:28:29.012 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:28:29.021 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:28:29.021 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-04-02T14:28:29.021 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:29.021 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:29.021 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:29.021 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:29.023 INFO:teuthology.run_tasks:Running task install... 2024-04-02T14:28:29.031 DEBUG:teuthology.task.install:project ceph 2024-04-02T14:28:29.032 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}} 2024-04-02T14:28:29.032 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'} 2024-04-02T14:28:29.032 INFO:teuthology.task.install:Using flavor: default 2024-04-02T14:28:29.038 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-02T14:28:29.038 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-02T14:28:29.039 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T14:28:29.039 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:28:29.041 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T14:28:29.041 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:28:29.042 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T14:28:29.042 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:28:29.190 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T14:28:29.190 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T14:28:29.260 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T14:28:29.261 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T14:28:29.269 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T14:28:29.269 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T14:28:29.301 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T14:28:29.301 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T14:28:29.301 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T14:28:29.333 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T14:28:29.333 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T14:28:29.404 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T14:28:29.468 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T14:28:29.469 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:28:29.469 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T14:28:29.473 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T14:28:29.473 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-02T14:28:29.473 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T14:28:29.487 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T14:28:29.504 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T14:28:29.505 DEBUG:teuthology.orchestra.run.smithi113:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T14:28:29.557 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-04-02T14:28:29.558 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-04-02T14:28:29.572 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T14:28:29.572 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T14:28:29.577 DEBUG:teuthology.orchestra.run.smithi113:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T14:28:29.661 DEBUG:teuthology.orchestra.run.smithi113:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T14:28:29.718 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T14:28:29.729 INFO:teuthology.orchestra.run.smithi113.stdout:check_obsoletes = 1 2024-04-02T14:28:29.730 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum clean all 2024-04-02T14:28:29.800 INFO:teuthology.orchestra.run.smithi204.stdout:74 files removed 2024-04-02T14:28:29.827 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T14:28:29.859 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T14:28:29.955 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-04-02T14:28:29.960 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-04-02T14:28:29.967 INFO:teuthology.orchestra.run.smithi113.stdout:74 files removed 2024-04-02T14:28:29.994 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T14:28:30.337 INFO:teuthology.orchestra.run.smithi062.stdout:74 files removed 2024-04-02T14:28:30.381 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T14:28:30.470 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 237 kB/s | 83 kB 00:00 2024-04-02T14:28:30.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph packages for x86_64 199 kB/s | 83 kB 00:00 2024-04-02T14:28:30.846 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-04-02T14:28:31.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-02T14:28:31.238 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-04-02T14:28:31.346 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 190 kB/s | 83 kB 00:00 2024-04-02T14:28:31.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-02T14:28:31.685 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-04-02T14:28:31.858 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-04-02T14:28:31.939 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-04-02T14:28:32.523 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-04-02T14:28:33.108 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-04-02T14:28:33.268 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-04-02T14:28:33.273 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-04-02T14:28:34.007 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-02T14:28:35.069 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-04-02T14:28:36.039 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-02T14:28:36.242 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-02T14:28:40.168 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-04-02T14:28:41.443 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 108 kB/s | 16 kB 00:00 2024-04-02T14:28:42.295 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 29 MB/s | 21 MB 00:00 2024-04-02T14:28:44.354 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 765 kB/s | 8.0 MB 00:10 2024-04-02T14:28:47.337 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-02T14:28:49.863 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T14:28:49.864 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T14:28:49.878 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T14:28:49.880 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T14:28:49.880 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T14:28:49.881 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T14:28:49.883 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T14:28:49.886 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T14:28:49.887 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T14:28:49.968 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-04-02T14:28:49.991 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-04-02T14:28:49.991 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-04-02T14:28:49.991 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-04-02T14:28:49.991 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-04-02T14:28:49.991 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T14:28:49.991 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T14:28:49.991 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T14:28:49.991 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T14:28:49.992 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T14:28:49.993 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T14:28:49.994 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T14:28:49.995 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T14:28:49.996 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T14:28:49.997 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T14:28:49.998 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T14:28:49.999 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T14:28:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T14:28:50.001 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T14:28:50.002 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T14:28:50.002 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T14:28:50.002 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T14:28:50.002 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T14:28:50.002 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T14:28:50.002 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T14:28:50.003 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T14:28:50.004 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T14:28:50.004 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T14:28:50.004 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T14:28:50.004 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T14:28:50.004 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T14:28:50.004 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T14:28:50.004 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T14:28:50.004 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T14:28:50.004 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T14:28:50.005 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T14:28:50.006 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T14:28:50.006 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T14:28:50.006 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T14:28:50.006 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-04-02T14:28:50.006 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T14:28:50.006 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:28:50.006 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-04-02T14:28:50.006 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-04-02T14:28:50.006 INFO:teuthology.orchestra.run.smithi204.stdout:Install 150 Packages 2024-04-02T14:28:50.006 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 24 Packages 2024-04-02T14:28:50.007 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:28:50.007 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 226 M 2024-04-02T14:28:50.008 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-04-02T14:28:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-04-02T14:28:50.697 INFO:teuthology.orchestra.run.smithi204.stdout:(2/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 5.5 MB/s | 890 kB 00:00 2024-04-02T14:28:50.747 INFO:teuthology.orchestra.run.smithi204.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 2.8 MB/s | 145 kB 00:00 2024-04-02T14:28:51.022 INFO:teuthology.orchestra.run.smithi204.stdout:(4/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-04-02T14:28:51.222 INFO:teuthology.orchestra.run.smithi204.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-04-02T14:28:51.298 INFO:teuthology.orchestra.run.smithi204.stdout:(6/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 5.8 MB/s | 5.4 MB 00:00 2024-04-02T14:28:51.781 INFO:teuthology.orchestra.run.smithi204.stdout:(7/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 8.4 MB/s | 4.7 MB 00:00 2024-04-02T14:28:52.707 INFO:teuthology.orchestra.run.smithi204.stdout:(8/174): ceph-osd-19.0.0-2609.g191d963a.el9.x86 12 MB/s | 17 MB 00:01 2024-04-02T14:28:52.782 INFO:teuthology.orchestra.run.smithi204.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 336 kB/s | 25 kB 00:00 2024-04-02T14:28:53.324 INFO:teuthology.orchestra.run.smithi204.stdout:(10/174): ceph-common-19.0.0-2609.g191d963a.el9 7.0 MB/s | 21 MB 00:02 2024-04-02T14:28:53.432 INFO:teuthology.orchestra.run.smithi204.stdout:(11/174): ceph-radosgw-19.0.0-2609.g191d963a.el 6.0 MB/s | 9.8 MB 00:01 2024-04-02T14:28:53.457 INFO:teuthology.orchestra.run.smithi204.stdout:(12/174): cephfs-mirror-19.0.0-2609.g191d963a.e 1.6 MB/s | 225 kB 00:00 2024-04-02T14:28:53.474 INFO:teuthology.orchestra.run.smithi204.stdout:(13/174): cephfs-shell-19.0.0-2609.g191d963a.el 646 kB/s | 27 kB 00:00 2024-04-02T14:28:53.516 INFO:teuthology.orchestra.run.smithi204.stdout:(14/174): libcephfs-devel-19.0.0-2609.g191d963a 542 kB/s | 32 kB 00:00 2024-04-02T14:28:53.583 INFO:teuthology.orchestra.run.smithi204.stdout:(15/174): libcephfs2-19.0.0-2609.g191d963a.el9. 6.7 MB/s | 747 kB 00:00 2024-04-02T14:28:53.609 INFO:teuthology.orchestra.run.smithi204.stdout:(16/174): libcephsqlite-19.0.0-2609.g191d963a.e 1.7 MB/s | 164 kB 00:00 2024-04-02T14:28:53.642 INFO:teuthology.orchestra.run.smithi204.stdout:(17/174): librados-devel-19.0.0-2609.g191d963a. 2.1 MB/s | 127 kB 00:00 2024-04-02T14:28:53.701 INFO:teuthology.orchestra.run.smithi204.stdout:(18/174): libradosstriper1-19.0.0-2609.g191d963 5.3 MB/s | 500 kB 00:00 2024-04-02T14:28:53.784 INFO:teuthology.orchestra.run.smithi204.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 543 kB/s | 45 kB 00:00 2024-04-02T14:28:53.876 INFO:teuthology.orchestra.run.smithi204.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 1.4 MB/s | 134 kB 00:00 2024-04-02T14:28:53.968 INFO:teuthology.orchestra.run.smithi204.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 1.7 MB/s | 161 kB 00:00 2024-04-02T14:28:54.077 INFO:teuthology.orchestra.run.smithi204.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 2.9 MB/s | 320 kB 00:00 2024-04-02T14:28:54.177 INFO:teuthology.orchestra.run.smithi204.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 2.9 MB/s | 299 kB 00:00 2024-04-02T14:28:54.261 INFO:teuthology.orchestra.run.smithi204.stdout:(24/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.2 MB/s | 99 kB 00:00 2024-04-02T14:28:54.336 INFO:teuthology.orchestra.run.smithi204.stdout:(25/174): librgw2-19.0.0-2609.g191d963a.el9.x86 7.4 MB/s | 5.1 MB 00:00 2024-04-02T14:28:54.361 INFO:teuthology.orchestra.run.smithi204.stdout:(26/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 853 kB/s | 85 kB 00:00 2024-04-02T14:28:54.437 INFO:teuthology.orchestra.run.smithi204.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 2.2 MB/s | 170 kB 00:00 2024-04-02T14:28:54.529 INFO:teuthology.orchestra.run.smithi204.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 370 kB/s | 34 kB 00:00 2024-04-02T14:28:54.613 INFO:teuthology.orchestra.run.smithi204.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 1.7 MB/s | 141 kB 00:00 2024-04-02T14:28:54.713 INFO:teuthology.orchestra.run.smithi204.stdout:(30/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 8.2 MB/s | 3.1 MB 00:00 2024-04-02T14:28:55.121 INFO:teuthology.orchestra.run.smithi204.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 7.0 MB/s | 3.5 MB 00:00 2024-04-02T14:28:55.214 INFO:teuthology.orchestra.run.smithi204.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 2.7 MB/s | 249 kB 00:00 2024-04-02T14:28:55.297 INFO:teuthology.orchestra.run.smithi204.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 592 kB/s | 49 kB 00:00 2024-04-02T14:28:55.372 INFO:teuthology.orchestra.run.smithi204.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 219 kB/s | 16 kB 00:00 2024-04-02T14:28:55.456 INFO:teuthology.orchestra.run.smithi204.stdout:(35/174): ceph-volume-19.0.0-2609.g191d963a.el9 3.1 MB/s | 264 kB 00:00 2024-04-02T14:28:55.664 INFO:teuthology.orchestra.run.smithi204.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 7.8 MB/s | 7.4 MB 00:00 2024-04-02T14:28:55.690 INFO:teuthology.orchestra.run.smithi204.stdout:(37/174): cephadm-19.0.0-2609.g191d963a.el9.noa 3.2 MB/s | 762 kB 00:00 2024-04-02T14:28:55.748 INFO:teuthology.orchestra.run.smithi204.stdout:(38/174): cephfs-top-19.0.0-2609.g191d963a.el9. 218 kB/s | 18 kB 00:00 2024-04-02T14:28:56.182 INFO:teuthology.orchestra.run.smithi204.stdout:(39/174): ceph-test-19.0.0-2609.g191d963a.el9.x 14 MB/s | 48 MB 00:03 2024-04-02T14:28:56.266 INFO:teuthology.orchestra.run.smithi204.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 662 kB 00:00 2024-04-02T14:28:56.291 INFO:teuthology.orchestra.run.smithi204.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 980 kB/s | 106 kB 00:00 2024-04-02T14:28:56.318 INFO:teuthology.orchestra.run.smithi204.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 1.6 MB/s | 920 kB 00:00 2024-04-02T14:28:56.343 INFO:teuthology.orchestra.run.smithi204.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.9 MB/s | 307 kB 00:00 2024-04-02T14:28:56.360 INFO:teuthology.orchestra.run.smithi204.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 462 kB/s | 32 kB 00:00 2024-04-02T14:28:56.386 INFO:teuthology.orchestra.run.smithi204.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 412 kB/s | 17 kB 00:00 2024-04-02T14:28:56.419 INFO:teuthology.orchestra.run.smithi204.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 172 kB/s | 10 kB 00:00 2024-04-02T14:28:56.478 INFO:teuthology.orchestra.run.smithi204.stdout:(47/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-04-02T14:28:56.503 INFO:teuthology.orchestra.run.smithi204.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 159 kB 00:00 2024-04-02T14:28:56.529 INFO:teuthology.orchestra.run.smithi204.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 410 kB/s | 44 kB 00:00 2024-04-02T14:28:56.554 INFO:teuthology.orchestra.run.smithi204.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-04-02T14:28:56.588 INFO:teuthology.orchestra.run.smithi204.stdout:(51/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-04-02T14:28:56.613 INFO:teuthology.orchestra.run.smithi204.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 562 kB/s | 46 kB 00:00 2024-04-02T14:28:56.638 INFO:teuthology.orchestra.run.smithi204.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 282 kB 00:00 2024-04-02T14:28:56.663 INFO:teuthology.orchestra.run.smithi204.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-04-02T14:28:56.688 INFO:teuthology.orchestra.run.smithi204.stdout:(55/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-04-02T14:28:56.722 INFO:teuthology.orchestra.run.smithi204.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 406 kB/s | 34 kB 00:00 2024-04-02T14:28:56.823 INFO:teuthology.orchestra.run.smithi204.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-02T14:28:56.865 INFO:teuthology.orchestra.run.smithi204.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 8.7 MB/s | 1.5 MB 00:00 2024-04-02T14:28:56.890 INFO:teuthology.orchestra.run.smithi204.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 4.9 MB/s | 817 kB 00:00 2024-04-02T14:28:56.915 INFO:teuthology.orchestra.run.smithi204.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.5 MB/s | 132 kB 00:00 2024-04-02T14:28:56.957 INFO:teuthology.orchestra.run.smithi204.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-02T14:28:57.015 INFO:teuthology.orchestra.run.smithi204.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 776 kB/s | 78 kB 00:00 2024-04-02T14:28:57.100 INFO:teuthology.orchestra.run.smithi204.stdout:(63/174): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.7 MB 00:00 2024-04-02T14:28:57.208 INFO:teuthology.orchestra.run.smithi204.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-04-02T14:28:57.224 INFO:teuthology.orchestra.run.smithi204.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 118 kB/s | 24 kB 00:00 2024-04-02T14:28:57.241 INFO:teuthology.orchestra.run.smithi204.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 130 kB/s | 18 kB 00:00 2024-04-02T14:28:57.267 INFO:teuthology.orchestra.run.smithi204.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-04-02T14:28:57.284 INFO:teuthology.orchestra.run.smithi204.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 760 kB/s | 32 kB 00:00 2024-04-02T14:28:57.334 INFO:teuthology.orchestra.run.smithi204.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.2 MB/s | 217 kB 00:00 2024-04-02T14:28:57.409 INFO:teuthology.orchestra.run.smithi204.stdout:(70/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-04-02T14:28:57.434 INFO:teuthology.orchestra.run.smithi204.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-04-02T14:28:57.460 INFO:teuthology.orchestra.run.smithi204.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 558 kB/s | 69 kB 00:00 2024-04-02T14:28:57.476 INFO:teuthology.orchestra.run.smithi204.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 993 kB/s | 41 kB 00:00 2024-04-02T14:28:57.502 INFO:teuthology.orchestra.run.smithi204.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-04-02T14:28:57.527 INFO:teuthology.orchestra.run.smithi204.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-04-02T14:28:57.552 INFO:teuthology.orchestra.run.smithi204.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-02T14:28:57.569 INFO:teuthology.orchestra.run.smithi204.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 709 kB/s | 47 kB 00:00 2024-04-02T14:28:57.586 INFO:teuthology.orchestra.run.smithi204.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 733 kB/s | 43 kB 00:00 2024-04-02T14:28:57.603 INFO:teuthology.orchestra.run.smithi204.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-04-02T14:28:57.679 INFO:teuthology.orchestra.run.smithi204.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 445 kB/s | 41 kB 00:00 2024-04-02T14:28:57.762 INFO:teuthology.orchestra.run.smithi204.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-04-02T14:28:57.987 INFO:teuthology.orchestra.run.smithi204.stdout:(82/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-04-02T14:28:58.004 INFO:teuthology.orchestra.run.smithi204.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 108 kB/s | 35 kB 00:00 2024-04-02T14:28:58.021 INFO:teuthology.orchestra.run.smithi204.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 180 kB/s | 46 kB 00:00 2024-04-02T14:28:58.046 INFO:teuthology.orchestra.run.smithi204.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-04-02T14:28:58.074 INFO:teuthology.orchestra.run.smithi204.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-02T14:28:58.097 INFO:teuthology.orchestra.run.smithi204.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 837 kB/s | 63 kB 00:00 2024-04-02T14:28:58.131 INFO:teuthology.orchestra.run.smithi204.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 524 kB/s | 44 kB 00:00 2024-04-02T14:28:58.148 INFO:teuthology.orchestra.run.smithi204.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-02T14:28:58.173 INFO:teuthology.orchestra.run.smithi204.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-02T14:28:58.206 INFO:teuthology.orchestra.run.smithi204.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-04-02T14:28:58.223 INFO:teuthology.orchestra.run.smithi204.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-04-02T14:28:58.249 INFO:teuthology.orchestra.run.smithi204.stdout:(93/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-04-02T14:28:58.274 INFO:teuthology.orchestra.run.smithi204.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2024-04-02T14:28:58.416 INFO:teuthology.orchestra.run.smithi204.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 976 kB/s | 187 kB 00:00 2024-04-02T14:28:58.441 INFO:teuthology.orchestra.run.smithi204.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-02T14:28:58.466 INFO:teuthology.orchestra.run.smithi204.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-02T14:28:58.500 INFO:teuthology.orchestra.run.smithi204.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-04-02T14:28:58.708 INFO:teuthology.orchestra.run.smithi204.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 80 kB/s | 21 kB 00:00 2024-04-02T14:28:58.734 INFO:teuthology.orchestra.run.smithi204.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 95 kB/s | 25 kB 00:00 2024-04-02T14:28:59.059 INFO:teuthology.orchestra.run.smithi204.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 701 kB/s | 245 kB 00:00 2024-04-02T14:28:59.087 INFO:teuthology.orchestra.run.smithi204.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 701 kB/s | 248 kB 00:00 2024-04-02T14:28:59.138 INFO:teuthology.orchestra.run.smithi204.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-02T14:28:59.163 INFO:teuthology.orchestra.run.smithi204.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 902 kB/s | 94 kB 00:00 2024-04-02T14:28:59.255 INFO:teuthology.orchestra.run.smithi204.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 7.2 MB/s | 863 kB 00:00 2024-04-02T14:28:59.406 INFO:teuthology.orchestra.run.smithi204.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.6 MB/s | 4.1 MB 00:00 2024-04-02T14:28:59.814 INFO:teuthology.orchestra.run.smithi204.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 51 kB/s | 19 kB 00:00 2024-04-02T14:28:59.964 INFO:teuthology.orchestra.run.smithi204.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 685 kB/s | 548 kB 00:00 2024-04-02T14:28:59.998 INFO:teuthology.orchestra.run.smithi204.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 431 kB/s | 309 kB 00:00 2024-04-02T14:29:00.073 INFO:teuthology.orchestra.run.smithi204.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-04-02T14:29:00.117 INFO:teuthology.orchestra.run.smithi204.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 408 kB/s | 49 kB 00:00 2024-04-02T14:29:00.184 INFO:teuthology.orchestra.run.smithi204.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 609 kB/s | 67 kB 00:00 2024-04-02T14:29:00.260 INFO:teuthology.orchestra.run.smithi204.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-02T14:29:00.426 INFO:teuthology.orchestra.run.smithi204.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-04-02T14:29:00.493 INFO:teuthology.orchestra.run.smithi204.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-04-02T14:29:00.535 INFO:teuthology.orchestra.run.smithi204.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-04-02T14:29:00.619 INFO:teuthology.orchestra.run.smithi204.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-04-02T14:29:00.652 INFO:teuthology.orchestra.run.smithi204.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-02T14:29:00.738 INFO:teuthology.orchestra.run.smithi204.stdout:(119/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.8 MB/s | 4.4 MB 00:00 2024-04-02T14:29:00.762 INFO:teuthology.orchestra.run.smithi204.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-04-02T14:29:00.787 INFO:teuthology.orchestra.run.smithi204.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-02T14:29:00.862 INFO:teuthology.orchestra.run.smithi204.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-02T14:29:00.887 INFO:teuthology.orchestra.run.smithi204.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 282 kB/s | 35 kB 00:00 2024-04-02T14:29:00.921 INFO:teuthology.orchestra.run.smithi204.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-04-02T14:29:01.038 INFO:teuthology.orchestra.run.smithi204.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 960 kB/s | 144 kB 00:00 2024-04-02T14:29:01.063 INFO:teuthology.orchestra.run.smithi204.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-04-02T14:29:01.191 INFO:teuthology.orchestra.run.smithi204.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.2 MB/s | 2.0 MB 00:00 2024-04-02T14:29:01.238 INFO:teuthology.orchestra.run.smithi204.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 88 kB/s | 18 kB 00:00 2024-04-02T14:29:01.264 INFO:teuthology.orchestra.run.smithi204.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 115 kB/s | 23 kB 00:00 2024-04-02T14:29:01.297 INFO:teuthology.orchestra.run.smithi204.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 183 kB/s | 19 kB 00:00 2024-04-02T14:29:01.373 INFO:teuthology.orchestra.run.smithi204.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-02T14:29:01.406 INFO:teuthology.orchestra.run.smithi204.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-04-02T14:29:01.465 INFO:teuthology.orchestra.run.smithi204.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-04-02T14:29:01.490 INFO:teuthology.orchestra.run.smithi204.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 672 kB/s | 79 kB 00:00 2024-04-02T14:29:01.515 INFO:teuthology.orchestra.run.smithi204.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-04-02T14:29:01.582 INFO:teuthology.orchestra.run.smithi204.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-02T14:29:01.607 INFO:teuthology.orchestra.run.smithi204.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-02T14:29:01.624 INFO:teuthology.orchestra.run.smithi204.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-04-02T14:29:01.691 INFO:teuthology.orchestra.run.smithi204.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-04-02T14:29:01.717 INFO:teuthology.orchestra.run.smithi204.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-02T14:29:01.742 INFO:teuthology.orchestra.run.smithi204.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-02T14:29:01.809 INFO:teuthology.orchestra.run.smithi204.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 504 kB/s | 59 kB 00:00 2024-04-02T14:29:01.834 INFO:teuthology.orchestra.run.smithi204.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 304 kB/s | 36 kB 00:00 2024-04-02T14:29:01.860 INFO:teuthology.orchestra.run.smithi204.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 435 kB/s | 51 kB 00:00 2024-04-02T14:29:01.927 INFO:teuthology.orchestra.run.smithi204.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-02T14:29:01.952 INFO:teuthology.orchestra.run.smithi204.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 765 kB/s | 90 kB 00:00 2024-04-02T14:29:01.977 INFO:teuthology.orchestra.run.smithi204.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-02T14:29:02.027 INFO:teuthology.orchestra.run.smithi204.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-02T14:29:02.069 INFO:teuthology.orchestra.run.smithi204.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-02T14:29:02.128 INFO:teuthology.orchestra.run.smithi204.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-02T14:29:02.220 INFO:teuthology.orchestra.run.smithi204.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-04-02T14:29:02.295 INFO:teuthology.orchestra.run.smithi204.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-02T14:29:02.337 INFO:teuthology.orchestra.run.smithi204.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 988 kB/s | 41 kB 00:00 2024-04-02T14:29:02.379 INFO:teuthology.orchestra.run.smithi204.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-04-02T14:29:02.454 INFO:teuthology.orchestra.run.smithi204.stdout:(155/174): librados2-19.0.0-2609.g191d963a.el9. 7.9 MB/s | 3.4 MB 00:00 2024-04-02T14:29:02.479 INFO:teuthology.orchestra.run.smithi204.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.1 MB/s | 321 kB 00:00 2024-04-02T14:29:02.547 INFO:teuthology.orchestra.run.smithi204.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 781 kB/s | 71 kB 00:00 2024-04-02T14:29:02.572 INFO:teuthology.orchestra.run.smithi204.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 396 kB/s | 36 kB 00:00 2024-04-02T14:29:02.631 INFO:teuthology.orchestra.run.smithi204.stdout:(159/174): librbd1-19.0.0-2609.g191d963a.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-02T14:29:02.656 INFO:teuthology.orchestra.run.smithi204.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-04-02T14:29:02.681 INFO:teuthology.orchestra.run.smithi204.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-04-02T14:29:02.706 INFO:teuthology.orchestra.run.smithi204.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 305 kB/s | 23 kB 00:00 2024-04-02T14:29:02.732 INFO:teuthology.orchestra.run.smithi204.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-04-02T14:29:02.757 INFO:teuthology.orchestra.run.smithi204.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-04-02T14:29:02.782 INFO:teuthology.orchestra.run.smithi204.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-04-02T14:29:02.788 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 758 kB/s | 19 MB 00:25 2024-04-02T14:29:02.807 INFO:teuthology.orchestra.run.smithi204.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 572 kB/s | 29 kB 00:00 2024-04-02T14:29:02.833 INFO:teuthology.orchestra.run.smithi204.stdout:(167/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-04-02T14:29:02.975 INFO:teuthology.orchestra.run.smithi204.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 13 MB/s | 2.2 MB 00:00 2024-04-02T14:29:03.008 INFO:teuthology.orchestra.run.smithi204.stdout:(169/174): openssl-3.0.7-27.el9.x86_64.rpm 5.4 MB/s | 1.2 MB 00:00 2024-04-02T14:29:03.042 INFO:teuthology.orchestra.run.smithi204.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.8 MB/s | 465 kB 00:00 2024-04-02T14:29:03.068 INFO:teuthology.orchestra.run.smithi204.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 529 kB/s | 31 kB 00:00 2024-04-02T14:29:03.118 INFO:teuthology.orchestra.run.smithi204.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 14 MB/s | 1.1 MB 00:00 2024-04-02T14:29:03.268 INFO:teuthology.orchestra.run.smithi204.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.3 MB/s | 2.3 MB 00:00 2024-04-02T14:29:03.385 INFO:teuthology.orchestra.run.smithi204.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 234 kB/s | 74 kB 00:00 2024-04-02T14:29:03.391 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-04-02T14:29:03.391 INFO:teuthology.orchestra.run.smithi204.stdout:Total 17 MB/s | 226 MB 00:13 2024-04-02T14:29:03.760 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-04-02T14:29:04.037 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-04-02T14:29:04.038 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-04-02T14:29:05.779 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-04-02T14:29:05.779 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-04-02T14:29:07.347 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-04-02T14:29:08.998 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-02T14:29:10.967 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-02T14:29:11.608 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 739 kB/s | 19 MB 00:26 2024-04-02T14:29:12.616 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-04-02T14:29:12.739 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T14:29:12.773 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T14:29:12.804 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T14:29:12.814 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T14:29:12.861 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T14:29:12.923 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T14:29:12.957 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T14:29:12.983 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T14:29:13.025 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T14:29:13.346 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T14:29:13.392 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T14:29:13.513 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T14:29:13.535 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T14:29:13.609 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T14:29:13.639 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T14:29:13.681 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T14:29:13.848 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T14:29:13.895 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T14:29:13.924 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T14:29:14.019 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T14:29:14.042 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T14:29:14.089 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T14:29:14.129 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T14:29:14.152 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T14:29:14.167 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T14:29:14.352 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T14:29:14.383 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T14:29:14.415 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T14:29:14.524 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T14:29:14.617 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T14:29:14.687 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T14:29:14.768 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T14:29:14.812 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T14:29:15.038 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T14:29:15.145 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T14:29:15.238 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T14:29:15.275 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T14:29:15.302 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T14:29:15.337 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T14:29:15.412 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T14:29:15.492 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T14:29:15.529 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T14:29:15.593 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T14:29:15.628 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T14:29:15.672 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T14:29:15.704 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T14:29:15.732 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T14:29:15.802 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T14:29:15.837 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T14:29:15.880 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T14:29:15.909 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T14:29:15.977 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T14:29:16.032 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T14:29:16.114 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T14:29:16.248 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T14:29:16.284 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.1 MB 00:01 2024-04-02T14:29:16.301 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T14:29:16.825 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T14:29:16.905 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T14:29:16.980 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T14:29:17.025 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T14:29:17.033 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-02T14:29:17.034 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T14:29:17.065 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T14:29:17.114 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T14:29:17.152 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T14:29:17.182 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T14:29:17.426 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T14:29:17.605 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T14:29:17.668 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T14:29:17.712 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T14:29:17.734 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-04-02T14:29:18.091 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T14:29:18.150 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T14:29:18.212 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T14:29:18.283 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T14:29:18.514 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T14:29:18.659 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T14:29:18.744 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T14:29:18.780 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T14:29:18.823 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T14:29:18.865 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T14:29:18.998 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T14:29:19.071 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T14:29:19.101 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T14:29:19.134 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T14:29:19.228 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T14:29:19.268 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T14:29:19.307 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T14:29:19.333 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T14:29:19.358 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T14:29:19.430 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T14:29:19.567 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T14:29:19.675 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T14:29:19.717 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T14:29:19.765 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T14:29:19.829 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T14:29:19.911 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T14:29:19.912 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T14:29:19.935 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T14:29:19.936 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T14:29:19.938 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T14:29:19.940 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T14:29:19.942 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T14:29:19.946 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T14:29:19.948 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T14:29:20.013 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T14:29:20.042 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-02T14:29:20.062 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T14:29:20.075 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-04-02T14:29:20.075 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-04-02T14:29:20.076 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-04-02T14:29:20.076 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-04-02T14:29:20.076 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T14:29:20.076 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T14:29:20.076 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T14:29:20.076 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T14:29:20.076 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T14:29:20.076 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T14:29:20.077 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T14:29:20.077 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T14:29:20.077 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T14:29:20.077 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T14:29:20.077 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T14:29:20.077 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T14:29:20.077 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T14:29:20.077 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T14:29:20.077 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T14:29:20.077 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T14:29:20.077 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T14:29:20.078 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T14:29:20.078 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T14:29:20.078 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T14:29:20.078 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T14:29:20.078 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T14:29:20.078 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T14:29:20.078 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T14:29:20.078 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T14:29:20.079 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T14:29:20.079 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T14:29:20.079 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T14:29:20.079 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T14:29:20.079 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T14:29:20.079 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T14:29:20.079 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T14:29:20.080 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T14:29:20.080 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T14:29:20.080 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T14:29:20.080 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T14:29:20.080 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T14:29:20.080 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-04-02T14:29:20.080 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T14:29:20.080 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T14:29:20.080 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T14:29:20.081 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T14:29:20.081 INFO:teuthology.orchestra.run.smithi062.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T14:29:20.081 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T14:29:20.081 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T14:29:20.081 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T14:29:20.081 INFO:teuthology.orchestra.run.smithi062.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T14:29:20.081 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T14:29:20.082 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T14:29:20.082 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T14:29:20.082 INFO:teuthology.orchestra.run.smithi062.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T14:29:20.082 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T14:29:20.082 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T14:29:20.082 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T14:29:20.082 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T14:29:20.082 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T14:29:20.083 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T14:29:20.083 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T14:29:20.083 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T14:29:20.083 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T14:29:20.083 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T14:29:20.083 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T14:29:20.083 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-04-02T14:29:20.083 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T14:29:20.084 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T14:29:20.084 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T14:29:20.084 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T14:29:20.084 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T14:29:20.084 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T14:29:20.084 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T14:29:20.084 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T14:29:20.085 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T14:29:20.085 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T14:29:20.085 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T14:29:20.085 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T14:29:20.085 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T14:29:20.085 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T14:29:20.085 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T14:29:20.085 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T14:29:20.085 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T14:29:20.085 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T14:29:20.086 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T14:29:20.086 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T14:29:20.086 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T14:29:20.086 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T14:29:20.086 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T14:29:20.087 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T14:29:20.087 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T14:29:20.087 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T14:29:20.087 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T14:29:20.087 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T14:29:20.087 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T14:29:20.088 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T14:29:20.088 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T14:29:20.088 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T14:29:20.088 INFO:teuthology.orchestra.run.smithi062.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T14:29:20.088 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T14:29:20.088 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T14:29:20.088 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T14:29:20.088 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T14:29:20.088 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T14:29:20.088 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T14:29:20.089 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T14:29:20.089 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T14:29:20.089 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T14:29:20.089 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T14:29:20.089 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T14:29:20.089 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T14:29:20.089 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T14:29:20.089 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T14:29:20.089 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T14:29:20.090 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T14:29:20.090 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T14:29:20.090 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T14:29:20.090 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T14:29:20.090 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T14:29:20.090 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T14:29:20.090 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T14:29:20.090 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T14:29:20.091 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T14:29:20.091 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T14:29:20.091 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T14:29:20.091 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T14:29:20.091 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T14:29:20.091 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T14:29:20.091 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T14:29:20.092 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T14:29:20.092 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T14:29:20.092 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T14:29:20.092 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T14:29:20.092 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T14:29:20.092 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T14:29:20.092 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T14:29:20.092 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T14:29:20.092 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T14:29:20.092 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T14:29:20.092 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T14:29:20.093 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T14:29:20.093 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T14:29:20.093 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T14:29:20.093 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T14:29:20.093 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T14:29:20.093 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T14:29:20.093 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T14:29:20.093 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T14:29:20.093 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T14:29:20.093 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T14:29:20.093 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T14:29:20.094 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T14:29:20.094 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T14:29:20.094 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T14:29:20.094 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T14:29:20.094 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T14:29:20.094 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T14:29:20.094 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T14:29:20.094 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T14:29:20.094 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T14:29:20.095 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T14:29:20.096 INFO:teuthology.orchestra.run.smithi062.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T14:29:20.096 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T14:29:20.096 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T14:29:20.096 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T14:29:20.096 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T14:29:20.096 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T14:29:20.096 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-04-02T14:29:20.096 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T14:29:20.097 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:29:20.097 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-04-02T14:29:20.097 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-04-02T14:29:20.097 INFO:teuthology.orchestra.run.smithi062.stdout:Install 150 Packages 2024-04-02T14:29:20.097 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 24 Packages 2024-04-02T14:29:20.097 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:29:20.098 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 226 M 2024-04-02T14:29:20.098 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-04-02T14:29:20.191 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T14:29:20.228 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T14:29:20.260 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T14:29:20.291 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T14:29:20.319 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T14:29:20.353 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T14:29:20.390 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 8.2 MB/s | 21 MB 00:02 2024-04-02T14:29:20.400 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T14:29:20.432 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T14:29:20.462 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T14:29:20.538 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T14:29:20.572 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T14:29:20.606 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T14:29:20.647 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T14:29:20.682 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T14:29:20.778 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T14:29:20.861 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T14:29:20.904 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T14:29:20.936 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T14:29:20.968 INFO:teuthology.orchestra.run.smithi062.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-02T14:29:20.983 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T14:29:21.080 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T14:29:21.129 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T14:29:21.385 INFO:teuthology.orchestra.run.smithi062.stdout:(2/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 7.0 MB/s | 5.4 MB 00:00 2024-04-02T14:29:21.452 INFO:teuthology.orchestra.run.smithi062.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 2.2 MB/s | 145 kB 00:00 2024-04-02T14:29:21.752 INFO:teuthology.orchestra.run.smithi062.stdout:(4/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 7.9 MB/s | 2.4 MB 00:00 2024-04-02T14:29:21.836 INFO:teuthology.orchestra.run.smithi062.stdout:(5/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 1.0 MB/s | 890 kB 00:00 2024-04-02T14:29:21.867 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T14:29:21.918 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T14:29:21.936 INFO:teuthology.orchestra.run.smithi062.stdout:(6/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-04-02T14:29:21.952 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T14:29:22.000 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T14:29:22.079 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T14:29:22.111 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T14:29:22.290 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T14:29:22.317 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T14:29:22.361 INFO:teuthology.orchestra.run.smithi062.stdout:(7/174): ceph-common-19.0.0-2609.g191d963a.el9. 12 MB/s | 21 MB 00:01 2024-04-02T14:29:22.395 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T14:29:23.039 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T14:29:23.156 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T14:29:23.240 INFO:teuthology.orchestra.run.smithi062.stdout:(8/174): ceph-radosgw-19.0.0-2609.g191d963a.el9 11 MB/s | 9.8 MB 00:00 2024-04-02T14:29:23.295 INFO:teuthology.orchestra.run.smithi062.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 471 kB/s | 25 kB 00:00 2024-04-02T14:29:23.570 INFO:teuthology.orchestra.run.smithi062.stdout:(10/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 10 MB/s | 17 MB 00:01 2024-04-02T14:29:23.629 INFO:teuthology.orchestra.run.smithi062.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 3.8 MB/s | 225 kB 00:00 2024-04-02T14:29:23.737 INFO:teuthology.orchestra.run.smithi062.stdout:(12/174): ceph-mon-19.0.0-2609.g191d963a.el9.x8 2.5 MB/s | 4.7 MB 00:01 2024-04-02T14:29:23.763 INFO:teuthology.orchestra.run.smithi062.stdout:(13/174): cephfs-shell-19.0.0-2609.g191d963a.el 204 kB/s | 27 kB 00:00 2024-04-02T14:29:23.805 INFO:teuthology.orchestra.run.smithi062.stdout:(14/174): libcephfs-devel-19.0.0-2609.g191d963a 480 kB/s | 32 kB 00:00 2024-04-02T14:29:23.872 INFO:teuthology.orchestra.run.smithi062.stdout:(15/174): libcephfs2-19.0.0-2609.g191d963a.el9. 6.8 MB/s | 747 kB 00:00 2024-04-02T14:29:23.905 INFO:teuthology.orchestra.run.smithi062.stdout:(16/174): libcephsqlite-19.0.0-2609.g191d963a.e 1.6 MB/s | 164 kB 00:00 2024-04-02T14:29:23.939 INFO:teuthology.orchestra.run.smithi062.stdout:(17/174): librados-devel-19.0.0-2609.g191d963a. 1.9 MB/s | 127 kB 00:00 2024-04-02T14:29:24.048 INFO:teuthology.orchestra.run.smithi062.stdout:(18/174): libradosstriper1-19.0.0-2609.g191d963 3.5 MB/s | 500 kB 00:00 2024-04-02T14:29:24.165 INFO:teuthology.orchestra.run.smithi062.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 389 kB/s | 45 kB 00:00 2024-04-02T14:29:24.307 INFO:teuthology.orchestra.run.smithi062.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 957 kB/s | 134 kB 00:00 2024-04-02T14:29:24.361 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T14:29:24.443 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T14:29:24.449 INFO:teuthology.orchestra.run.smithi062.stdout:(21/174): librgw2-19.0.0-2609.g191d963a.el9.x86 10 MB/s | 5.1 MB 00:00 2024-04-02T14:29:24.474 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T14:29:24.490 INFO:teuthology.orchestra.run.smithi062.stdout:(22/174): python3-cephfs-19.0.0-2609.g191d963a. 884 kB/s | 161 kB 00:00 2024-04-02T14:29:24.505 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T14:29:24.532 INFO:teuthology.orchestra.run.smithi062.stdout:(23/174): python3-rados-19.0.0-2609.g191d963a.e 3.8 MB/s | 320 kB 00:00 2024-04-02T14:29:24.533 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T14:29:24.575 INFO:teuthology.orchestra.run.smithi062.stdout:(24/174): python3-rbd-19.0.0-2609.g191d963a.el9 3.5 MB/s | 299 kB 00:00 2024-04-02T14:29:24.608 INFO:teuthology.orchestra.run.smithi062.stdout:(25/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.3 MB/s | 99 kB 00:00 2024-04-02T14:29:24.642 INFO:teuthology.orchestra.run.smithi062.stdout:(26/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-02T14:29:24.775 INFO:teuthology.orchestra.run.smithi062.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 1.3 MB/s | 170 kB 00:00 2024-04-02T14:29:24.845 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T14:29:24.880 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T14:29:24.909 INFO:teuthology.orchestra.run.smithi062.stdout:(28/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 10 MB/s | 3.1 MB 00:00 2024-04-02T14:29:24.926 INFO:teuthology.orchestra.run.smithi062.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2609.g 227 kB/s | 34 kB 00:00 2024-04-02T14:29:24.972 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T14:29:24.984 INFO:teuthology.orchestra.run.smithi062.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 1.9 MB/s | 141 kB 00:00 2024-04-02T14:29:25.002 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T14:29:25.038 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T14:29:25.411 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-02T14:29:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T14:29:25.533 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T14:29:25.604 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T14:29:25.627 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T14:29:25.736 INFO:teuthology.orchestra.run.smithi062.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 9.9 MB/s | 7.4 MB 00:00 2024-04-02T14:29:25.843 INFO:teuthology.orchestra.run.smithi062.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 3.9 MB/s | 3.5 MB 00:00 2024-04-02T14:29:25.877 INFO:teuthology.orchestra.run.smithi062.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2609.g19 1.7 MB/s | 249 kB 00:00 2024-04-02T14:29:25.952 INFO:teuthology.orchestra.run.smithi062.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 222 kB/s | 16 kB 00:00 2024-04-02T14:29:25.977 INFO:teuthology.orchestra.run.smithi062.stdout:(35/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 371 kB/s | 49 kB 00:00 2024-04-02T14:29:26.011 INFO:teuthology.orchestra.run.smithi062.stdout:(36/174): ceph-volume-19.0.0-2609.g191d963a.el9 4.5 MB/s | 264 kB 00:00 2024-04-02T14:29:26.061 INFO:teuthology.orchestra.run.smithi062.stdout:(37/174): cephfs-top-19.0.0-2609.g191d963a.el9. 370 kB/s | 18 kB 00:00 2024-04-02T14:29:26.245 INFO:teuthology.orchestra.run.smithi062.stdout:(38/174): cephadm-19.0.0-2609.g191d963a.el9.noa 2.8 MB/s | 762 kB 00:00 2024-04-02T14:29:26.304 INFO:teuthology.orchestra.run.smithi062.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 662 kB 00:00 2024-04-02T14:29:26.346 INFO:teuthology.orchestra.run.smithi062.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 9.1 MB/s | 920 kB 00:00 2024-04-02T14:29:26.371 INFO:teuthology.orchestra.run.smithi062.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.7 MB/s | 106 kB 00:00 2024-04-02T14:29:26.404 INFO:teuthology.orchestra.run.smithi062.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-04-02T14:29:26.430 INFO:teuthology.orchestra.run.smithi062.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 581 kB/s | 32 kB 00:00 2024-04-02T14:29:26.580 INFO:teuthology.orchestra.run.smithi062.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-04-02T14:29:26.605 INFO:teuthology.orchestra.run.smithi062.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 111 kB/s | 17 kB 00:00 2024-04-02T14:29:26.631 INFO:teuthology.orchestra.run.smithi062.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 203 kB/s | 10 kB 00:00 2024-04-02T14:29:26.665 INFO:teuthology.orchestra.run.smithi062.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-04-02T14:29:26.690 INFO:teuthology.orchestra.run.smithi062.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 786 kB/s | 44 kB 00:00 2024-04-02T14:29:26.732 INFO:teuthology.orchestra.run.smithi062.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 661 kB 00:00 2024-04-02T14:29:26.774 INFO:teuthology.orchestra.run.smithi062.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.4 MB/s | 578 kB 00:00 2024-04-02T14:29:26.799 INFO:teuthology.orchestra.run.smithi062.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 763 kB/s | 46 kB 00:00 2024-04-02T14:29:26.866 INFO:teuthology.orchestra.run.smithi062.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-04-02T14:29:26.900 INFO:teuthology.orchestra.run.smithi062.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 1.9 MB/s | 187 kB 00:00 2024-04-02T14:29:26.933 INFO:teuthology.orchestra.run.smithi062.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-04-02T14:29:26.960 INFO:teuthology.orchestra.run.smithi062.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 612 kB/s | 34 kB 00:00 2024-04-02T14:29:27.534 INFO:teuthology.orchestra.run.smithi062.stdout:(56/174): ceph-test-19.0.0-2609.g191d963a.el9.x 11 MB/s | 48 MB 00:04 2024-04-02T14:29:27.651 INFO:teuthology.orchestra.run.smithi062.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 6.6 MB/s | 4.7 MB 00:00 2024-04-02T14:29:27.709 INFO:teuthology.orchestra.run.smithi062.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 2.1 MB/s | 1.5 MB 00:00 2024-04-02T14:29:27.743 INFO:teuthology.orchestra.run.smithi062.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 4.2 MB/s | 817 kB 00:00 2024-04-02T14:29:27.768 INFO:teuthology.orchestra.run.smithi062.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 132 kB 00:00 2024-04-02T14:29:27.883 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T14:29:27.915 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T14:29:27.926 INFO:teuthology.orchestra.run.smithi113.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T14:29:27.927 INFO:teuthology.orchestra.run.smithi113.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T14:29:27.927 INFO:teuthology.orchestra.run.smithi062.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.6 MB/s | 1.0 MB 00:00 2024-04-02T14:29:27.942 INFO:teuthology.orchestra.run.smithi113.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T14:29:27.943 INFO:teuthology.orchestra.run.smithi113.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T14:29:27.944 INFO:teuthology.orchestra.run.smithi113.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T14:29:27.946 INFO:teuthology.orchestra.run.smithi113.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T14:29:27.948 INFO:teuthology.orchestra.run.smithi113.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T14:29:27.951 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T14:29:27.953 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T14:29:28.036 INFO:teuthology.orchestra.run.smithi062.stdout:(62/174): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.7 MB 00:00 2024-04-02T14:29:28.039 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-02T14:29:28.057 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================================ 2024-04-02T14:29:28.057 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-02T14:29:28.057 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================================ 2024-04-02T14:29:28.057 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T14:29:28.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T14:29:28.059 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T14:29:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T14:29:28.061 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T14:29:28.062 INFO:teuthology.orchestra.run.smithi113.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T14:29:28.062 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T14:29:28.062 INFO:teuthology.orchestra.run.smithi113.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T14:29:28.062 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T14:29:28.062 INFO:teuthology.orchestra.run.smithi113.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T14:29:28.062 INFO:teuthology.orchestra.run.smithi113.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T14:29:28.062 INFO:teuthology.orchestra.run.smithi113.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T14:29:28.062 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T14:29:28.062 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T14:29:28.062 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-02T14:29:28.062 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T14:29:28.063 INFO:teuthology.orchestra.run.smithi113.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T14:29:28.063 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T14:29:28.063 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T14:29:28.063 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T14:29:28.063 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T14:29:28.063 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T14:29:28.063 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T14:29:28.064 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T14:29:28.064 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T14:29:28.064 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T14:29:28.064 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T14:29:28.064 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T14:29:28.064 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T14:29:28.065 INFO:teuthology.orchestra.run.smithi113.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T14:29:28.065 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T14:29:28.065 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T14:29:28.065 INFO:teuthology.orchestra.run.smithi113.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T14:29:28.065 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T14:29:28.065 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T14:29:28.066 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T14:29:28.066 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T14:29:28.066 INFO:teuthology.orchestra.run.smithi113.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T14:29:28.066 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T14:29:28.066 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T14:29:28.067 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T14:29:28.067 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T14:29:28.067 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T14:29:28.067 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T14:29:28.067 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T14:29:28.067 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T14:29:28.068 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T14:29:28.068 INFO:teuthology.orchestra.run.smithi113.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T14:29:28.068 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T14:29:28.068 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T14:29:28.068 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T14:29:28.069 INFO:teuthology.orchestra.run.smithi113.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T14:29:28.069 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T14:29:28.069 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T14:29:28.069 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T14:29:28.069 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T14:29:28.069 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T14:29:28.070 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T14:29:28.070 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T14:29:28.070 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T14:29:28.070 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T14:29:28.070 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T14:29:28.070 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T14:29:28.070 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T14:29:28.070 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T14:29:28.071 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T14:29:28.071 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T14:29:28.071 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T14:29:28.071 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T14:29:28.071 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T14:29:28.071 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T14:29:28.071 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T14:29:28.071 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T14:29:28.071 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T14:29:28.072 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T14:29:28.072 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T14:29:28.072 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T14:29:28.072 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T14:29:28.072 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T14:29:28.072 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T14:29:28.072 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T14:29:28.072 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T14:29:28.073 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T14:29:28.073 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T14:29:28.073 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T14:29:28.073 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T14:29:28.073 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T14:29:28.073 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T14:29:28.073 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T14:29:28.074 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T14:29:28.074 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T14:29:28.074 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T14:29:28.074 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T14:29:28.074 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T14:29:28.074 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T14:29:28.075 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T14:29:28.075 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T14:29:28.075 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T14:29:28.075 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T14:29:28.075 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T14:29:28.075 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T14:29:28.075 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T14:29:28.075 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T14:29:28.076 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T14:29:28.076 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T14:29:28.076 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T14:29:28.076 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T14:29:28.076 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T14:29:28.076 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T14:29:28.076 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T14:29:28.076 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T14:29:28.077 INFO:teuthology.orchestra.run.smithi113.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T14:29:28.078 INFO:teuthology.orchestra.run.smithi113.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T14:29:28.078 INFO:teuthology.orchestra.run.smithi113.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T14:29:28.078 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T14:29:28.078 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T14:29:28.078 INFO:teuthology.orchestra.run.smithi113.stdout:Installing weak dependencies: 2024-04-02T14:29:28.078 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T14:29:28.078 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:29:28.078 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-02T14:29:28.078 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================================ 2024-04-02T14:29:28.078 INFO:teuthology.orchestra.run.smithi113.stdout:Install 150 Packages 2024-04-02T14:29:28.078 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 24 Packages 2024-04-02T14:29:28.079 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:29:28.079 INFO:teuthology.orchestra.run.smithi062.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 272 kB/s | 78 kB 00:00 2024-04-02T14:29:28.080 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 226 M 2024-04-02T14:29:28.080 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-02T14:29:28.236 INFO:teuthology.orchestra.run.smithi062.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-04-02T14:29:28.262 INFO:teuthology.orchestra.run.smithi062.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 139 kB/s | 24 kB 00:00 2024-04-02T14:29:28.286 INFO:teuthology.orchestra.run.smithi062.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 90 kB/s | 18 kB 00:00 2024-04-02T14:29:28.337 INFO:teuthology.orchestra.run.smithi062.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2024-04-02T14:29:28.362 INFO:teuthology.orchestra.run.smithi062.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 429 kB/s | 32 kB 00:00 2024-04-02T14:29:28.453 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T14:29:28.494 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T14:29:28.504 INFO:teuthology.orchestra.run.smithi062.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 4.8 MB 00:00 2024-04-02T14:29:28.529 INFO:teuthology.orchestra.run.smithi062.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-04-02T14:29:28.554 INFO:teuthology.orchestra.run.smithi062.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-04-02T14:29:28.558 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T14:29:28.558 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T14:29:28.558 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T14:29:28.558 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:29:28.595 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T14:29:28.596 INFO:teuthology.orchestra.run.smithi062.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 69 kB 00:00 2024-04-02T14:29:28.621 INFO:teuthology.orchestra.run.smithi062.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 627 kB/s | 41 kB 00:00 2024-04-02T14:29:28.647 INFO:teuthology.orchestra.run.smithi062.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 238 kB 00:00 2024-04-02T14:29:28.701 INFO:teuthology.orchestra.run.smithi062.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-02T14:29:28.739 INFO:teuthology.orchestra.run.smithi062.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 211 kB 00:00 2024-04-02T14:29:28.764 INFO:teuthology.orchestra.run.smithi062.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 412 kB/s | 47 kB 00:00 2024-04-02T14:29:28.798 INFO:teuthology.orchestra.run.smithi062.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 741 kB/s | 43 kB 00:00 2024-04-02T14:29:28.870 INFO:teuthology.orchestra.run.smithi113.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-02T14:29:29.004 INFO:teuthology.orchestra.run.smithi062.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 165 kB/s | 39 kB 00:00 2024-04-02T14:29:29.115 INFO:teuthology.orchestra.run.smithi062.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 132 kB/s | 41 kB 00:00 2024-04-02T14:29:29.304 INFO:teuthology.orchestra.run.smithi113.stdout:(2/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 7.9 MB/s | 5.4 MB 00:00 2024-04-02T14:29:29.366 INFO:teuthology.orchestra.run.smithi062.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-04-02T14:29:29.379 INFO:teuthology.orchestra.run.smithi113.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 1.9 MB/s | 145 kB 00:00 2024-04-02T14:29:29.399 INFO:teuthology.orchestra.run.smithi062.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 576 kB/s | 220 kB 00:00 2024-04-02T14:29:29.433 INFO:teuthology.orchestra.run.smithi062.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 114 kB/s | 35 kB 00:00 2024-04-02T14:29:29.466 INFO:teuthology.orchestra.run.smithi062.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 46 kB 00:00 2024-04-02T14:29:29.491 INFO:teuthology.orchestra.run.smithi062.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 300 kB 00:00 2024-04-02T14:29:29.517 INFO:teuthology.orchestra.run.smithi062.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 708 kB/s | 56 kB 00:00 2024-04-02T14:29:29.542 INFO:teuthology.orchestra.run.smithi062.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-04-02T14:29:29.567 INFO:teuthology.orchestra.run.smithi062.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 598 kB/s | 44 kB 00:00 2024-04-02T14:29:29.592 INFO:teuthology.orchestra.run.smithi062.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-02T14:29:29.655 INFO:teuthology.orchestra.run.smithi113.stdout:(4/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-04-02T14:29:29.659 INFO:teuthology.orchestra.run.smithi062.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-04-02T14:29:29.710 INFO:teuthology.orchestra.run.smithi062.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.1 MB 00:00 2024-04-02T14:29:29.735 INFO:teuthology.orchestra.run.smithi062.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 772 kB/s | 103 kB 00:00 2024-04-02T14:29:29.769 INFO:teuthology.orchestra.run.smithi062.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-04-02T14:29:29.802 INFO:teuthology.orchestra.run.smithi062.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 187 kB 00:00 2024-04-02T14:29:29.821 INFO:teuthology.orchestra.run.smithi113.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 9.0 MB/s | 1.5 MB 00:00 2024-04-02T14:29:29.871 INFO:teuthology.orchestra.run.smithi113.stdout:(6/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 890 kB/s | 890 kB 00:01 2024-04-02T14:29:29.952 INFO:teuthology.orchestra.run.smithi062.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 874 kB/s | 187 kB 00:00 2024-04-02T14:29:30.011 INFO:teuthology.orchestra.run.smithi062.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 941 kB/s | 195 kB 00:00 2024-04-02T14:29:30.045 INFO:teuthology.orchestra.run.smithi062.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-02T14:29:30.112 INFO:teuthology.orchestra.run.smithi062.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-04-02T14:29:30.137 INFO:teuthology.orchestra.run.smithi062.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 172 kB/s | 21 kB 00:00 2024-04-02T14:29:30.154 INFO:teuthology.orchestra.run.smithi062.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 236 kB/s | 25 kB 00:00 2024-04-02T14:29:30.230 INFO:teuthology.orchestra.run.smithi113.stdout:(7/174): ceph-common-19.0.0-2609.g191d963a.el9. 13 MB/s | 21 MB 00:01 2024-04-02T14:29:30.254 INFO:teuthology.orchestra.run.smithi062.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.1 MB/s | 245 kB 00:00 2024-04-02T14:29:30.304 INFO:teuthology.orchestra.run.smithi062.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-04-02T14:29:30.305 INFO:teuthology.orchestra.run.smithi113.stdout:(8/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 9.7 MB/s | 4.7 MB 00:00 2024-04-02T14:29:30.338 INFO:teuthology.orchestra.run.smithi062.stdout:(103/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.3 MB/s | 248 kB 00:00 2024-04-02T14:29:30.356 INFO:teuthology.orchestra.run.smithi113.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 506 kB/s | 25 kB 00:00 2024-04-02T14:29:30.455 INFO:teuthology.orchestra.run.smithi062.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 12 MB/s | 4.1 MB 00:00 2024-04-02T14:29:30.639 INFO:teuthology.orchestra.run.smithi062.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 67 kB/s | 22 kB 00:00 2024-04-02T14:29:30.756 INFO:teuthology.orchestra.run.smithi062.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-02T14:29:30.831 INFO:teuthology.orchestra.run.smithi062.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 263 kB/s | 19 kB 00:00 2024-04-02T14:29:30.898 INFO:teuthology.orchestra.run.smithi062.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-02T14:29:30.947 INFO:teuthology.orchestra.run.smithi113.stdout:(10/174): ceph-radosgw-19.0.0-2609.g191d963a.el 14 MB/s | 9.8 MB 00:00 2024-04-02T14:29:30.957 INFO:teuthology.orchestra.run.smithi062.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 450 kB/s | 26 kB 00:00 2024-04-02T14:29:31.014 INFO:teuthology.orchestra.run.smithi113.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 3.3 MB/s | 225 kB 00:00 2024-04-02T14:29:31.024 INFO:teuthology.orchestra.run.smithi062.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 738 kB/s | 49 kB 00:00 2024-04-02T14:29:31.073 INFO:teuthology.orchestra.run.smithi113.stdout:(12/174): cephfs-shell-19.0.0-2609.g191d963a.el 464 kB/s | 27 kB 00:00 2024-04-02T14:29:31.123 INFO:teuthology.orchestra.run.smithi113.stdout:(13/174): libcephfs-devel-19.0.0-2609.g191d963a 635 kB/s | 32 kB 00:00 2024-04-02T14:29:31.149 INFO:teuthology.orchestra.run.smithi062.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-02T14:29:31.174 INFO:teuthology.orchestra.run.smithi062.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 451 kB/s | 67 kB 00:00 2024-04-02T14:29:31.223 INFO:teuthology.orchestra.run.smithi113.stdout:(14/174): libcephfs2-19.0.0-2609.g191d963a.el9. 7.3 MB/s | 747 kB 00:00 2024-04-02T14:29:31.224 INFO:teuthology.orchestra.run.smithi062.stdout:(113/174): protobuf-compiler-3.14.0-13.el9.x86_ 975 kB/s | 863 kB 00:00 2024-04-02T14:29:31.258 INFO:teuthology.orchestra.run.smithi062.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-02T14:29:31.299 INFO:teuthology.orchestra.run.smithi113.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 2.1 MB/s | 164 kB 00:00 2024-04-02T14:29:31.320 INFO:teuthology.orchestra.run.smithi062.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 838 kB 00:00 2024-04-02T14:29:31.342 INFO:teuthology.orchestra.run.smithi062.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 530 kB/s | 43 kB 00:00 2024-04-02T14:29:31.366 INFO:teuthology.orchestra.run.smithi113.stdout:(16/174): librados-devel-19.0.0-2609.g191d963a. 1.9 MB/s | 127 kB 00:00 2024-04-02T14:29:31.375 INFO:teuthology.orchestra.run.smithi062.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 558 kB/s | 32 kB 00:00 2024-04-02T14:29:31.430 INFO:teuthology.orchestra.run.smithi062.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 172 kB/s | 15 kB 00:00 2024-04-02T14:29:31.450 INFO:teuthology.orchestra.run.smithi113.stdout:(17/174): libradosstriper1-19.0.0-2609.g191d963 5.9 MB/s | 500 kB 00:00 2024-04-02T14:29:31.470 INFO:teuthology.orchestra.run.smithi062.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-02T14:29:31.518 INFO:teuthology.orchestra.run.smithi062.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.0 MB/s | 358 kB 00:00 2024-04-02T14:29:31.560 INFO:teuthology.orchestra.run.smithi062.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-02T14:29:31.594 INFO:teuthology.orchestra.run.smithi062.stdout:(122/174): python3-asyncssh-2.13.2-5.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-04-02T14:29:31.619 INFO:teuthology.orchestra.run.smithi062.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-04-02T14:29:31.652 INFO:teuthology.orchestra.run.smithi062.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-04-02T14:29:31.686 INFO:teuthology.orchestra.run.smithi062.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-02T14:29:31.733 INFO:teuthology.orchestra.run.smithi062.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 134 kB/s | 11 kB 00:00 2024-04-02T14:29:31.762 INFO:teuthology.orchestra.run.smithi062.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 238 kB/s | 18 kB 00:00 2024-04-02T14:29:31.787 INFO:teuthology.orchestra.run.smithi062.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 440 kB/s | 23 kB 00:00 2024-04-02T14:29:31.820 INFO:teuthology.orchestra.run.smithi062.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 335 kB/s | 19 kB 00:00 2024-04-02T14:29:31.846 INFO:teuthology.orchestra.run.smithi062.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 339 kB/s | 20 kB 00:00 2024-04-02T14:29:31.875 INFO:teuthology.orchestra.run.smithi113.stdout:(18/174): librgw2-19.0.0-2609.g191d963a.el9.x86 12 MB/s | 5.1 MB 00:00 2024-04-02T14:29:31.921 INFO:teuthology.orchestra.run.smithi062.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-02T14:29:31.925 INFO:teuthology.orchestra.run.smithi113.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 909 kB/s | 45 kB 00:00 2024-04-02T14:29:31.988 INFO:teuthology.orchestra.run.smithi062.stdout:(132/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.2 MB/s | 2.0 MB 00:00 2024-04-02T14:29:31.992 INFO:teuthology.orchestra.run.smithi113.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 2.0 MB/s | 134 kB 00:00 2024-04-02T14:29:32.036 INFO:teuthology.orchestra.run.smithi062.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 244 kB/s | 46 kB 00:00 2024-04-02T14:29:32.059 INFO:teuthology.orchestra.run.smithi113.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 2.4 MB/s | 161 kB 00:00 2024-04-02T14:29:32.081 INFO:teuthology.orchestra.run.smithi062.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 495 kB/s | 79 kB 00:00 2024-04-02T14:29:32.134 INFO:teuthology.orchestra.run.smithi113.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 4.2 MB/s | 320 kB 00:00 2024-04-02T14:29:32.210 INFO:teuthology.orchestra.run.smithi113.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 3.9 MB/s | 299 kB 00:00 2024-04-02T14:29:32.214 INFO:teuthology.orchestra.run.smithi062.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 256 kB/s | 58 kB 00:00 2024-04-02T14:29:32.248 INFO:teuthology.orchestra.run.smithi062.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-02T14:29:32.265 INFO:teuthology.orchestra.run.smithi062.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 333 kB/s | 16 kB 00:00 2024-04-02T14:29:32.277 INFO:teuthology.orchestra.run.smithi113.stdout:(24/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.5 MB/s | 99 kB 00:00 2024-04-02T14:29:32.299 INFO:teuthology.orchestra.run.smithi062.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-02T14:29:32.341 INFO:teuthology.orchestra.run.smithi062.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 308 kB/s | 28 kB 00:00 2024-04-02T14:29:32.344 INFO:teuthology.orchestra.run.smithi113.stdout:(25/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-02T14:29:32.358 INFO:teuthology.orchestra.run.smithi062.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-04-02T14:29:32.391 INFO:teuthology.orchestra.run.smithi062.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-02T14:29:32.409 INFO:teuthology.orchestra.run.smithi062.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 731 kB/s | 36 kB 00:00 2024-04-02T14:29:32.434 INFO:teuthology.orchestra.run.smithi062.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 794 kB/s | 59 kB 00:00 2024-04-02T14:29:32.459 INFO:teuthology.orchestra.run.smithi062.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 768 kB/s | 51 kB 00:00 2024-04-02T14:29:32.493 INFO:teuthology.orchestra.run.smithi062.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-02T14:29:32.518 INFO:teuthology.orchestra.run.smithi062.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-02T14:29:32.552 INFO:teuthology.orchestra.run.smithi062.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-02T14:29:32.577 INFO:teuthology.orchestra.run.smithi062.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-04-02T14:29:32.619 INFO:teuthology.orchestra.run.smithi113.stdout:(26/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 11 MB/s | 3.1 MB 00:00 2024-04-02T14:29:32.635 INFO:teuthology.orchestra.run.smithi062.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-02T14:29:32.686 INFO:teuthology.orchestra.run.smithi113.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-02T14:29:32.745 INFO:teuthology.orchestra.run.smithi113.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 581 kB/s | 34 kB 00:00 2024-04-02T14:29:32.761 INFO:teuthology.orchestra.run.smithi062.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-02T14:29:32.812 INFO:teuthology.orchestra.run.smithi113.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 2.1 MB/s | 141 kB 00:00 2024-04-02T14:29:33.178 INFO:teuthology.orchestra.run.smithi062.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 6.4 MB/s | 2.3 MB 00:00 2024-04-02T14:29:33.270 INFO:teuthology.orchestra.run.smithi062.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-04-02T14:29:33.295 INFO:teuthology.orchestra.run.smithi062.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.7 MB/s | 41 kB 00:00 2024-04-02T14:29:33.314 INFO:teuthology.orchestra.run.smithi113.stdout:(30/174): ceph-test-19.0.0-2609.g191d963a.el9.x 16 MB/s | 48 MB 00:02 2024-04-02T14:29:33.337 INFO:teuthology.orchestra.run.smithi062.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.9 MB/s | 247 kB 00:00 2024-04-02T14:29:33.379 INFO:teuthology.orchestra.run.smithi062.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-04-02T14:29:33.386 INFO:teuthology.orchestra.run.smithi113.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 6.2 MB/s | 3.5 MB 00:00 2024-04-02T14:29:33.437 INFO:teuthology.orchestra.run.smithi113.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 4.9 MB/s | 249 kB 00:00 2024-04-02T14:29:33.495 INFO:teuthology.orchestra.run.smithi113.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 844 kB/s | 49 kB 00:00 2024-04-02T14:29:33.554 INFO:teuthology.orchestra.run.smithi113.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 282 kB/s | 16 kB 00:00 2024-04-02T14:29:33.629 INFO:teuthology.orchestra.run.smithi062.stdout:(156/174): librados2-19.0.0-2609.g191d963a.el9. 3.2 MB/s | 3.4 MB 00:01 2024-04-02T14:29:33.630 INFO:teuthology.orchestra.run.smithi113.stdout:(35/174): ceph-volume-19.0.0-2609.g191d963a.el9 3.4 MB/s | 264 kB 00:00 2024-04-02T14:29:33.654 INFO:teuthology.orchestra.run.smithi062.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 260 kB/s | 71 kB 00:00 2024-04-02T14:29:33.730 INFO:teuthology.orchestra.run.smithi062.stdout:(158/174): librbd1-19.0.0-2609.g191d963a.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-04-02T14:29:33.788 INFO:teuthology.orchestra.run.smithi062.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.4 MB/s | 192 kB 00:00 2024-04-02T14:29:33.796 INFO:teuthology.orchestra.run.smithi113.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-04-02T14:29:33.830 INFO:teuthology.orchestra.run.smithi113.stdout:(37/174): cephadm-19.0.0-2609.g191d963a.el9.noa 3.7 MB/s | 762 kB 00:00 2024-04-02T14:29:33.847 INFO:teuthology.orchestra.run.smithi113.stdout:(38/174): cephfs-top-19.0.0-2609.g191d963a.el9. 363 kB/s | 18 kB 00:00 2024-04-02T14:29:33.864 INFO:teuthology.orchestra.run.smithi062.stdout:(160/174): elfutils-debuginfod-client-0.190-2.e 156 kB/s | 36 kB 00:00 2024-04-02T14:29:33.889 INFO:teuthology.orchestra.run.smithi062.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 230 kB/s | 23 kB 00:00 2024-04-02T14:29:33.956 INFO:teuthology.orchestra.run.smithi062.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-04-02T14:29:34.006 INFO:teuthology.orchestra.run.smithi062.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 767 kB/s | 108 kB 00:00 2024-04-02T14:29:34.022 INFO:teuthology.orchestra.run.smithi113.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-04-02T14:29:34.040 INFO:teuthology.orchestra.run.smithi062.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-04-02T14:29:34.073 INFO:teuthology.orchestra.run.smithi113.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.0 MB/s | 920 kB 00:00 2024-04-02T14:29:34.082 INFO:teuthology.orchestra.run.smithi062.stdout:(165/174): elfutils-libs-0.190-2.el9.x86_64.rpm 728 kB/s | 256 kB 00:00 2024-04-02T14:29:34.098 INFO:teuthology.orchestra.run.smithi113.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-04-02T14:29:34.132 INFO:teuthology.orchestra.run.smithi113.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-04-02T14:29:34.133 INFO:teuthology.orchestra.run.smithi062.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 512 kB/s | 64 kB 00:00 2024-04-02T14:29:34.157 INFO:teuthology.orchestra.run.smithi113.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 543 kB/s | 32 kB 00:00 2024-04-02T14:29:34.157 INFO:teuthology.orchestra.run.smithi062.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 246 kB/s | 29 kB 00:00 2024-04-02T14:29:34.249 INFO:teuthology.orchestra.run.smithi113.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-02T14:29:34.274 INFO:teuthology.orchestra.run.smithi113.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 17 kB 00:00 2024-04-02T14:29:34.299 INFO:teuthology.orchestra.run.smithi113.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-04-02T14:29:34.325 INFO:teuthology.orchestra.run.smithi113.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-04-02T14:29:34.350 INFO:teuthology.orchestra.run.smithi113.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-04-02T14:29:34.384 INFO:teuthology.orchestra.run.smithi113.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-04-02T14:29:34.409 INFO:teuthology.orchestra.run.smithi113.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-04-02T14:29:34.434 INFO:teuthology.orchestra.run.smithi113.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 939 kB/s | 46 kB 00:00 2024-04-02T14:29:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.4 MB/s | 1.2 MB 00:00 2024-04-02T14:29:34.460 INFO:teuthology.orchestra.run.smithi113.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-02T14:29:34.485 INFO:teuthology.orchestra.run.smithi113.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-02T14:29:34.519 INFO:teuthology.orchestra.run.smithi113.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-04-02T14:29:34.539 INFO:teuthology.orchestra.run.smithi113.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 682 kB/s | 34 kB 00:00 2024-04-02T14:29:34.591 INFO:teuthology.orchestra.run.smithi062.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.3 MB/s | 2.3 MB 00:00 2024-04-02T14:29:34.625 INFO:teuthology.orchestra.run.smithi062.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.5 MB/s | 465 kB 00:00 2024-04-02T14:29:34.678 INFO:teuthology.orchestra.run.smithi113.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-02T14:29:34.717 INFO:teuthology.orchestra.run.smithi062.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.7 MB/s | 2.2 MB 00:00 2024-04-02T14:29:34.720 INFO:teuthology.orchestra.run.smithi113.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.5 MB 00:00 2024-04-02T14:29:34.753 INFO:teuthology.orchestra.run.smithi113.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 817 kB 00:00 2024-04-02T14:29:34.759 INFO:teuthology.orchestra.run.smithi062.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 187 kB/s | 31 kB 00:00 2024-04-02T14:29:34.779 INFO:teuthology.orchestra.run.smithi113.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 132 kB 00:00 2024-04-02T14:29:34.801 INFO:teuthology.orchestra.run.smithi062.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 899 kB/s | 74 kB 00:00 2024-04-02T14:29:34.954 INFO:teuthology.orchestra.run.smithi113.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.7 MB 00:00 2024-04-02T14:29:34.988 INFO:teuthology.orchestra.run.smithi113.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-04-02T14:29:35.013 INFO:teuthology.orchestra.run.smithi113.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-02T14:29:35.176 INFO:teuthology.orchestra.run.smithi062.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 2.0 MB/s | 1.1 MB 00:00 2024-04-02T14:29:35.185 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-04-02T14:29:35.185 INFO:teuthology.orchestra.run.smithi062.stdout:Total 15 MB/s | 226 MB 00:15 2024-04-02T14:29:35.188 INFO:teuthology.orchestra.run.smithi113.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-04-02T14:29:35.205 INFO:teuthology.orchestra.run.smithi113.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 128 kB/s | 24 kB 00:00 2024-04-02T14:29:35.222 INFO:teuthology.orchestra.run.smithi113.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 538 kB/s | 18 kB 00:00 2024-04-02T14:29:35.256 INFO:teuthology.orchestra.run.smithi113.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-02T14:29:35.273 INFO:teuthology.orchestra.run.smithi113.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 629 kB/s | 32 kB 00:00 2024-04-02T14:29:35.448 INFO:teuthology.orchestra.run.smithi113.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-04-02T14:29:35.474 INFO:teuthology.orchestra.run.smithi113.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-04-02T14:29:35.498 INFO:teuthology.orchestra.run.smithi113.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-04-02T14:29:35.524 INFO:teuthology.orchestra.run.smithi113.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-04-02T14:29:35.549 INFO:teuthology.orchestra.run.smithi113.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 818 kB/s | 41 kB 00:00 2024-04-02T14:29:35.575 INFO:teuthology.orchestra.run.smithi113.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-04-02T14:29:35.600 INFO:teuthology.orchestra.run.smithi113.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-04-02T14:29:35.692 INFO:teuthology.orchestra.run.smithi113.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 211 kB 00:00 2024-04-02T14:29:35.717 INFO:teuthology.orchestra.run.smithi113.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 405 kB/s | 47 kB 00:00 2024-04-02T14:29:35.742 INFO:teuthology.orchestra.run.smithi113.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 853 kB/s | 43 kB 00:00 2024-04-02T14:29:35.943 INFO:teuthology.orchestra.run.smithi113.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 194 kB/s | 39 kB 00:00 2024-04-02T14:29:36.235 INFO:teuthology.orchestra.run.smithi113.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-02T14:29:36.260 INFO:teuthology.orchestra.run.smithi113.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 130 kB/s | 41 kB 00:00 2024-04-02T14:29:36.282 INFO:teuthology.orchestra.run.smithi113.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 4.6 MB/s | 220 kB 00:00 2024-04-02T14:29:36.299 INFO:teuthology.orchestra.run.smithi113.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 899 kB/s | 35 kB 00:00 2024-04-02T14:29:36.316 INFO:teuthology.orchestra.run.smithi113.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.3 MB/s | 46 kB 00:00 2024-04-02T14:29:36.350 INFO:teuthology.orchestra.run.smithi113.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-04-02T14:29:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-04-02T14:29:36.377 INFO:teuthology.orchestra.run.smithi113.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 964 kB/s | 56 kB 00:00 2024-04-02T14:29:36.400 INFO:teuthology.orchestra.run.smithi113.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-02T14:29:36.426 INFO:teuthology.orchestra.run.smithi113.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 879 kB/s | 44 kB 00:00 2024-04-02T14:29:36.451 INFO:teuthology.orchestra.run.smithi113.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-02T14:29:36.476 INFO:teuthology.orchestra.run.smithi113.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-02T14:29:36.539 INFO:teuthology.orchestra.run.smithi113.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-04-02T14:29:36.560 INFO:teuthology.orchestra.run.smithi113.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-04-02T14:29:36.586 INFO:teuthology.orchestra.run.smithi113.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 124 kB 00:00 2024-04-02T14:29:36.611 INFO:teuthology.orchestra.run.smithi113.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-02T14:29:36.758 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-04-02T14:29:36.758 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-04-02T14:29:37.636 INFO:teuthology.orchestra.run.smithi113.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 178 kB/s | 187 kB 00:01 2024-04-02T14:29:37.945 INFO:teuthology.orchestra.run.smithi113.stdout:(95/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 2.1 MB/s | 17 MB 00:08 2024-04-02T14:29:38.561 INFO:teuthology.orchestra.run.smithi113.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 412 kB/s | 803 kB 00:01 2024-04-02T14:29:38.804 INFO:teuthology.orchestra.run.smithi113.stdout:(97/174): libquadmath-11.4.1-3.el9.x86_64.rpm 167 kB/s | 195 kB 00:01 2024-04-02T14:29:38.904 INFO:teuthology.orchestra.run.smithi113.stdout:(98/174): python3-requests-2.25.1-8.el9.noarch. 130 kB/s | 125 kB 00:00 2024-04-02T14:29:39.288 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-04-02T14:29:39.289 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-04-02T14:29:39.337 INFO:teuthology.orchestra.run.smithi113.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 28 kB/s | 21 kB 00:00 2024-04-02T14:29:39.512 INFO:teuthology.orchestra.run.smithi113.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 36 kB/s | 25 kB 00:00 2024-04-02T14:29:40.504 INFO:teuthology.orchestra.run.smithi113.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 210 kB/s | 245 kB 00:01 2024-04-02T14:29:40.536 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T14:29:40.537 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-04-02T14:29:40.537 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-04-02T14:29:40.537 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-04-02T14:29:40.537 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-04-02T14:29:40.537 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-04-02T14:29:40.537 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-04-02T14:29:40.537 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-04-02T14:29:40.537 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:29:40.671 INFO:teuthology.orchestra.run.smithi113.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 214 kB/s | 248 kB 00:01 2024-04-02T14:29:40.701 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T14:29:40.717 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T14:29:40.717 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:29:40.717 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T14:29:40.717 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:29:40.771 INFO:teuthology.orchestra.run.smithi113.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-04-02T14:29:41.139 INFO:teuthology.orchestra.run.smithi113.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-04-02T14:29:41.330 INFO:teuthology.orchestra.run.smithi113.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.8 MB/s | 548 kB 00:00 2024-04-02T14:29:41.381 INFO:teuthology.orchestra.run.smithi113.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 6.0 MB/s | 309 kB 00:00 2024-04-02T14:29:41.406 INFO:teuthology.orchestra.run.smithi113.stdout:(107/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 104 kB/s | 94 kB 00:00 2024-04-02T14:29:41.431 INFO:teuthology.orchestra.run.smithi113.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 394 kB/s | 19 kB 00:00 2024-04-02T14:29:41.473 INFO:teuthology.orchestra.run.smithi113.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 624 kB/s | 26 kB 00:00 2024-04-02T14:29:41.515 INFO:teuthology.orchestra.run.smithi113.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-02T14:29:41.557 INFO:teuthology.orchestra.run.smithi113.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-02T14:29:41.607 INFO:teuthology.orchestra.run.smithi113.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-02T14:29:41.683 INFO:teuthology.orchestra.run.smithi113.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-04-02T14:29:41.758 INFO:teuthology.orchestra.run.smithi113.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-02T14:29:41.783 INFO:teuthology.orchestra.run.smithi113.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-04-02T14:29:41.808 INFO:teuthology.orchestra.run.smithi113.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 866 kB/s | 43 kB 00:00 2024-04-02T14:29:41.834 INFO:teuthology.orchestra.run.smithi113.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-04-02T14:29:41.859 INFO:teuthology.orchestra.run.smithi113.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-02T14:29:41.884 INFO:teuthology.orchestra.run.smithi113.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-02T14:29:41.910 INFO:teuthology.orchestra.run.smithi113.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-02T14:29:41.935 INFO:teuthology.orchestra.run.smithi113.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-04-02T14:29:41.960 INFO:teuthology.orchestra.run.smithi113.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 705 kB/s | 35 kB 00:00 2024-04-02T14:29:41.985 INFO:teuthology.orchestra.run.smithi113.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.9 MB/s | 203 kB 00:00 2024-04-02T14:29:42.036 INFO:teuthology.orchestra.run.smithi113.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-04-02T14:29:42.061 INFO:teuthology.orchestra.run.smithi113.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-02T14:29:42.086 INFO:teuthology.orchestra.run.smithi113.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-02T14:29:42.112 INFO:teuthology.orchestra.run.smithi113.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-02T14:29:42.137 INFO:teuthology.orchestra.run.smithi113.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-04-02T14:29:42.162 INFO:teuthology.orchestra.run.smithi113.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-04-02T14:29:42.187 INFO:teuthology.orchestra.run.smithi113.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-02T14:29:42.199 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T14:29:42.224 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T14:29:42.224 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:29:42.224 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T14:29:42.224 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T14:29:42.224 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T14:29:42.224 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:29:42.229 INFO:teuthology.orchestra.run.smithi113.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-04-02T14:29:42.263 INFO:teuthology.orchestra.run.smithi113.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 612 kB/s | 46 kB 00:00 2024-04-02T14:29:42.280 INFO:teuthology.orchestra.run.smithi113.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-02T14:29:42.305 INFO:teuthology.orchestra.run.smithi113.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-04-02T14:29:42.331 INFO:teuthology.orchestra.run.smithi113.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-02T14:29:42.356 INFO:teuthology.orchestra.run.smithi113.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-04-02T14:29:42.381 INFO:teuthology.orchestra.run.smithi113.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-04-02T14:29:42.407 INFO:teuthology.orchestra.run.smithi113.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-04-02T14:29:42.432 INFO:teuthology.orchestra.run.smithi113.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-04-02T14:29:42.457 INFO:teuthology.orchestra.run.smithi113.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-02T14:29:42.457 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T14:29:42.479 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T14:29:42.479 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:29:42.480 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T14:29:42.480 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T14:29:42.480 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T14:29:42.480 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:29:42.482 INFO:teuthology.orchestra.run.smithi113.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-02T14:29:42.509 INFO:teuthology.orchestra.run.smithi113.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-04-02T14:29:42.533 INFO:teuthology.orchestra.run.smithi113.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-02T14:29:42.558 INFO:teuthology.orchestra.run.smithi113.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-02T14:29:42.575 INFO:teuthology.orchestra.run.smithi113.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-04-02T14:29:42.609 INFO:teuthology.orchestra.run.smithi113.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-04-02T14:29:42.626 INFO:teuthology.orchestra.run.smithi113.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-04-02T14:29:42.651 INFO:teuthology.orchestra.run.smithi113.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-04-02T14:29:42.701 INFO:teuthology.orchestra.run.smithi113.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-02T14:29:42.925 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T14:29:42.947 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T14:29:42.947 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:29:42.947 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T14:29:42.947 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T14:29:42.947 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T14:29:42.947 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:29:42.977 INFO:teuthology.orchestra.run.smithi113.stdout:(150/174): librbd1-19.0.0-2609.g191d963a.el9.x8 11 MB/s | 3.1 MB 00:00 2024-04-02T14:29:43.035 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T14:29:43.069 INFO:teuthology.orchestra.run.smithi113.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-04-02T14:29:43.144 INFO:teuthology.orchestra.run.smithi113.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-02T14:29:43.177 INFO:teuthology.orchestra.run.smithi113.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-02T14:29:43.219 INFO:teuthology.orchestra.run.smithi113.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-04-02T14:29:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-04-02T14:29:43.437 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T14:29:43.467 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T14:29:43.499 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T14:29:43.560 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T14:29:43.590 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T14:29:43.653 INFO:teuthology.orchestra.run.smithi113.stdout:(156/174): librados2-19.0.0-2609.g191d963a.el9. 3.4 MB/s | 3.4 MB 00:01 2024-04-02T14:29:44.137 INFO:teuthology.orchestra.run.smithi113.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 83 kB/s | 71 kB 00:00 2024-04-02T14:29:44.396 INFO:teuthology.orchestra.run.smithi113.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 49 kB/s | 36 kB 00:00 2024-04-02T14:29:44.623 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T14:29:44.659 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T14:29:45.087 INFO:teuthology.orchestra.run.smithi113.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 202 kB/s | 192 kB 00:00 2024-04-02T14:29:45.505 INFO:teuthology.orchestra.run.smithi113.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 231 kB/s | 256 kB 00:01 2024-04-02T14:29:45.620 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T14:29:45.642 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T14:29:45.742 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T14:29:45.788 INFO:teuthology.orchestra.run.smithi113.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 33 kB/s | 23 kB 00:00 2024-04-02T14:29:45.913 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T14:29:45.944 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T14:29:45.967 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T14:29:45.967 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:29:45.967 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T14:29:45.968 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T14:29:45.968 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T14:29:45.968 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:29:46.012 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T14:29:46.022 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T14:29:46.067 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T14:29:46.213 INFO:teuthology.orchestra.run.smithi113.stdout:(162/174): openssl-devel-3.0.7-27.el9.x86_64.rp 579 kB/s | 4.1 MB 00:07 2024-04-02T14:29:46.522 INFO:teuthology.orchestra.run.smithi113.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 107 kB/s | 108 kB 00:01 2024-04-02T14:29:46.822 INFO:teuthology.orchestra.run.smithi113.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 154 kB/s | 155 kB 00:01 2024-04-02T14:29:46.946 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T14:29:46.981 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T14:29:47.004 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T14:29:47.004 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:29:47.004 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T14:29:47.004 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T14:29:47.004 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T14:29:47.004 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:29:47.048 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T14:29:47.069 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T14:29:47.070 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:29:47.070 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T14:29:47.070 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:29:47.116 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T14:29:47.138 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T14:29:47.138 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:29:47.138 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T14:29:47.138 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T14:29:47.138 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T14:29:47.138 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:29:47.213 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-04-02T14:29:47.231 INFO:teuthology.orchestra.run.smithi113.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 133 kB/s | 136 kB 00:01 2024-04-02T14:29:47.332 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T14:29:47.364 INFO:teuthology.orchestra.run.smithi113.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 78 kB/s | 64 kB 00:00 2024-04-02T14:29:47.383 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T14:29:47.423 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T14:29:47.450 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T14:29:47.468 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T14:29:47.472 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T14:29:47.472 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:29:47.472 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T14:29:47.472 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T14:29:47.473 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T14:29:47.473 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:29:47.560 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T14:29:47.614 INFO:teuthology.orchestra.run.smithi113.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 36 kB/s | 29 kB 00:00 2024-04-02T14:29:47.661 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T14:29:47.724 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T14:29:47.764 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T14:29:47.865 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T14:29:48.304 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T14:29:48.353 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T14:29:48.515 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T14:29:48.546 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T14:29:48.673 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T14:29:48.715 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T14:29:48.777 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T14:29:49.099 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T14:29:49.176 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T14:29:49.201 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T14:29:49.356 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T14:29:49.409 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T14:29:49.511 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T14:29:49.581 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T14:29:49.613 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T14:29:49.681 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T14:29:49.764 INFO:teuthology.orchestra.run.smithi113.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 489 kB/s | 1.2 MB 00:02 2024-04-02T14:29:49.953 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T14:29:50.030 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T14:29:50.071 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T14:29:50.437 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T14:29:50.656 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T14:29:50.751 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T14:29:50.832 INFO:teuthology.orchestra.run.smithi113.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 637 kB/s | 2.2 MB 00:03 2024-04-02T14:29:50.993 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T14:29:51.082 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T14:29:51.315 INFO:teuthology.orchestra.run.smithi113.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 300 kB/s | 465 kB 00:01 2024-04-02T14:29:51.471 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T14:29:51.556 INFO:teuthology.orchestra.run.smithi113.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 43 kB/s | 31 kB 00:00 2024-04-02T14:29:51.674 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T14:29:51.817 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T14:29:51.879 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T14:29:51.904 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T14:29:51.917 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T14:29:51.946 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T14:29:51.976 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T14:29:51.985 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T14:29:52.003 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T14:29:52.006 INFO:teuthology.orchestra.run.smithi113.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 532 kB/s | 2.3 MB 00:04 2024-04-02T14:29:52.108 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T14:29:52.172 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T14:29:52.186 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T14:29:52.230 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T14:29:52.302 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T14:29:52.326 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T14:29:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T14:29:52.369 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T14:29:52.390 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T14:29:52.407 INFO:teuthology.orchestra.run.smithi113.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 87 kB/s | 74 kB 00:00 2024-04-02T14:29:52.431 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T14:29:52.452 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T14:29:52.495 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T14:29:52.507 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T14:29:52.527 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T14:29:52.548 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T14:29:52.567 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T14:29:52.589 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T14:29:52.643 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T14:29:52.674 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T14:29:52.704 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T14:29:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T14:29:52.737 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T14:29:52.823 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T14:29:52.860 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T14:29:52.965 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T14:29:53.031 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T14:29:53.172 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T14:29:53.272 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T14:29:53.311 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T14:29:53.330 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T14:29:53.359 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T14:29:53.533 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T14:29:53.715 INFO:teuthology.orchestra.run.smithi113.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 458 kB/s | 1.1 MB 00:02 2024-04-02T14:29:53.721 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-02T14:29:53.721 INFO:teuthology.orchestra.run.smithi113.stdout:Total 8.8 MB/s | 226 MB 00:25 2024-04-02T14:29:54.073 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-02T14:29:54.158 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T14:29:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T14:29:54.261 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T14:29:54.281 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T14:29:54.344 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-02T14:29:54.344 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-02T14:29:54.362 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T14:29:54.362 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T14:29:54.393 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T14:29:54.444 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T14:29:54.469 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T14:29:54.486 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T14:29:54.495 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T14:29:54.495 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T14:29:54.581 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T14:29:54.583 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T14:29:54.611 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T14:29:54.621 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T14:29:54.621 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T14:29:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T14:29:54.706 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T14:29:54.732 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T14:29:54.742 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T14:29:54.766 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T14:29:54.791 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T14:29:54.798 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T14:29:54.816 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T14:29:54.847 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T14:29:54.849 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T14:29:54.889 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T14:29:54.921 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T14:29:54.946 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T14:29:54.973 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T14:29:54.997 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T14:29:55.022 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T14:29:55.028 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T14:29:55.028 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T14:29:55.217 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T14:29:55.217 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T14:29:55.272 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T14:29:55.291 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T14:29:55.324 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T14:29:55.350 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T14:29:55.375 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T14:29:55.409 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T14:29:55.473 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T14:29:55.502 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T14:29:55.611 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T14:29:55.685 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T14:29:56.044 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-02T14:29:56.044 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-02T14:29:56.271 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T14:29:56.332 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T14:29:56.421 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T14:29:56.562 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T14:29:56.599 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T14:29:56.640 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T14:29:56.702 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T14:29:56.764 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T14:29:56.839 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T14:29:56.905 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T14:29:57.110 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T14:29:57.176 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T14:29:57.215 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T14:29:57.275 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T14:29:57.409 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T14:29:57.454 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T14:29:57.507 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T14:29:57.546 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T14:29:57.579 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T14:29:57.755 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T14:29:57.936 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T14:29:58.071 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T14:29:58.130 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T14:29:58.192 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T14:29:58.292 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T14:29:58.559 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T14:29:58.626 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T14:29:58.798 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T14:29:58.882 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T14:29:58.924 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T14:29:58.963 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T14:29:59.014 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T14:29:59.060 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T14:29:59.129 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T14:29:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T14:29:59.242 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T14:29:59.384 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T14:29:59.452 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T14:29:59.496 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T14:29:59.564 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T14:29:59.608 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T14:29:59.758 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T14:29:59.884 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T14:29:59.944 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T14:29:59.989 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T14:30:00.074 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T14:30:00.230 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T14:30:00.296 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T14:30:00.897 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T14:30:00.898 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T14:30:00.898 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T14:30:00.898 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T14:30:00.898 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T14:30:00.898 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T14:30:00.898 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T14:30:00.898 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T14:30:00.898 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T14:30:00.898 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T14:30:00.899 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T14:30:00.900 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T14:30:00.900 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T14:30:00.900 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T14:30:00.900 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T14:30:00.900 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T14:30:00.900 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T14:30:00.900 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T14:30:00.900 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T14:30:00.900 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T14:30:00.918 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T14:30:00.918 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T14:30:00.918 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T14:30:00.919 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T14:30:00.920 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T14:30:00.921 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T14:30:00.922 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T14:30:00.923 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T14:30:00.924 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T14:30:00.925 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T14:30:00.926 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T14:30:00.927 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T14:30:00.928 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T14:30:00.929 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T14:30:00.930 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T14:30:00.930 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T14:30:00.930 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T14:30:00.930 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T14:30:00.930 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T14:30:00.930 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T14:30:00.930 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T14:30:00.930 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T14:30:01.327 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T14:30:01.379 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T14:30:01.412 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T14:30:01.467 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T14:30:01.581 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T14:30:01.621 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T14:30:01.864 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T14:30:01.901 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T14:30:02.002 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T14:30:02.841 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T14:30:02.975 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T14:30:03.180 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-02T14:30:03.318 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T14:30:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T14:30:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T14:30:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T14:30:03.424 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T14:30:03.486 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T14:30:03.520 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T14:30:03.554 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T14:30:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T14:30:03.916 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T14:30:03.950 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T14:30:03.950 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T14:30:03.951 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T14:30:03.952 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T14:30:03.952 INFO:teuthology.orchestra.run.smithi204.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T14:30:03.952 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.952 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.952 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T14:30:03.952 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T14:30:03.952 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T14:30:03.952 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T14:30:03.952 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T14:30:03.952 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T14:30:03.952 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T14:30:03.953 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T14:30:03.953 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T14:30:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-04-02T14:30:03.953 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T14:30:03.953 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T14:30:03.953 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T14:30:03.953 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T14:30:03.953 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.953 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.954 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.954 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.954 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:03.954 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.954 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.954 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.954 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:03.954 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:03.955 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:03.955 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:03.955 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:03.955 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.955 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.955 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:03.955 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.955 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.956 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.956 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:03.956 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:03.957 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.957 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.957 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:03.958 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T14:30:03.958 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T14:30:03.958 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T14:30:03.958 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T14:30:03.958 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T14:30:03.958 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T14:30:03.958 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T14:30:03.958 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T14:30:03.959 INFO:teuthology.orchestra.run.smithi204.stdout: jq-1.6-16.el9.x86_64 2024-04-02T14:30:03.959 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T14:30:03.959 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T14:30:03.959 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T14:30:03.959 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.959 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.960 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.960 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T14:30:03.960 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T14:30:03.960 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T14:30:03.960 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T14:30:03.960 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T14:30:03.960 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.960 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.961 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T14:30:03.961 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.961 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T14:30:03.961 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T14:30:03.961 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T14:30:03.961 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T14:30:03.961 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T14:30:03.962 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T14:30:03.962 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T14:30:03.962 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T14:30:03.962 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T14:30:03.962 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T14:30:03.962 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T14:30:03.962 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T14:30:03.963 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T14:30:03.963 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T14:30:03.963 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T14:30:03.963 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T14:30:03.963 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T14:30:03.963 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T14:30:03.963 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T14:30:03.963 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T14:30:03.964 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T14:30:03.964 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T14:30:03.964 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T14:30:03.964 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T14:30:03.964 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.964 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.964 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.964 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T14:30:03.965 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T14:30:03.965 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T14:30:03.965 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T14:30:03.965 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T14:30:03.965 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T14:30:03.965 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T14:30:03.965 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T14:30:03.965 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T14:30:03.966 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T14:30:03.966 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T14:30:03.966 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T14:30:03.966 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T14:30:03.966 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T14:30:03.966 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T14:30:03.966 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T14:30:03.966 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T14:30:03.967 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T14:30:03.967 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T14:30:03.967 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T14:30:03.967 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T14:30:03.967 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T14:30:03.967 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T14:30:03.967 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T14:30:03.967 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T14:30:03.969 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T14:30:03.970 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T14:30:03.970 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T14:30:03.970 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T14:30:03.970 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T14:30:03.970 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T14:30:03.970 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T14:30:03.970 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T14:30:03.971 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T14:30:03.971 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T14:30:03.971 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T14:30:03.971 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T14:30:03.971 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T14:30:03.971 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T14:30:03.971 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T14:30:03.971 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T14:30:03.972 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T14:30:03.973 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T14:30:03.974 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T14:30:03.974 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T14:30:03.974 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T14:30:03.974 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T14:30:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T14:30:03.975 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T14:30:03.975 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:30:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-04-02T14:30:04.092 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T14:30:04.114 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T14:30:04.205 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T14:30:04.235 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T14:30:04.273 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T14:30:04.418 DEBUG:teuthology.parallel:result is None 2024-04-02T14:30:04.435 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T14:30:04.492 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T14:30:04.520 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T14:30:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T14:30:04.622 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T14:30:04.646 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T14:30:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T14:30:04.709 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T14:30:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T14:30:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T14:30:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T14:30:04.780 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T14:30:04.798 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T14:30:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T14:30:04.964 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T14:30:05.003 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T14:30:05.035 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T14:30:05.134 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T14:30:05.221 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T14:30:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T14:30:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T14:30:05.281 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T14:30:05.355 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T14:30:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T14:30:05.416 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T14:30:05.418 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T14:30:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T14:30:05.634 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T14:30:05.740 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T14:30:05.831 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T14:30:05.862 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T14:30:05.889 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T14:30:05.925 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T14:30:05.998 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T14:30:06.013 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T14:30:06.049 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T14:30:06.052 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T14:30:06.091 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T14:30:06.132 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T14:30:06.155 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T14:30:06.159 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T14:30:06.190 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T14:30:06.234 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T14:30:06.266 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T14:30:06.294 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T14:30:06.364 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T14:30:06.399 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T14:30:06.443 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T14:30:06.471 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T14:30:06.542 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T14:30:06.594 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T14:30:06.667 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T14:30:06.802 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T14:30:06.855 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T14:30:07.366 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T14:30:07.443 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T14:30:07.518 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T14:30:07.562 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T14:30:07.572 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T14:30:07.602 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T14:30:07.643 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T14:30:07.681 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T14:30:07.710 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T14:30:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T14:30:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T14:30:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T14:30:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T14:30:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T14:30:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T14:30:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T14:30:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T14:30:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T14:30:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T14:30:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T14:30:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T14:30:08.993 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T14:30:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T14:30:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T14:30:09.108 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T14:30:09.210 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T14:30:09.258 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T14:30:09.288 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T14:30:09.330 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T14:30:09.424 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T14:30:09.463 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T14:30:09.503 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T14:30:09.529 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T14:30:09.553 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T14:30:09.626 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T14:30:09.754 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T14:30:09.796 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T14:30:09.864 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T14:30:09.866 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T14:30:09.904 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T14:30:09.952 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T14:30:10.032 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T14:30:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T14:30:10.045 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T14:30:10.045 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T14:30:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:10.095 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T14:30:10.225 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T14:30:10.275 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T14:30:10.411 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T14:30:10.456 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T14:30:10.488 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T14:30:10.528 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T14:30:10.556 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T14:30:10.590 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T14:30:10.629 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T14:30:10.661 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T14:30:10.690 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T14:30:10.767 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T14:30:10.801 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T14:30:10.834 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T14:30:10.867 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T14:30:10.902 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T14:30:10.991 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T14:30:11.072 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T14:30:11.116 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T14:30:11.148 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T14:30:11.194 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T14:30:11.300 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T14:30:11.358 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T14:30:12.091 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T14:30:12.130 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T14:30:12.163 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T14:30:12.212 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T14:30:12.302 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T14:30:12.340 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T14:30:12.515 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T14:30:12.545 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T14:30:12.623 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T14:30:13.261 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T14:30:13.399 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T14:30:14.613 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T14:30:14.697 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T14:30:14.727 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T14:30:14.759 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T14:30:14.786 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T14:30:15.096 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T14:30:15.125 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T14:30:15.214 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T14:30:15.246 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T14:30:15.282 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T14:30:15.741 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T14:30:15.778 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T14:30:15.849 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T14:30:15.873 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T14:30:18.117 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T14:30:18.161 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T14:30:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T14:30:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T14:30:18.812 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T14:30:18.812 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T14:30:18.812 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T14:30:18.812 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:18.841 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T14:30:23.601 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T14:30:23.602 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-04-02T14:30:23.602 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-04-02T14:30:23.602 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-04-02T14:30:23.602 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-04-02T14:30:23.602 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-04-02T14:30:23.602 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /root 2024-04-02T14:30:23.602 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-04-02T14:30:23.602 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:23.851 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T14:30:23.903 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T14:30:23.903 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:23.903 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T14:30:23.903 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:25.593 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T14:30:25.679 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T14:30:25.679 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:25.679 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T14:30:25.679 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T14:30:25.680 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T14:30:25.680 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:25.944 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T14:30:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T14:30:26.014 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:26.015 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T14:30:26.015 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T14:30:26.015 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T14:30:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:26.520 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T14:30:26.592 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T14:30:26.592 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:26.593 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T14:30:26.593 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T14:30:26.593 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T14:30:26.593 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:26.713 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T14:30:27.391 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T14:30:27.431 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T14:30:27.486 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T14:30:27.591 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T14:30:27.626 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T14:30:28.975 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T14:30:29.034 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T14:30:30.190 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T14:30:30.245 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T14:30:30.402 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T14:30:30.584 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T14:30:30.630 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T14:30:30.725 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T14:30:30.725 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:30.725 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T14:30:30.725 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T14:30:30.725 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T14:30:30.725 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:30.809 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T14:30:30.851 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T14:30:30.854 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T14:30:30.854 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /sys 2024-04-02T14:30:30.854 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /proc 2024-04-02T14:30:30.854 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /mnt 2024-04-02T14:30:30.855 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /var/tmp 2024-04-02T14:30:30.855 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /home 2024-04-02T14:30:30.855 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /root 2024-04-02T14:30:30.855 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /tmp 2024-04-02T14:30:30.855 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:30.954 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T14:30:31.022 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T14:30:31.037 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T14:30:31.037 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:31.037 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T14:30:31.037 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:32.007 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T14:30:32.041 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T14:30:32.114 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T14:30:32.114 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:32.114 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T14:30:32.115 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T14:30:32.115 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T14:30:32.115 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:32.167 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T14:30:32.243 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T14:30:32.243 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:32.243 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T14:30:32.243 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:32.451 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T14:30:32.525 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T14:30:32.531 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T14:30:32.531 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:32.531 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T14:30:32.531 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T14:30:32.531 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T14:30:32.531 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:32.549 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T14:30:32.549 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:32.549 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T14:30:32.550 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T14:30:32.550 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T14:30:32.550 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:32.775 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T14:30:32.797 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T14:30:32.797 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:32.797 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T14:30:32.797 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T14:30:32.797 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T14:30:32.797 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:32.943 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T14:30:33.014 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T14:30:33.014 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:33.015 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T14:30:33.015 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T14:30:33.015 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T14:30:33.015 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:33.235 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T14:30:33.256 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T14:30:33.256 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:33.256 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T14:30:33.256 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T14:30:33.256 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T14:30:33.257 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:33.344 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T14:30:33.788 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T14:30:33.818 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T14:30:33.850 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T14:30:33.912 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T14:30:33.941 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T14:30:34.974 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T14:30:35.010 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T14:30:35.974 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T14:30:36.002 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T14:30:36.101 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T14:30:36.263 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T14:30:36.297 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T14:30:36.318 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T14:30:36.318 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:36.318 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T14:30:36.318 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T14:30:36.318 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T14:30:36.318 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:36.363 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T14:30:36.373 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T14:30:36.419 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T14:30:37.291 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T14:30:37.323 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T14:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T14:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T14:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T14:30:37.346 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T14:30:37.346 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:37.390 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T14:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T14:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T14:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:37.458 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T14:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T14:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T14:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T14:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T14:30:37.481 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:37.792 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T14:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T14:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T14:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T14:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T14:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T14:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:38.125 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T14:30:38.171 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T14:30:38.206 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T14:30:38.269 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T14:30:38.550 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T14:30:38.681 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T14:30:38.727 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T14:30:38.780 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T14:30:38.873 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T14:30:39.028 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T14:30:39.064 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T14:30:39.108 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T14:30:39.184 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T14:30:39.282 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T14:30:39.402 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T14:30:39.473 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T14:30:40.438 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T14:30:40.491 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T14:30:40.548 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T14:30:41.943 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T14:30:42.046 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T14:30:42.099 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T14:30:42.247 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T14:30:42.269 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T14:30:42.288 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T14:30:42.307 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T14:30:42.318 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T14:30:42.340 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T14:30:42.345 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T14:30:42.374 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T14:30:42.418 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T14:30:42.418 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T14:30:42.525 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T14:30:42.570 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T14:30:42.612 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T14:30:42.612 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T14:30:42.637 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T14:30:42.677 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T14:30:42.716 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T14:30:42.768 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T14:30:42.778 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T14:30:42.812 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T14:30:42.828 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T14:30:42.844 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T14:30:42.873 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T14:30:42.876 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T14:30:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T14:30:42.906 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T14:30:42.939 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T14:30:42.952 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T14:30:43.004 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T14:30:43.007 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T14:30:43.038 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T14:30:43.112 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T14:30:43.160 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T14:30:43.194 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T14:30:43.228 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T14:30:43.261 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T14:30:43.294 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T14:30:43.331 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T14:30:43.331 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T14:30:43.613 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T14:30:43.662 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T14:30:43.681 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T14:30:43.703 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T14:30:43.703 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T14:30:43.850 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T14:30:43.901 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T14:30:43.931 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T14:30:43.992 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T14:30:44.032 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T14:30:44.156 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T14:30:44.447 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T14:30:44.447 INFO:teuthology.orchestra.run.smithi062.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-02T14:30:44.447 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:44.496 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T14:30:44.582 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T14:30:44.623 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T14:30:44.703 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T14:30:44.744 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T14:30:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T14:30:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T14:30:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T14:30:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T14:30:44.909 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T14:30:44.938 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T14:30:44.939 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T14:30:45.023 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T14:30:45.049 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T14:30:45.091 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T14:30:45.117 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T14:30:45.142 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T14:30:45.174 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T14:30:45.214 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T14:30:45.246 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T14:30:45.272 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T14:30:45.297 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T14:30:45.322 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T14:30:45.347 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T14:30:45.353 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T14:30:45.353 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T14:30:45.542 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T14:30:45.542 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T14:30:45.625 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T14:30:45.656 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T14:30:45.683 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T14:30:45.709 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T14:30:45.734 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T14:30:45.798 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T14:30:49.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T14:30:49.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T14:30:49.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T14:30:49.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T14:30:49.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T14:30:49.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T14:30:49.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T14:30:49.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T14:30:49.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T14:30:49.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T14:30:49.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T14:30:49.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T14:30:49.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T14:30:49.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T14:30:49.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T14:30:49.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T14:30:49.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T14:30:49.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T14:30:49.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T14:30:49.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T14:30:49.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T14:30:49.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T14:30:49.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T14:30:49.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T14:30:49.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T14:30:49.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T14:30:49.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T14:30:49.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T14:30:49.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T14:30:49.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T14:30:49.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T14:30:49.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T14:30:49.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T14:30:49.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T14:30:49.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T14:30:49.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T14:30:49.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T14:30:49.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T14:30:49.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T14:30:49.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T14:30:49.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T14:30:49.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T14:30:49.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T14:30:49.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T14:30:49.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T14:30:49.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T14:30:49.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T14:30:49.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T14:30:49.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T14:30:49.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T14:30:49.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T14:30:49.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T14:30:49.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T14:30:49.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T14:30:49.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T14:30:49.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T14:30:49.769 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T14:30:49.769 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T14:30:49.769 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T14:30:49.769 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T14:30:49.769 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T14:30:49.769 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T14:30:49.769 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T14:30:49.769 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T14:30:49.769 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T14:30:49.769 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T14:30:49.770 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T14:30:49.770 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T14:30:49.770 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T14:30:49.770 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T14:30:49.770 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T14:30:49.770 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T14:30:49.770 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T14:30:49.771 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T14:30:49.771 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T14:30:49.771 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T14:30:49.771 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T14:30:49.771 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T14:30:49.771 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T14:30:49.771 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T14:30:49.771 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T14:30:49.771 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T14:30:49.771 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T14:30:49.772 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T14:30:49.772 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T14:30:49.772 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T14:30:49.772 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T14:30:49.772 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T14:30:49.772 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T14:30:49.772 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T14:30:49.772 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T14:30:49.772 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T14:30:49.773 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T14:30:49.773 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T14:30:49.773 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T14:30:49.773 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T14:30:49.773 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T14:30:49.773 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T14:30:49.773 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T14:30:49.773 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T14:30:49.773 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T14:30:49.773 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T14:30:49.774 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T14:30:49.774 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T14:30:49.774 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T14:30:49.774 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T14:30:49.774 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T14:30:49.774 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T14:30:49.774 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T14:30:49.774 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T14:30:49.774 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T14:30:49.775 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T14:30:49.775 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T14:30:49.775 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T14:30:49.775 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T14:30:49.775 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T14:30:49.775 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T14:30:49.775 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T14:30:49.775 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T14:30:49.775 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T14:30:49.776 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T14:30:49.776 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T14:30:49.776 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T14:30:49.776 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T14:30:49.776 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T14:30:49.776 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T14:30:49.776 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T14:30:49.776 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T14:30:49.776 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T14:30:49.777 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T14:30:49.777 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T14:30:49.777 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T14:30:49.777 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T14:30:49.777 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T14:30:49.777 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T14:30:49.777 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T14:30:49.777 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T14:30:49.777 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T14:30:49.778 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T14:30:49.778 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T14:30:49.778 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T14:30:49.778 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T14:30:49.778 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T14:30:49.778 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T14:30:49.778 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T14:30:49.778 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T14:30:49.778 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T14:30:49.778 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T14:30:49.779 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T14:30:49.779 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T14:30:49.779 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T14:30:49.779 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T14:30:49.779 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T14:30:49.779 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T14:30:49.779 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T14:30:49.779 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T14:30:49.779 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T14:30:49.779 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T14:30:49.779 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T14:30:49.780 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T14:30:49.780 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T14:30:49.780 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T14:30:49.780 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T14:30:49.780 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T14:30:49.780 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T14:30:49.780 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T14:30:49.780 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T14:30:49.780 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T14:30:49.780 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T14:30:49.780 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T14:30:49.781 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T14:30:49.781 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T14:30:49.781 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T14:30:49.781 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T14:30:49.781 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T14:30:49.781 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T14:30:49.781 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T14:30:49.781 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T14:30:49.781 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T14:30:49.781 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T14:30:49.781 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T14:30:49.782 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T14:30:49.782 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T14:30:49.782 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T14:30:49.782 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T14:30:49.782 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T14:30:51.381 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T14:30:51.382 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T14:30:51.383 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T14:30:51.384 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T14:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T14:30:51.386 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T14:30:51.387 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T14:30:51.388 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T14:30:51.389 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T14:30:51.390 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T14:30:51.390 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T14:30:51.390 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T14:30:51.390 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T14:30:51.390 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T14:30:51.390 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T14:30:51.390 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T14:30:51.390 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T14:30:51.390 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T14:30:51.390 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T14:30:51.390 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T14:30:51.391 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T14:30:51.392 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T14:30:51.392 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T14:30:51.393 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T14:30:51.394 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T14:30:51.395 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T14:30:51.396 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T14:30:51.397 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T14:30:51.397 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T14:30:51.397 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T14:30:51.397 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T14:30:51.397 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T14:30:51.397 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T14:30:51.397 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T14:30:52.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T14:30:52.612 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:52.612 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-04-02T14:30:52.612 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:52.612 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T14:30:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:52.615 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.616 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.616 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:52.616 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.616 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.616 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.616 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:52.617 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:52.617 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.617 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.617 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:52.617 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T14:30:52.617 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T14:30:52.617 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T14:30:52.617 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T14:30:52.617 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T14:30:52.617 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T14:30:52.618 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T14:30:52.618 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T14:30:52.618 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.6-16.el9.x86_64 2024-04-02T14:30:52.618 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T14:30:52.618 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T14:30:52.618 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T14:30:52.618 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.619 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.619 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.619 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T14:30:52.619 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T14:30:52.619 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T14:30:52.619 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T14:30:52.619 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T14:30:52.619 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.619 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.619 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T14:30:52.620 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.620 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T14:30:52.620 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T14:30:52.620 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T14:30:52.620 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T14:30:52.620 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T14:30:52.620 INFO:teuthology.orchestra.run.smithi062.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T14:30:52.620 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T14:30:52.621 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T14:30:52.622 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T14:30:52.623 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T14:30:52.623 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T14:30:52.623 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T14:30:52.623 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T14:30:52.623 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T14:30:52.623 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T14:30:52.623 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T14:30:52.623 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T14:30:52.624 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T14:30:52.624 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T14:30:52.624 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T14:30:52.624 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T14:30:52.624 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T14:30:52.624 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T14:30:52.624 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T14:30:52.624 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T14:30:52.624 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T14:30:52.624 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T14:30:52.625 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T14:30:52.626 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T14:30:52.627 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T14:30:52.628 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T14:30:52.628 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T14:30:52.628 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T14:30:52.628 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T14:30:52.628 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T14:30:52.628 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:30:52.628 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-02T14:30:53.412 DEBUG:teuthology.parallel:result is None 2024-04-02T14:30:54.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T14:30:54.570 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:54.570 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T14:30:54.571 INFO:teuthology.orchestra.run.smithi113.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T14:30:54.572 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.573 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:54.574 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: jq-1.6-16.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T14:30:54.575 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T14:30:54.576 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T14:30:54.577 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T14:30:54.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T14:30:54.579 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T14:30:54.580 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.581 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T14:30:54.582 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.583 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.583 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T14:30:54.583 INFO:teuthology.orchestra.run.smithi113.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T14:30:54.583 INFO:teuthology.orchestra.run.smithi113.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T14:30:54.583 INFO:teuthology.orchestra.run.smithi113.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T14:30:54.583 INFO:teuthology.orchestra.run.smithi113.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T14:30:54.583 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T14:30:54.583 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T14:30:54.583 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T14:30:54.583 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-02T14:30:54.583 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-02T14:30:55.055 DEBUG:teuthology.parallel:result is None 2024-04-02T14:30:55.056 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:30:55.280 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T14:30:55.335 INFO:teuthology.orchestra.run.smithi062.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T14:30:55.335 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T14:30:55.335 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T14:30:55.337 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:30:55.575 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T14:30:55.599 INFO:teuthology.orchestra.run.smithi113.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T14:30:55.599 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T14:30:55.599 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T14:30:55.601 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:30:55.739 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T14:30:55.765 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T14:30:55.766 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T14:30:55.766 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T14:30:55.768 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-02T14:30:55.768 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:30:55.768 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T14:30:55.846 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-02T14:30:55.846 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T14:30:55.881 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T14:30:55.881 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T14:30:55.916 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-02T14:30:55.916 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:30:55.916 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T14:30:55.990 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T14:30:56.109 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-02T14:30:56.110 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T14:30:56.141 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T14:30:56.213 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T14:30:56.213 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T14:30:56.242 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T14:30:56.309 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-02T14:30:56.309 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:30:56.309 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T14:30:56.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T14:30:56.495 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-02T14:30:56.495 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T14:30:56.526 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T14:30:56.596 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T14:30:56.596 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T14:30:56.624 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T14:30:56.693 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-02T14:30:56.694 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:30:56.694 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T14:30:56.782 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T14:30:56.894 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-02T14:30:56.894 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T14:30:56.925 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T14:30:56.999 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T14:30:56.999 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T14:30:57.026 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T14:30:57.093 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-02T14:30:57.207 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'subvols': {'create': 2}} 2024-04-02T14:30:57.208 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:30:57.208 INFO:tasks.cephadm:Cluster fsid is 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:30:57.208 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-02T14:30:57.208 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.62', 'mon.b': '172.21.15.113', 'mon.c': '172.21.15.204'} 2024-04-02T14:30:57.209 INFO:tasks.cephadm:First mon is mon.a on smithi062 2024-04-02T14:30:57.209 INFO:tasks.cephadm:First mgr is x 2024-04-02T14:30:57.209 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-02T14:30:57.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2024-04-02T14:30:57.291 DEBUG:teuthology.orchestra.run.smithi113:> sudo hostname $(hostname -s) 2024-04-02T14:30:57.324 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-04-02T14:30:57.351 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-02T14:30:57.351 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:30:57.501 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78154/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2609-g191d963a', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2609.g191d963a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-02 02:04:04.615852', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240402.004512', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-02T14:30:57.641 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20240402.004512, sha1 191d963ac9de7322da8aff20b567b3fa458e2ef6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:30:57.643 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm 2024-04-02T14:30:57.643 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T14:30:58.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 14:30 /home/ubuntu/cephtest/cephadm 2024-04-02T14:30:58.426 DEBUG:teuthology.orchestra.run.smithi113:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T14:30:59.143 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 14:30 /home/ubuntu/cephtest/cephadm 2024-04-02T14:30:59.144 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T14:30:59.617 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 14:30 /home/ubuntu/cephtest/cephadm 2024-04-02T14:30:59.618 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T14:30:59.683 DEBUG:teuthology.orchestra.run.smithi113:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T14:30:59.702 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T14:30:59.765 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 on all hosts... 2024-04-02T14:30:59.765 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T14:30:59.773 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T14:30:59.775 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T14:31:00.559 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T14:31:00.656 INFO:teuthology.orchestra.run.smithi113.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T14:31:01.000 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T14:31:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-02T14:31:37.152 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T14:31:37.152 INFO:teuthology.orchestra.run.smithi113.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T14:31:37.152 INFO:teuthology.orchestra.run.smithi113.stdout: "repo_digests": [ 2024-04-02T14:31:37.152 INFO:teuthology.orchestra.run.smithi113.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T14:31:37.152 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-02T14:31:37.153 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-02T14:31:42.965 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-02T14:31:42.966 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T14:31:42.966 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T14:31:42.967 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-04-02T14:31:42.967 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T14:31:42.967 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-04-02T14:31:42.967 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-02T14:31:44.913 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:31:44.913 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T14:31:44.913 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T14:31:44.913 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2024-04-02T14:31:44.913 INFO:teuthology.orchestra.run.smithi062.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T14:31:44.913 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:31:44.913 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:31:44.973 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2024-04-02T14:31:45.079 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph 2024-04-02T14:31:45.114 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-04-02T14:31:45.149 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2024-04-02T14:31:45.254 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /etc/ceph 2024-04-02T14:31:45.286 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-04-02T14:31:45.317 INFO:tasks.cephadm:Writing seed config... 2024-04-02T14:31:45.319 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-04-02T14:31:45.319 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-02T14:31:45.319 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-02T14:31:45.319 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-02T14:31:45.319 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-02T14:31:45.319 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-02T14:31:45.320 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-02T14:31:45.320 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-02T14:31:45.320 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-02T14:31:45.320 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-02T14:31:45.320 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-02T14:31:45.320 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-02T14:31:45.320 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-02T14:31:45.320 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-02T14:31:45.320 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-02T14:31:45.320 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-02T14:31:45.320 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-02T14:31:45.321 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-02T14:31:45.322 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-02T14:31:45.323 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-02T14:31:45.323 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-02T14:31:45.323 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-02T14:31:45.323 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-02T14:31:45.323 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-02T14:31:45.323 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-02T14:31:45.324 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:31:45.324 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-02T14:31:45.384 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-02T14:31:45.385 DEBUG:teuthology.orchestra.run.smithi062:mon.a> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.a.service 2024-04-02T14:31:45.432 DEBUG:teuthology.orchestra.run.smithi062:mgr.x> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.x.service 2024-04-02T14:31:45.478 INFO:tasks.cephadm:Bootstrapping... 2024-04-02T14:31:45.478 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 -v bootstrap --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.62 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:31:45.759 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-04-02T14:31:45.759 INFO:teuthology.orchestra.run.smithi062.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6', '-v', 'bootstrap', '--fsid', '9e86b65e-f0fd-11ee-b647-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.62', '--skip-admin-label'] 2024-04-02T14:31:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-02T14:31:45.759 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying podman|docker is present... 2024-04-02T14:31:45.816 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T14:31:45.816 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying lvm2 is present... 2024-04-02T14:31:45.817 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying time synchronization is in place... 2024-04-02T14:31:45.836 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T14:31:45.836 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T14:31:45.856 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T14:31:45.856 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-04-02T14:31:45.881 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2024-04-02T14:31:45.904 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2024-04-02T14:31:45.904 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2024-04-02T14:31:45.904 INFO:teuthology.orchestra.run.smithi062.stdout:Repeating the final host check... 2024-04-02T14:31:45.968 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T14:31:45.968 INFO:teuthology.orchestra.run.smithi062.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-02T14:31:45.968 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl is present 2024-04-02T14:31:45.968 INFO:teuthology.orchestra.run.smithi062.stdout:lvcreate is present 2024-04-02T14:31:45.985 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T14:31:45.985 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T14:31:46.003 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T14:31:46.003 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-04-02T14:31:46.026 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2024-04-02T14:31:46.050 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2024-04-02T14:31:46.051 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2024-04-02T14:31:46.051 INFO:teuthology.orchestra.run.smithi062.stdout:Host looks OK 2024-04-02T14:31:46.051 INFO:teuthology.orchestra.run.smithi062.stdout:Cluster fsid: 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:31:46.051 INFO:teuthology.orchestra.run.smithi062.stdout:Acquiring lock 139840517427888 on /run/cephadm/9e86b65e-f0fd-11ee-b647-cb9ed24678a4.lock 2024-04-02T14:31:46.051 INFO:teuthology.orchestra.run.smithi062.stdout:Lock 139840517427888 acquired on /run/cephadm/9e86b65e-f0fd-11ee-b647-cb9ed24678a4.lock 2024-04-02T14:31:46.054 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 3300 ... 2024-04-02T14:31:46.054 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 6789 ... 2024-04-02T14:31:46.054 INFO:teuthology.orchestra.run.smithi062.stdout:Base mon IP(s) is [172.21.15.62:3300, 172.21.15.62:6789], mon addrv is [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-04-02T14:31:46.069 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.62 metric 100 2024-04-02T14:31:46.069 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.62 metric 100 2024-04-02T14:31:46.082 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-02T14:31:46.082 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-02T14:31:46.083 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-04-02T14:31:46.094 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-02T14:31:46.095 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-02T14:31:46.095 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T14:31:46.095 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-02T14:31:46.095 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-02T14:31:46.095 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T14:31:46.098 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2024-04-02T14:31:46.098 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2024-04-02T14:31:46.098 INFO:teuthology.orchestra.run.smithi062.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-02T14:31:46.101 INFO:teuthology.orchestra.run.smithi062.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-02T14:31:46.104 INFO:teuthology.orchestra.run.smithi062.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T14:31:46.747 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f 2024-04-02T14:31:46.747 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T14:31:46.747 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Getting image source signatures 2024-04-02T14:31:46.748 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:db8c11fadded9ebce76f2e720ef87f32e8913958c61c526c89a5e4e2c41a8200 2024-04-02T14:31:46.748 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:11da15500eca1f826c328eb3265caa30b0a3730510e2a32023084d6fe9cee20c 2024-04-02T14:31:46.748 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying config sha256:f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f 2024-04-02T14:31:46.748 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-02T14:31:48.111 INFO:teuthology.orchestra.run.smithi062.stdout:ceph: stdout ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T14:31:48.112 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph version: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T14:31:48.112 INFO:teuthology.orchestra.run.smithi062.stdout:Extracting ceph user uid/gid from container image... 2024-04-02T14:31:49.276 INFO:teuthology.orchestra.run.smithi062.stdout:stat: stdout 167 167 2024-04-02T14:31:49.276 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial keys... 2024-04-02T14:31:50.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDVFgxmExcDMBAA8z+gX3t/ohdbiJR5jnFw9Q== 2024-04-02T14:31:51.832 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDXFgxmqmNtBRAAE+honSPrNphRTGLM2hXVrg== 2024-04-02T14:31:53.100 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDYFgxmyVwfExAAjEY2NZa76HQ63VoIOO72bA== 2024-04-02T14:31:53.102 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial monmap... 2024-04-02T14:31:54.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T14:31:54.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-02T14:31:54.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:31:54.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T14:31:54.484 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool for a [v2:172.21.15.62:3300,v1:172.21.15.62:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T14:31:54.484 INFO:teuthology.orchestra.run.smithi062.stdout:setting min_mon_release = quincy 2024-04-02T14:31:54.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: set fsid to 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:31:54.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T14:31:54.485 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:31:54.485 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mon... 2024-04-02T14:31:55.730 INFO:teuthology.orchestra.run.smithi062.stdout:create mon.a on 2024-04-02T14:31:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-02T14:31:56.345 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-02T14:31:56.618 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4.target. 2024-04-02T14:31:56.618 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4.target. 2024-04-02T14:31:57.001 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.a 2024-04-02T14:31:57.001 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.a.service: Unit ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.a.service not loaded. 2024-04-02T14:31:57.296 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4.target.wants/ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.a.service → /etc/systemd/system/ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@.service. 2024-04-02T14:31:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:57 smithi062 podman[31727]: 2024-04-02 14:31:57.729076272 +0000 UTC m=+0.153688317 container create 5b8f7ba934b7aa15257503266da8ec9c47a52d273a2490f4a2c3db43ba4098cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-02T14:31:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:57 smithi062 podman[31727]: 2024-04-02 14:31:57.974860149 +0000 UTC m=+0.399472207 container init 5b8f7ba934b7aa15257503266da8ec9c47a52d273a2490f4a2c3db43ba4098cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-02T14:31:58.102 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T14:31:58.103 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-04-02T14:31:58.118 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T14:31:58.118 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-04-02T14:31:58.118 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-04-02T14:31:58.118 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T14:31:58.119 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon to start... 2024-04-02T14:31:58.119 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon... 2024-04-02T14:31:58.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:57 smithi062 podman[31727]: 2024-04-02 14:31:57.987068264 +0000 UTC m=+0.411680319 container start 5b8f7ba934b7aa15257503266da8ec9c47a52d273a2490f4a2c3db43ba4098cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-02T14:31:58.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:58 smithi062 ceph-mon[31745]: mkfs 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:31:58.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:58 smithi062 ceph-mon[31745]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-02T14:31:58.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:58 smithi062 bash[31727]: 5b8f7ba934b7aa15257503266da8ec9c47a52d273a2490f4a2c3db43ba4098cc 2024-04-02T14:31:58.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:58 smithi062 systemd[1]: Started Ceph mon.a for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:31:59.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-02T14:31:59.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: monmap epoch 1 2024-04-02T14:31:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:31:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: last_changed 2024-04-02T14:31:53.643129+0000 2024-04-02T14:31:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: created 2024-04-02T14:31:53.643129+0000 2024-04-02T14:31:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: min_mon_release 19 (squid) 2024-04-02T14:31:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: election_strategy: 1 2024-04-02T14:31:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-02T14:31:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: fsmap 2024-04-02T14:31:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: osdmap e1: 0 total, 0 up, 0 in 2024-04-02T14:31:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: mgrmap e1: no daemons active 2024-04-02T14:31:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:31:59 smithi062 ceph-mon[31745]: from='client.? 172.21.15.62:0/666399310' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-02T14:31:59.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout cluster: 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout id: 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout services: 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.786801s) 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout data: 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-02T14:31:59.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pgs: 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.752+0000 7f134b200700 1 Processor -- start 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.753+0000 7f134b200700 1 -- start start 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.753+0000 7f134b200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c10a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.753+0000 7f134b200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f134c10ad00 con 0x7f134c10a310 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.754+0000 7f1349e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c10a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.754+0000 7f1349e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c10a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46444/0 (socket says 172.21.15.62:46444) 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.754+0000 7f1349e00700 1 -- 172.21.15.62:0/667614226 learned_addr learned my addr 172.21.15.62:0/667614226 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.755+0000 7f1349e00700 1 -- 172.21.15.62:0/667614226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c10b4c0 con 0x7f134c10a310 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.755+0000 7f1349e00700 1 --2- 172.21.15.62:0/667614226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c10a730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f133c0098a0 tx=0x7f133c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a622a2fdbf6d8d61 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.755+0000 7f1348a00700 1 -- 172.21.15.62:0/667614226 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f133c004160 con 0x7f134c10a310 2024-04-02T14:31:59.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.755+0000 7f1348a00700 1 -- 172.21.15.62:0/667614226 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133c004300 con 0x7f134c10a310 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.755+0000 7f1348a00700 1 -- 172.21.15.62:0/667614226 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f133c0036a0 con 0x7f134c10a310 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.756+0000 7f134b200700 1 -- 172.21.15.62:0/667614226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 msgr2=0x7f134c10a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.756+0000 7f134b200700 1 --2- 172.21.15.62:0/667614226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c10a730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f133c0098a0 tx=0x7f133c009bb0 comp rx=0 tx=0).stop 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.756+0000 7f134b200700 1 -- 172.21.15.62:0/667614226 shutdown_connections 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.756+0000 7f134b200700 1 --2- 172.21.15.62:0/667614226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c10a730 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.756+0000 7f134b200700 1 -- 172.21.15.62:0/667614226 >> 172.21.15.62:0/667614226 conn(0x7f134c07d700 msgr2=0x7f134c07db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.756+0000 7f134b200700 1 -- 172.21.15.62:0/667614226 shutdown_connections 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.756+0000 7f134b200700 1 -- 172.21.15.62:0/667614226 wait complete. 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.757+0000 7f134b200700 1 Processor -- start 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.758+0000 7f134b200700 1 -- start start 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.758+0000 7f134b200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.758+0000 7f134b200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f134c10b9f0 con 0x7f134c10a310 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.758+0000 7f1349e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:31:59.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.759+0000 7f1349e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c19d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46454/0 (socket says 172.21.15.62:46454) 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.759+0000 7f1349e00700 1 -- 172.21.15.62:0/666399310 learned_addr learned my addr 172.21.15.62:0/666399310 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.759+0000 7f1349e00700 1 -- 172.21.15.62:0/666399310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c19df10 con 0x7f134c10a310 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.759+0000 7f1349e00700 1 --2- 172.21.15.62:0/666399310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c19d9d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f133c000c00 tx=0x7f133c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.760+0000 7f1342a00700 1 -- 172.21.15.62:0/666399310 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f133c003db0 con 0x7f134c10a310 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.760+0000 7f1342a00700 1 -- 172.21.15.62:0/666399310 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133c011630 con 0x7f134c10a310 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.760+0000 7f134b200700 1 -- 172.21.15.62:0/666399310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134c19e1a0 con 0x7f134c10a310 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.760+0000 7f1342a00700 1 -- 172.21.15.62:0/666399310 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f133c011950 con 0x7f134c10a310 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.760+0000 7f134b200700 1 -- 172.21.15.62:0/666399310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134c1a0e90 con 0x7f134c10a310 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.760+0000 7f1342a00700 1 -- 172.21.15.62:0/666399310 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f133c011af0 con 0x7f134c10a310 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.760+0000 7f1342a00700 1 -- 172.21.15.62:0/666399310 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f133c01c450 con 0x7f134c10a310 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.761+0000 7f134b200700 1 -- 172.21.15.62:0/666399310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134c063c20 con 0x7f134c10a310 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.764+0000 7f1342a00700 1 -- 172.21.15.62:0/666399310 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f133c032020 con 0x7f134c10a310 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.848+0000 7f134b200700 1 -- 172.21.15.62:0/666399310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f134c02deb0 con 0x7f134c10a310 2024-04-02T14:31:59.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.849+0000 7f1342a00700 1 -- 172.21.15.62:0/666399310 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f133c004260 con 0x7f134c10a310 2024-04-02T14:31:59.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.850+0000 7f134b200700 1 -- 172.21.15.62:0/666399310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 msgr2=0x7f134c19d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:31:59.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.850+0000 7f134b200700 1 --2- 172.21.15.62:0/666399310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c19d9d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f133c000c00 tx=0x7f133c004750 comp rx=0 tx=0).stop 2024-04-02T14:31:59.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.851+0000 7f134b200700 1 -- 172.21.15.62:0/666399310 shutdown_connections 2024-04-02T14:31:59.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.851+0000 7f134b200700 1 --2- 172.21.15.62:0/666399310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134c10a310 0x7f134c19d9d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:31:59.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.851+0000 7f134b200700 1 -- 172.21.15.62:0/666399310 >> 172.21.15.62:0/666399310 conn(0x7f134c07d700 msgr2=0x7f134c194670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:31:59.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.851+0000 7f134b200700 1 -- 172.21.15.62:0/666399310 shutdown_connections 2024-04-02T14:31:59.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:31:58.851+0000 7f134b200700 1 -- 172.21.15.62:0/666399310 wait complete. 2024-04-02T14:31:59.533 INFO:teuthology.orchestra.run.smithi062.stdout:mon is available 2024-04-02T14:31:59.533 INFO:teuthology.orchestra.run.smithi062.stdout:Assimilating anything we can from ceph.conf... 2024-04-02T14:32:00.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:00 smithi062 ceph-mon[31745]: from='client.? 172.21.15.62:0/2529160664' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T14:32:00.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:00 smithi062 ceph-mon[31745]: from='client.? 172.21.15.62:0/2529160664' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-02T14:32:00.954 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:32:00.954 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T14:32:00.954 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:32:00.955 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-04-02T14:32:00.955 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T14:32:00.955 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T14:32:00.956 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T14:32:00.956 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T14:32:00.956 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:32:00.956 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T14:32:00.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T14:32:00.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:32:00.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T14:32:00.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-02T14:32:00.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T14:32:00.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T14:32:00.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T14:32:00.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T14:32:00.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T14:32:00.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.175+0000 7fa7f5e00700 1 Processor -- start 2024-04-02T14:32:00.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.175+0000 7fa7f5e00700 1 -- start start 2024-04-02T14:32:00.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.176+0000 7fa7f5e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f0108750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:00.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.176+0000 7fa7f5e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f0108d20 con 0x7fa7f0108330 2024-04-02T14:32:00.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.176+0000 7fa7eea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f0108750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:00.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.176+0000 7fa7eea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f0108750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46460/0 (socket says 172.21.15.62:46460) 2024-04-02T14:32:00.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.176+0000 7fa7eea00700 1 -- 172.21.15.62:0/2580156361 learned_addr learned my addr 172.21.15.62:0/2580156361 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:00.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.177+0000 7fa7eea00700 1 -- 172.21.15.62:0/2580156361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f0109530 con 0x7fa7f0108330 2024-04-02T14:32:00.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.177+0000 7fa7eea00700 1 --2- 172.21.15.62:0/2580156361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f0108750 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00098a0 tx=0x7fa7e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae892e8ea696842 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:00.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.177+0000 7fa7ee000700 1 -- 172.21.15.62:0/2580156361 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7e0004160 con 0x7fa7f0108330 2024-04-02T14:32:00.960 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.177+0000 7fa7ee000700 1 -- 172.21.15.62:0/2580156361 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e0004300 con 0x7fa7f0108330 2024-04-02T14:32:00.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.177+0000 7fa7ee000700 1 -- 172.21.15.62:0/2580156361 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7e00036a0 con 0x7fa7f0108330 2024-04-02T14:32:00.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.178+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2580156361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 msgr2=0x7fa7f0108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:00.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.178+0000 7fa7f5e00700 1 --2- 172.21.15.62:0/2580156361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f0108750 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00098a0 tx=0x7fa7e0009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:00.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.178+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2580156361 shutdown_connections 2024-04-02T14:32:00.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.178+0000 7fa7f5e00700 1 --2- 172.21.15.62:0/2580156361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f0108750 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:00.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.178+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2580156361 >> 172.21.15.62:0/2580156361 conn(0x7fa7f0103910 msgr2=0x7fa7f0105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:00.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.178+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2580156361 shutdown_connections 2024-04-02T14:32:00.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.178+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2580156361 wait complete. 2024-04-02T14:32:00.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.179+0000 7fa7f5e00700 1 Processor -- start 2024-04-02T14:32:00.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.180+0000 7fa7f5e00700 1 -- start start 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.180+0000 7fa7f5e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f007ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.180+0000 7fa7f5e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f0109a60 con 0x7fa7f0108330 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.180+0000 7fa7eea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f007ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.180+0000 7fa7eea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f007ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46462/0 (socket says 172.21.15.62:46462) 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.180+0000 7fa7eea00700 1 -- 172.21.15.62:0/2529160664 learned_addr learned my addr 172.21.15.62:0/2529160664 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.181+0000 7fa7eea00700 1 -- 172.21.15.62:0/2529160664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f007cf90 con 0x7fa7f0108330 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.181+0000 7fa7eea00700 1 --2- 172.21.15.62:0/2529160664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f007ca50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0000c00 tx=0x7fa7e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.181+0000 7fa7f4a00700 1 -- 172.21.15.62:0/2529160664 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7e0003db0 con 0x7fa7f0108330 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.181+0000 7fa7f4a00700 1 -- 172.21.15.62:0/2529160664 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e0011630 con 0x7fa7f0108330 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.181+0000 7fa7f4a00700 1 -- 172.21.15.62:0/2529160664 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7e0011950 con 0x7fa7f0108330 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.181+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2529160664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f007b180 con 0x7fa7f0108330 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.181+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2529160664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f007b6c0 con 0x7fa7f0108330 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.182+0000 7fa7f4a00700 1 -- 172.21.15.62:0/2529160664 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa7e0011af0 con 0x7fa7f0108330 2024-04-02T14:32:00.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.182+0000 7fa7f4a00700 1 -- 172.21.15.62:0/2529160664 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa7e001c450 con 0x7fa7f0108330 2024-04-02T14:32:00.965 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.183+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2529160664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f0063c20 con 0x7fa7f0108330 2024-04-02T14:32:00.965 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.185+0000 7fa7f4a00700 1 -- 172.21.15.62:0/2529160664 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa7e0032020 con 0x7fa7f0108330 2024-04-02T14:32:00.965 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.264+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2529160664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa7f0194fe0 con 0x7fa7f0108330 2024-04-02T14:32:00.965 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.273+0000 7fa7f4a00700 1 -- 172.21.15.62:0/2529160664 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+456 (secure 0 0 0) 0x7fa7f0194fe0 con 0x7fa7f0108330 2024-04-02T14:32:00.965 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.275+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2529160664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 msgr2=0x7fa7f007ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:00.965 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.275+0000 7fa7f5e00700 1 --2- 172.21.15.62:0/2529160664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f007ca50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0000c00 tx=0x7fa7e0004750 comp rx=0 tx=0).stop 2024-04-02T14:32:00.965 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.275+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2529160664 shutdown_connections 2024-04-02T14:32:00.966 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.275+0000 7fa7f5e00700 1 --2- 172.21.15.62:0/2529160664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7f0108330 0x7fa7f007ca50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:00.966 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.275+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2529160664 >> 172.21.15.62:0/2529160664 conn(0x7fa7f0103910 msgr2=0x7fa7f0192690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:00.966 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.275+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2529160664 shutdown_connections 2024-04-02T14:32:00.966 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:00.275+0000 7fa7f5e00700 1 -- 172.21.15.62:0/2529160664 wait complete. 2024-04-02T14:32:00.967 INFO:teuthology.orchestra.run.smithi062.stdout:Generating new minimal ceph.conf... 2024-04-02T14:32:01.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:01 smithi062 ceph-mon[31745]: from='client.? 172.21.15.62:0/1944463581' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:32:02.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.574+0000 7fb3af600700 1 Processor -- start 2024-04-02T14:32:02.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.574+0000 7fb3af600700 1 -- start start 2024-04-02T14:32:02.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.574+0000 7fb3af600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a807b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:02.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.574+0000 7fb3af600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a807b7f0 con 0x7fb3a807cdc0 2024-04-02T14:32:02.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.575+0000 7fb3acc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a807b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:02.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.575+0000 7fb3acc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a807b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46466/0 (socket says 172.21.15.62:46466) 2024-04-02T14:32:02.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.575+0000 7fb3acc00700 1 -- 172.21.15.62:0/622567774 learned_addr learned my addr 172.21.15.62:0/622567774 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:02.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.576+0000 7fb3acc00700 1 -- 172.21.15.62:0/622567774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a807b970 con 0x7fb3a807cdc0 2024-04-02T14:32:02.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.576+0000 7fb3acc00700 1 --2- 172.21.15.62:0/622567774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a807b220 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb3980098a0 tx=0x7fb398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb448dd1b6674894 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:02.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.576+0000 7fb3a7e00700 1 -- 172.21.15.62:0/622567774 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb398004160 con 0x7fb3a807cdc0 2024-04-02T14:32:02.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.576+0000 7fb3a7e00700 1 -- 172.21.15.62:0/622567774 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fb398004300 con 0x7fb3a807cdc0 2024-04-02T14:32:02.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.576+0000 7fb3a7e00700 1 -- 172.21.15.62:0/622567774 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb398011c00 con 0x7fb3a807cdc0 2024-04-02T14:32:02.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.577+0000 7fb3af600700 1 -- 172.21.15.62:0/622567774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 msgr2=0x7fb3a807b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:02.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.577+0000 7fb3af600700 1 --2- 172.21.15.62:0/622567774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a807b220 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb3980098a0 tx=0x7fb398009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:02.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.577+0000 7fb3af600700 1 -- 172.21.15.62:0/622567774 shutdown_connections 2024-04-02T14:32:02.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.577+0000 7fb3af600700 1 --2- 172.21.15.62:0/622567774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a807b220 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:02.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.577+0000 7fb3af600700 1 -- 172.21.15.62:0/622567774 >> 172.21.15.62:0/622567774 conn(0x7fb3a81038e0 msgr2=0x7fb3a8105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:02.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.577+0000 7fb3af600700 1 -- 172.21.15.62:0/622567774 shutdown_connections 2024-04-02T14:32:02.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.577+0000 7fb3af600700 1 -- 172.21.15.62:0/622567774 wait complete. 2024-04-02T14:32:02.539 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.578+0000 7fb3af600700 1 Processor -- start 2024-04-02T14:32:02.539 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.579+0000 7fb3af600700 1 -- start start 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.579+0000 7fb3af600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.579+0000 7fb3af600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a810e9a0 con 0x7fb3a807cdc0 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.579+0000 7fb3acc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.579+0000 7fb3acc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a819e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46472/0 (socket says 172.21.15.62:46472) 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.579+0000 7fb3acc00700 1 -- 172.21.15.62:0/1944463581 learned_addr learned my addr 172.21.15.62:0/1944463581 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.580+0000 7fb3acc00700 1 -- 172.21.15.62:0/1944463581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a819ea30 con 0x7fb3a807cdc0 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.580+0000 7fb3acc00700 1 --2- 172.21.15.62:0/1944463581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a819e4f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb398000c00 tx=0x7fb398011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.580+0000 7fb3a6000700 1 -- 172.21.15.62:0/1944463581 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb398003b20 con 0x7fb3a807cdc0 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.580+0000 7fb3a6000700 1 -- 172.21.15.62:0/1944463581 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fb398003cc0 con 0x7fb3a807cdc0 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.580+0000 7fb3a6000700 1 -- 172.21.15.62:0/1944463581 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3980249a0 con 0x7fb3a807cdc0 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.580+0000 7fb3af600700 1 -- 172.21.15.62:0/1944463581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a819ecc0 con 0x7fb3a807cdc0 2024-04-02T14:32:02.540 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.580+0000 7fb3af600700 1 -- 172.21.15.62:0/1944463581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a81a19b0 con 0x7fb3a807cdc0 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.581+0000 7fb3a6000700 1 -- 172.21.15.62:0/1944463581 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb398022020 con 0x7fb3a807cdc0 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.581+0000 7fb3a6000700 1 -- 172.21.15.62:0/1944463581 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb39802ddf0 con 0x7fb3a807cdc0 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.582+0000 7fb3af600700 1 -- 172.21.15.62:0/1944463581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a8063c20 con 0x7fb3a807cdc0 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.585+0000 7fb3a6000700 1 -- 172.21.15.62:0/1944463581 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb398042dd0 con 0x7fb3a807cdc0 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.662+0000 7fb3af600700 1 -- 172.21.15.62:0/1944463581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb3a81a1ef0 con 0x7fb3a807cdc0 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.662+0000 7fb3a6000700 1 -- 172.21.15.62:0/1944463581 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb398031030 con 0x7fb3a807cdc0 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.664+0000 7fb3af600700 1 -- 172.21.15.62:0/1944463581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 msgr2=0x7fb3a819e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.664+0000 7fb3af600700 1 --2- 172.21.15.62:0/1944463581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a819e4f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb398000c00 tx=0x7fb398011c80 comp rx=0 tx=0).stop 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.664+0000 7fb3af600700 1 -- 172.21.15.62:0/1944463581 shutdown_connections 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.664+0000 7fb3af600700 1 --2- 172.21.15.62:0/1944463581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3a807cdc0 0x7fb3a819e4f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.664+0000 7fb3af600700 1 -- 172.21.15.62:0/1944463581 >> 172.21.15.62:0/1944463581 conn(0x7fb3a81038e0 msgr2=0x7fb3a81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.664+0000 7fb3af600700 1 -- 172.21.15.62:0/1944463581 shutdown_connections 2024-04-02T14:32:02.541 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:01.664+0000 7fb3af600700 1 -- 172.21.15.62:0/1944463581 wait complete. 2024-04-02T14:32:02.542 INFO:teuthology.orchestra.run.smithi062.stdout:Restarting the monitor... 2024-04-02T14:32:02.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:02 smithi062 systemd[1]: Stopping Ceph mon.a for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:32:03.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[31741]: 2024-04-02T14:32:03.085+0000 7f840cc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:32:03.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[31741]: 2024-04-02T14:32:03.085+0000 7f840cc00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-02T14:32:03.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:03 smithi062 podman[31987]: 2024-04-02 14:32:03.126998325 +0000 UTC m=+0.152822229 container died 5b8f7ba934b7aa15257503266da8ec9c47a52d273a2490f4a2c3db43ba4098cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-02T14:32:14.299 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:13 smithi062 podman[31987]: 2024-04-02 14:32:13.925207778 +0000 UTC m=+10.951031707 container cleanup 5b8f7ba934b7aa15257503266da8ec9c47a52d273a2490f4a2c3db43ba4098cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-02T14:32:14.299 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:13 smithi062 bash[31987]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a 2024-04-02T14:32:15.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:15 smithi062 podman[32001]: 2024-04-02 14:32:15.125213228 +0000 UTC m=+11.976236526 container remove 5b8f7ba934b7aa15257503266da8ec9c47a52d273a2490f4a2c3db43ba4098cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-04-02T14:32:15.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:15 smithi062 systemd[1]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.a.service: Deactivated successfully. 2024-04-02T14:32:15.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:15 smithi062 systemd[1]: Stopped Ceph mon.a for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:32:15.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:15 smithi062 systemd[1]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.a.service: Consumed 2.032s CPU time. 2024-04-02T14:32:15.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:15 smithi062 systemd[1]: Starting Ceph mon.a for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:32:16.182 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:15 smithi062 podman[32097]: 2024-04-02 14:32:15.909271122 +0000 UTC m=+0.122352167 container create 6078bd3e7250e50ac00fc26479425fe00738ca02c598d3f54db0d0f07e79d1c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-02T14:32:16.182 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:15 smithi062 podman[32097]: 2024-04-02 14:32:15.818962909 +0000 UTC m=+0.032043917 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:32:16.280 INFO:teuthology.orchestra.run.smithi062.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-02T14:32:16.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 podman[32097]: 2024-04-02 14:32:16.179732541 +0000 UTC m=+0.392813593 container init 6078bd3e7250e50ac00fc26479425fe00738ca02c598d3f54db0d0f07e79d1c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:32:16.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 podman[32097]: 2024-04-02 14:32:16.191643274 +0000 UTC m=+0.404724352 container start 6078bd3e7250e50ac00fc26479425fe00738ca02c598d3f54db0d0f07e79d1c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326) 2024-04-02T14:32:16.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T14:32:16.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev), process ceph-mon, pid 2 2024-04-02T14:32:16.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: pidfile_write: ignore empty --pid-file 2024-04-02T14:32:16.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: load: jerasure load: lrc 2024-04-02T14:32:16.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: RocksDB version: 7.9.2 2024-04-02T14:32:16.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Git sha 0 2024-04-02T14:32:16.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Compile date 2024-04-02 01:02:34 2024-04-02T14:32:16.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: DB SUMMARY 2024-04-02T14:32:16.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: DB Session ID: CLPA30Q1JFGSCC8HCCSO 2024-04-02T14:32:16.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: CURRENT file: CURRENT 2024-04-02T14:32:16.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T14:32:16.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-02T14:32:16.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-02T14:32:16.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106504 ; 2024-04-02T14:32:16.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.error_if_exists: 0 2024-04-02T14:32:16.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.create_if_missing: 0 2024-04-02T14:32:16.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.paranoid_checks: 1 2024-04-02T14:32:16.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T14:32:16.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T14:32:16.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T14:32:16.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.env: 0x55f541754c60 2024-04-02T14:32:16.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T14:32:16.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.info_log: 0x55f543233b20 2024-04-02T14:32:16.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T14:32:16.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.statistics: (nil) 2024-04-02T14:32:16.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.use_fsync: 0 2024-04-02T14:32:16.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_log_file_size: 0 2024-04-02T14:32:16.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T14:32:16.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T14:32:16.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T14:32:16.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T14:32:16.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.allow_fallocate: 1 2024-04-02T14:32:16.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T14:32:16.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T14:32:16.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.use_direct_reads: 0 2024-04-02T14:32:16.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T14:32:16.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T14:32:16.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.db_log_dir: 2024-04-02T14:32:16.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.wal_dir: 2024-04-02T14:32:16.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T14:32:16.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T14:32:16.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T14:32:16.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T14:32:16.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T14:32:16.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T14:32:16.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T14:32:16.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T14:32:16.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.write_buffer_manager: 0x55f5432041e0 2024-04-02T14:32:16.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T14:32:16.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T14:32:16.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T14:32:16.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T14:32:16.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T14:32:16.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T14:32:16.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T14:32:16.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T14:32:16.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.unordered_write: 0 2024-04-02T14:32:16.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T14:32:16.451 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T14:32:16.451 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T14:32:16.451 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T14:32:16.451 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.row_cache: None 2024-04-02T14:32:16.451 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.wal_filter: None 2024-04-02T14:32:16.451 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T14:32:16.451 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T14:32:16.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.two_write_queues: 0 2024-04-02T14:32:16.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T14:32:16.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.wal_compression: 0 2024-04-02T14:32:16.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.atomic_flush: 0 2024-04-02T14:32:16.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T14:32:16.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T14:32:16.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T14:32:16.453 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.log_readahead_size: 0 2024-04-02T14:32:16.453 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T14:32:16.453 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T14:32:16.453 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T14:32:16.453 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T14:32:16.453 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T14:32:16.453 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T14:32:16.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T14:32:16.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_background_jobs: 2 2024-04-02T14:32:16.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_background_compactions: -1 2024-04-02T14:32:16.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_subcompactions: 1 2024-04-02T14:32:16.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T14:32:16.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T14:32:16.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T14:32:16.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T14:32:16.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T14:32:16.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T14:32:16.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T14:32:16.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T14:32:16.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_open_files: -1 2024-04-02T14:32:16.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T14:32:16.456 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T14:32:16.456 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T14:32:16.456 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T14:32:16.456 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_background_flushes: -1 2024-04-02T14:32:16.456 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Compression algorithms supported: 2024-04-02T14:32:16.456 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T14:32:16.456 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: kZSTD supported: 0 2024-04-02T14:32:16.456 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: kXpressCompression supported: 0 2024-04-02T14:32:16.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T14:32:16.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: kZlibCompression supported: 1 2024-04-02T14:32:16.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: kSnappyCompression supported: 1 2024-04-02T14:32:16.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: kLZ4Compression supported: 1 2024-04-02T14:32:16.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: kBZip2Compression supported: 0 2024-04-02T14:32:16.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T14:32:16.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T14:32:16.458 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-02T14:32:16.458 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T14:32:16.458 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T14:32:16.458 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.merge_operator: 2024-04-02T14:32:16.458 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_filter: None 2024-04-02T14:32:16.458 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T14:32:16.459 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T14:32:16.459 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T14:32:16.459 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T14:32:16.459 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f543233d00) 2024-04-02T14:32:16.459 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-04-02T14:32:16.459 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T14:32:16.460 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T14:32:16.460 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-04-02T14:32:16.460 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_type: 0 2024-04-02T14:32:16.460 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_index_type: 0 2024-04-02T14:32:16.460 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_shortening: 1 2024-04-02T14:32:16.460 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T14:32:16.460 INFO:journalctl@ceph.mon.a.smithi062.stdout: checksum: 4 2024-04-02T14:32:16.461 INFO:journalctl@ceph.mon.a.smithi062.stdout: no_block_cache: 0 2024-04-02T14:32:16.461 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache: 0x55f543237090 2024-04-02T14:32:16.461 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-04-02T14:32:16.461 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_options: 2024-04-02T14:32:16.461 INFO:journalctl@ceph.mon.a.smithi062.stdout: capacity : 536870912 2024-04-02T14:32:16.461 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_shard_bits : 4 2024-04-02T14:32:16.461 INFO:journalctl@ceph.mon.a.smithi062.stdout: strict_capacity_limit : 0 2024-04-02T14:32:16.461 INFO:journalctl@ceph.mon.a.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-04-02T14:32:16.462 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_compressed: (nil) 2024-04-02T14:32:16.462 INFO:journalctl@ceph.mon.a.smithi062.stdout: persistent_cache: (nil) 2024-04-02T14:32:16.462 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size: 4096 2024-04-02T14:32:16.462 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size_deviation: 10 2024-04-02T14:32:16.462 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_restart_interval: 16 2024-04-02T14:32:16.462 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_block_restart_interval: 1 2024-04-02T14:32:16.463 INFO:journalctl@ceph.mon.a.smithi062.stdout: metadata_block_size: 4096 2024-04-02T14:32:16.463 INFO:journalctl@ceph.mon.a.smithi062.stdout: partition_filters: 0 2024-04-02T14:32:16.463 INFO:journalctl@ceph.mon.a.smithi062.stdout: use_delta_encoding: 1 2024-04-02T14:32:16.463 INFO:journalctl@ceph.mon.a.smithi062.stdout: filter_policy: bloomfilter 2024-04-02T14:32:16.463 INFO:journalctl@ceph.mon.a.smithi062.stdout: whole_key_filtering: 1 2024-04-02T14:32:16.463 INFO:journalctl@ceph.mon.a.smithi062.stdout: verify_compression: 0 2024-04-02T14:32:16.463 INFO:journalctl@ceph.mon.a.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-04-02T14:32:16.463 INFO:journalctl@ceph.mon.a.smithi062.stdout: format_version: 5 2024-04-02T14:32:16.464 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_index_compression: 1 2024-04-02T14:32:16.464 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_align: 0 2024-04-02T14:32:16.464 INFO:journalctl@ceph.mon.a.smithi062.stdout: max_auto_readahead_size: 262144 2024-04-02T14:32:16.464 INFO:journalctl@ceph.mon.a.smithi062.stdout: prepopulate_block_cache: 0 2024-04-02T14:32:16.464 INFO:journalctl@ceph.mon.a.smithi062.stdout: initial_auto_readahead_size: 8192 2024-04-02T14:32:16.464 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T14:32:16.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T14:32:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T14:32:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compression: NoCompression 2024-04-02T14:32:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T14:32:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T14:32:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T14:32:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.num_levels: 7 2024-04-02T14:32:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T14:32:16.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T14:32:16.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T14:32:16.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T14:32:16.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T14:32:16.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T14:32:16.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T14:32:16.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T14:32:16.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T14:32:16.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T14:32:16.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T14:32:16.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T14:32:16.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T14:32:16.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T14:32:16.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T14:32:16.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T14:32:16.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T14:32:16.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T14:32:16.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T14:32:16.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T14:32:16.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T14:32:16.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T14:32:16.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T14:32:16.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T14:32:16.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T14:32:16.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T14:32:16.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T14:32:16.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T14:32:16.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T14:32:16.470 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T14:32:16.470 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T14:32:16.471 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T14:32:16.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T14:32:16.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T14:32:16.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T14:32:16.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T14:32:16.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T14:32:16.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T14:32:16.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T14:32:16.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T14:32:16.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T14:32:16.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T14:32:16.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T14:32:16.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T14:32:16.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T14:32:16.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T14:32:16.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T14:32:16.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T14:32:16.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T14:32:16.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T14:32:16.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T14:32:16.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T14:32:16.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T14:32:16.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-02T14:32:16.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.inplace_update_support: 0 2024-04-02T14:32:16.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T14:32:16.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T14:32:16.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T14:32:16.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T14:32:16.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.bloom_locality: 0 2024-04-02T14:32:16.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.max_successive_merges: 0 2024-04-02T14:32:16.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T14:32:16.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T14:32:16.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T14:32:16.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T14:32:16.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.ttl: 2592000 2024-04-02T14:32:16.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T14:32:16.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T14:32:16.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T14:32:16.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.enable_blob_files: false 2024-04-02T14:32:16.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.min_blob_size: 0 2024-04-02T14:32:16.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T14:32:16.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T14:32:16.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T14:32:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T14:32:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T14:32:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T14:32:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T14:32:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T14:32:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-02T14:32:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-02T14:32:16.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e7518d78-c546-4452-bf30-fc71edbf3083 2024-04-02T14:32:16.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712068336248610, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-02T14:32:16.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-02T14:32:16.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712068336250131, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101545, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99421, "index_size": 316, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14536, "raw_average_key_size": 49, "raw_value_size": 91503, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712068336, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e7518d78-c546-4452-bf30-fc71edbf3083", "db_session_id": "CLPA30Q1JFGSCC8HCCSO", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-02T14:32:16.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712068336250296, "job": 1, "event": "recovery_finished"} 2024-04-02T14:32:16.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-02T14:32:16.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T14:32:16.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f543326000 2024-04-02T14:32:16.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: DB pointer 0x55f543310000 2024-04-02T14:32:16.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T14:32:16.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T14:32:16.480 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** DB Stats ** 2024-04-02T14:32:16.480 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T14:32:16.480 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T14:32:16.481 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T14:32:16.481 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T14:32:16.481 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T14:32:16.481 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T14:32:16.481 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T14:32:16.481 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-04-02T14:32:16.481 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-04-02T14:32:16.481 INFO:journalctl@ceph.mon.a.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T14:32:16.482 INFO:journalctl@ceph.mon.a.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T14:32:16.482 INFO:journalctl@ceph.mon.a.smithi062.stdout: L0 2/0 101.03 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T14:32:16.482 INFO:journalctl@ceph.mon.a.smithi062.stdout: Sum 2/0 101.03 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T14:32:16.482 INFO:journalctl@ceph.mon.a.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T14:32:16.482 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-04-02T14:32:16.482 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-04-02T14:32:16.482 INFO:journalctl@ceph.mon.a.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T14:32:16.483 INFO:journalctl@ceph.mon.a.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T14:32:16.483 INFO:journalctl@ceph.mon.a.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 102.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T14:32:16.483 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-04-02T14:32:16.483 INFO:journalctl@ceph.mon.a.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T14:32:16.483 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-04-02T14:32:16.483 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T14:32:16.483 INFO:journalctl@ceph.mon.a.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T14:32:16.483 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T14:32:16.484 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T14:32:16.484 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T14:32:16.484 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T14:32:16.484 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative compaction: 0.00 GB write, 14.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T14:32:16.484 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval compaction: 0.00 GB write, 14.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T14:32:16.484 INFO:journalctl@ceph.mon.a.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T14:32:16.484 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache BinnedLRUCache@0x55f543237090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-02T14:32:16.485 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-02T14:32:16.485 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-04-02T14:32:16.485 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T14:32:16.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: starting mon.a rank 0 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:32:16.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: mon.a@-1(???) e1 preinit fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:32:16.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: mon.a@-1(???).mds e1 new map 2024-04-02T14:32:16.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: mon.a@-1(???).mds e1 print_map 2024-04-02T14:32:16.485 INFO:journalctl@ceph.mon.a.smithi062.stdout: e1 2024-04-02T14:32:16.486 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T14:32:16.486 INFO:journalctl@ceph.mon.a.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T14:32:16.486 INFO:journalctl@ceph.mon.a.smithi062.stdout: legacy client fscid: -1 2024-04-02T14:32:16.486 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-04-02T14:32:16.486 INFO:journalctl@ceph.mon.a.smithi062.stdout: No filesystems configured 2024-04-02T14:32:16.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T14:32:16.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T14:32:16.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T14:32:16.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T14:32:16.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-02T14:32:16.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 bash[32097]: 6078bd3e7250e50ac00fc26479425fe00738ca02c598d3f54db0d0f07e79d1c2 2024-04-02T14:32:16.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 systemd[1]: Started Ceph mon.a for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:32:16.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-02T14:32:16.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: monmap epoch 1 2024-04-02T14:32:16.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:32:16.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: last_changed 2024-04-02T14:31:53.643129+0000 2024-04-02T14:32:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: created 2024-04-02T14:31:53.643129+0000 2024-04-02T14:32:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: min_mon_release 19 (squid) 2024-04-02T14:32:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: election_strategy: 1 2024-04-02T14:32:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-02T14:32:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: fsmap 2024-04-02T14:32:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: osdmap e1: 0 total, 0 up, 0 in 2024-04-02T14:32:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:16 smithi062 ceph-mon[32116]: mgrmap e1: no daemons active 2024-04-02T14:32:17.955 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.983+0000 7fe253000700 1 Processor -- start 2024-04-02T14:32:17.955 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.983+0000 7fe253000700 1 -- start start 2024-04-02T14:32:17.955 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.984+0000 7fe253000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c10ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:17.956 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.984+0000 7fe253000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c10f2f0 con 0x7fe24c10e900 2024-04-02T14:32:17.956 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.984+0000 7fe24be00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c10ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:17.956 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.984+0000 7fe24be00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c10ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59100/0 (socket says 172.21.15.62:59100) 2024-04-02T14:32:17.956 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.984+0000 7fe24be00700 1 -- 172.21.15.62:0/696119075 learned_addr learned my addr 172.21.15.62:0/696119075 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:17.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.985+0000 7fe24be00700 1 -- 172.21.15.62:0/696119075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c10fab0 con 0x7fe24c10e900 2024-04-02T14:32:17.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.985+0000 7fe24be00700 1 --2- 172.21.15.62:0/696119075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c10ed20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009e80 tx=0x7fe23c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce81c57d9995b317 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:17.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.986+0000 7fe24b400700 1 -- 172.21.15.62:0/696119075 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe23c017070 con 0x7fe24c10e900 2024-04-02T14:32:17.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.986+0000 7fe24b400700 1 -- 172.21.15.62:0/696119075 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fe23c00b8c0 con 0x7fe24c10e900 2024-04-02T14:32:17.958 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.986+0000 7fe24b400700 1 -- 172.21.15.62:0/696119075 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe23c004620 con 0x7fe24c10e900 2024-04-02T14:32:17.958 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.986+0000 7fe253000700 1 -- 172.21.15.62:0/696119075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 msgr2=0x7fe24c10ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:17.958 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.986+0000 7fe253000700 1 --2- 172.21.15.62:0/696119075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c10ed20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009e80 tx=0x7fe23c00b330 comp rx=0 tx=0).stop 2024-04-02T14:32:17.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.987+0000 7fe253000700 1 -- 172.21.15.62:0/696119075 shutdown_connections 2024-04-02T14:32:17.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.987+0000 7fe253000700 1 --2- 172.21.15.62:0/696119075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c10ed20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:17.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.987+0000 7fe253000700 1 -- 172.21.15.62:0/696119075 >> 172.21.15.62:0/696119075 conn(0x7fe24c07d9e0 msgr2=0x7fe24c07fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:17.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.987+0000 7fe253000700 1 -- 172.21.15.62:0/696119075 shutdown_connections 2024-04-02T14:32:17.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.987+0000 7fe253000700 1 -- 172.21.15.62:0/696119075 wait complete. 2024-04-02T14:32:17.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.988+0000 7fe253000700 1 Processor -- start 2024-04-02T14:32:17.959 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.988+0000 7fe253000700 1 -- start start 2024-04-02T14:32:17.960 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.988+0000 7fe253000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c1a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:17.960 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.988+0000 7fe253000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c112e60 con 0x7fe24c10e900 2024-04-02T14:32:17.960 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.988+0000 7fe24be00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c1a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:17.960 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.989+0000 7fe24be00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c1a28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59110/0 (socket says 172.21.15.62:59110) 2024-04-02T14:32:17.962 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.989+0000 7fe24be00700 1 -- 172.21.15.62:0/2519979486 learned_addr learned my addr 172.21.15.62:0/2519979486 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:17.962 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.989+0000 7fe24be00700 1 -- 172.21.15.62:0/2519979486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c1a2e10 con 0x7fe24c10e900 2024-04-02T14:32:17.962 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.989+0000 7fe24be00700 1 --2- 172.21.15.62:0/2519979486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c1a28d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009fd0 tx=0x7fe23c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:17.962 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.989+0000 7fe249600700 1 -- 172.21.15.62:0/2519979486 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe23c021020 con 0x7fe24c10e900 2024-04-02T14:32:17.962 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.990+0000 7fe253000700 1 -- 172.21.15.62:0/2519979486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24c1a30a0 con 0x7fe24c10e900 2024-04-02T14:32:17.962 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.990+0000 7fe249600700 1 -- 172.21.15.62:0/2519979486 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fe23c00bde0 con 0x7fe24c10e900 2024-04-02T14:32:17.962 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.990+0000 7fe249600700 1 -- 172.21.15.62:0/2519979486 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe23c017040 con 0x7fe24c10e900 2024-04-02T14:32:17.962 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.990+0000 7fe253000700 1 -- 172.21.15.62:0/2519979486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24c1a5d90 con 0x7fe24c10e900 2024-04-02T14:32:17.962 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.990+0000 7fe249600700 1 -- 172.21.15.62:0/2519979486 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe23c024020 con 0x7fe24c10e900 2024-04-02T14:32:17.962 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.990+0000 7fe249600700 1 -- 172.21.15.62:0/2519979486 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe23c00f050 con 0x7fe24c10e900 2024-04-02T14:32:17.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.991+0000 7fe253000700 1 -- 172.21.15.62:0/2519979486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24c112fe0 con 0x7fe24c10e900 2024-04-02T14:32:17.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:16.994+0000 7fe249600700 1 -- 172.21.15.62:0/2519979486 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe23c02b020 con 0x7fe24c10e900 2024-04-02T14:32:17.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:17.071+0000 7fe253000700 1 -- 172.21.15.62:0/2519979486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe24c1a6360 con 0x7fe24c10e900 2024-04-02T14:32:17.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:17.076+0000 7fe249600700 1 -- 172.21.15.62:0/2519979486 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe23c00b860 con 0x7fe24c10e900 2024-04-02T14:32:17.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:17.077+0000 7fe253000700 1 -- 172.21.15.62:0/2519979486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 msgr2=0x7fe24c1a28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:17.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:17.077+0000 7fe253000700 1 --2- 172.21.15.62:0/2519979486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c1a28d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009fd0 tx=0x7fe23c003d00 comp rx=0 tx=0).stop 2024-04-02T14:32:17.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:17.077+0000 7fe253000700 1 -- 172.21.15.62:0/2519979486 shutdown_connections 2024-04-02T14:32:17.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:17.077+0000 7fe253000700 1 --2- 172.21.15.62:0/2519979486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c10e900 0x7fe24c1a28d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:17.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:17.077+0000 7fe253000700 1 -- 172.21.15.62:0/2519979486 >> 172.21.15.62:0/2519979486 conn(0x7fe24c07d9e0 msgr2=0x7fe24c07f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:17.963 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:17.077+0000 7fe253000700 1 -- 172.21.15.62:0/2519979486 shutdown_connections 2024-04-02T14:32:17.964 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:17.078+0000 7fe253000700 1 -- 172.21.15.62:0/2519979486 wait complete. 2024-04-02T14:32:17.964 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-02T14:32:17.986 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:32:17.986 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mgr... 2024-04-02T14:32:17.988 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-02T14:32:17.989 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-02T14:32:18.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:18 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2519979486' entity='client.admin' 2024-04-02T14:32:18.442 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.x 2024-04-02T14:32:18.442 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.x.service: Unit ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.x.service not loaded. 2024-04-02T14:32:18.716 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4.target.wants/ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.x.service → /etc/systemd/system/ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@.service. 2024-04-02T14:32:19.095 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:18 smithi062 systemd[1]: Starting Ceph mgr.x for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:32:19.401 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:19 smithi062 podman[32331]: 2024-04-02 14:32:19.104293399 +0000 UTC m=+0.126272581 container create 4e5db03de101204795db3714573125b818cd25ff55604c51c55fe0fb7d772d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-04-02T14:32:19.402 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:19 smithi062 podman[32331]: 2024-04-02 14:32:19.012198547 +0000 UTC m=+0.034177706 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:32:19.402 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:19 smithi062 podman[32331]: 2024-04-02 14:32:19.399680221 +0000 UTC m=+0.421659410 container init 4e5db03de101204795db3714573125b818cd25ff55604c51c55fe0fb7d772d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x, ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-02T14:32:19.548 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T14:32:19.549 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-04-02T14:32:19.567 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T14:32:19.567 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-04-02T14:32:19.567 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-04-02T14:32:19.567 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T14:32:19.598 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T14:32:19.598 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-04-02T14:32:19.619 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T14:32:19.619 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-04-02T14:32:19.619 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-04-02T14:32:19.619 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-02T14:32:19.620 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr to start... 2024-04-02T14:32:19.620 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr... 2024-04-02T14:32:19.775 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:19 smithi062 podman[32331]: 2024-04-02 14:32:19.411483358 +0000 UTC m=+0.433462513 container start 4e5db03de101204795db3714573125b818cd25ff55604c51c55fe0fb7d772d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-02T14:32:19.775 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:19 smithi062 bash[32331]: 4e5db03de101204795db3714573125b818cd25ff55604c51c55fe0fb7d772d59 2024-04-02T14:32:19.775 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:19 smithi062 systemd[1]: Started Ceph mgr.x for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:32:20.298 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:20 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:20.257+0000 7f0c37fb0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T14:32:20.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:20 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/483587569' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T14:32:20.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:20 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:20.396+0000 7f0c37fb0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-04-02T14:32:21.008 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T14:32:21.009 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T14:32:21.010 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T14:32:21.011 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T14:31:58.064994+0000", 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.204+0000 7f0a25a00700 1 Processor -- start 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.204+0000 7f0a25a00700 1 -- start start 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.204+0000 7f0a25a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.204+0000 7f0a25a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2007b790 con 0x7f0a2007cd60 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.205+0000 7f0a1ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.205+0000 7f0a1ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2007b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59138/0 (socket says 172.21.15.62:59138) 2024-04-02T14:32:21.012 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.205+0000 7f0a1ea00700 1 -- 172.21.15.62:0/2535474994 learned_addr learned my addr 172.21.15.62:0/2535474994 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.205+0000 7f0a1ea00700 1 -- 172.21.15.62:0/2535474994 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2007b910 con 0x7f0a2007cd60 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.206+0000 7f0a1ea00700 1 --2- 172.21.15.62:0/2535474994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2007b1c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0a100098a0 tx=0x7f0a10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4c3e8372092441 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.206+0000 7f0a1d600700 1 -- 172.21.15.62:0/2535474994 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a10004160 con 0x7f0a2007cd60 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.206+0000 7f0a1d600700 1 -- 172.21.15.62:0/2535474994 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f0a10004300 con 0x7f0a2007cd60 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.206+0000 7f0a1d600700 1 -- 172.21.15.62:0/2535474994 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a10011c00 con 0x7f0a2007cd60 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.207+0000 7f0a25a00700 1 -- 172.21.15.62:0/2535474994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 msgr2=0x7f0a2007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.207+0000 7f0a25a00700 1 --2- 172.21.15.62:0/2535474994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2007b1c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0a100098a0 tx=0x7f0a10009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.207+0000 7f0a25a00700 1 -- 172.21.15.62:0/2535474994 shutdown_connections 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.207+0000 7f0a25a00700 1 --2- 172.21.15.62:0/2535474994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2007b1c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.207+0000 7f0a25a00700 1 -- 172.21.15.62:0/2535474994 >> 172.21.15.62:0/2535474994 conn(0x7f0a20103890 msgr2=0x7f0a20105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.207+0000 7f0a25a00700 1 -- 172.21.15.62:0/2535474994 shutdown_connections 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.207+0000 7f0a25a00700 1 -- 172.21.15.62:0/2535474994 wait complete. 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.208+0000 7f0a25a00700 1 Processor -- start 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.208+0000 7f0a25a00700 1 -- start start 2024-04-02T14:32:21.013 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.209+0000 7f0a25a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.209+0000 7f0a25a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2010e940 con 0x7f0a2007cd60 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.209+0000 7f0a1ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.209+0000 7f0a1ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2019e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59140/0 (socket says 172.21.15.62:59140) 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.209+0000 7f0a1ea00700 1 -- 172.21.15.62:0/483587569 learned_addr learned my addr 172.21.15.62:0/483587569 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.209+0000 7f0a1ea00700 1 -- 172.21.15.62:0/483587569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2019e9f0 con 0x7f0a2007cd60 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.210+0000 7f0a1ea00700 1 --2- 172.21.15.62:0/483587569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2019e4b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0a10003fb0 tx=0x7f0a10011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.210+0000 7f0a17400700 1 -- 172.21.15.62:0/483587569 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a100039c0 con 0x7f0a2007cd60 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.210+0000 7f0a17400700 1 -- 172.21.15.62:0/483587569 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f0a10003b60 con 0x7f0a2007cd60 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.210+0000 7f0a17400700 1 -- 172.21.15.62:0/483587569 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a1001a6b0 con 0x7f0a2007cd60 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.210+0000 7f0a25a00700 1 -- 172.21.15.62:0/483587569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2019ec80 con 0x7f0a2007cd60 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.210+0000 7f0a25a00700 1 -- 172.21.15.62:0/483587569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a201a1970 con 0x7f0a2007cd60 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.211+0000 7f0a17400700 1 -- 172.21.15.62:0/483587569 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0a1001acf0 con 0x7f0a2007cd60 2024-04-02T14:32:21.014 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.211+0000 7f0a17400700 1 -- 172.21.15.62:0/483587569 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0a100193c0 con 0x7f0a2007cd60 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.211+0000 7f0a25a00700 1 -- 172.21.15.62:0/483587569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a20063c20 con 0x7f0a2007cd60 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.214+0000 7f0a17400700 1 -- 172.21.15.62:0/483587569 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0a10024770 con 0x7f0a2007cd60 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.297+0000 7f0a25a00700 1 -- 172.21.15.62:0/483587569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0a201a1b40 con 0x7f0a2007cd60 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.298+0000 7f0a17400700 1 -- 172.21.15.62:0/483587569 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0a10003d10 con 0x7f0a2007cd60 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.300+0000 7f0a25a00700 1 -- 172.21.15.62:0/483587569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 msgr2=0x7f0a2019e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.300+0000 7f0a25a00700 1 --2- 172.21.15.62:0/483587569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2019e4b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0a10003fb0 tx=0x7f0a10011bd0 comp rx=0 tx=0).stop 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.300+0000 7f0a25a00700 1 -- 172.21.15.62:0/483587569 shutdown_connections 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.300+0000 7f0a25a00700 1 --2- 172.21.15.62:0/483587569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a2007cd60 0x7f0a2019e4b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.300+0000 7f0a25a00700 1 -- 172.21.15.62:0/483587569 >> 172.21.15.62:0/483587569 conn(0x7f0a20103890 msgr2=0x7f0a201043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.301+0000 7f0a25a00700 1 -- 172.21.15.62:0/483587569 shutdown_connections 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:20.301+0000 7f0a25a00700 1 -- 172.21.15.62:0/483587569 wait complete. 2024-04-02T14:32:21.015 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (1/15)... 2024-04-02T14:32:21.572 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:21 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:21.257+0000 7f0c37fb0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T14:32:21.572 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:21 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:21.383+0000 7f0c37fb0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T14:32:21.572 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:21 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:21.570+0000 7f0c37fb0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T14:32:21.844 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:21 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:21.698+0000 7f0c37fb0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T14:32:21.844 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:21 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:21.811+0000 7f0c37fb0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T14:32:22.411 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:22 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:22.150+0000 7f0c37fb0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T14:32:22.411 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:22 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:22.294+0000 7f0c37fb0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T14:32:22.758 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:22 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:22.410+0000 7f0c37fb0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T14:32:22.758 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:22 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:22.536+0000 7f0c37fb0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T14:32:22.759 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:22 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:22.756+0000 7f0c37fb0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T14:32:24.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:23 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3009535467' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T14:32:24.487 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:24 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:24.152+0000 7f0c37fb0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T14:32:24.487 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:24 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:24.332+0000 7f0c37fb0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T14:32:24.505 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:32:24.505 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-04-02T14:32:24.505 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:32:24.505 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T14:32:24.505 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T14:32:24.505 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T14:32:24.505 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T14:32:24.506 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:24.506 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T14:32:24.506 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T14:32:24.506 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-04-02T14:32:24.506 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:24.506 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T14:32:24.506 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T14:32:24.506 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-02T14:32:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T14:32:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T14:32:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T14:32:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T14:32:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T14:32:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T14:32:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T14:32:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T14:31:58.064994+0000", 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.669+0000 7fbf19600700 1 Processor -- start 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.669+0000 7fbf19600700 1 -- start start 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.669+0000 7fbf19600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf14107600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.669+0000 7fbf19600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14107bd0 con 0x7fbf141071e0 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.670+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf14107600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.670+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf14107600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59936/0 (socket says 172.21.15.62:59936) 2024-04-02T14:32:24.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.670+0000 7fbf12a00700 1 -- 172.21.15.62:0/3117920611 learned_addr learned my addr 172.21.15.62:0/3117920611 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.670+0000 7fbf12a00700 1 -- 172.21.15.62:0/3117920611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf141083e0 con 0x7fbf141071e0 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.671+0000 7fbf12a00700 1 --2- 172.21.15.62:0/3117920611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf14107600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf040098a0 tx=0x7fbf04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a719640fcb8d9392 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.671+0000 7fbf11600700 1 -- 172.21.15.62:0/3117920611 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf04004160 con 0x7fbf141071e0 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.671+0000 7fbf11600700 1 -- 172.21.15.62:0/3117920611 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fbf04004300 con 0x7fbf141071e0 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.671+0000 7fbf11600700 1 -- 172.21.15.62:0/3117920611 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf04011c00 con 0x7fbf141071e0 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.671+0000 7fbf19600700 1 -- 172.21.15.62:0/3117920611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 msgr2=0x7fbf14107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.671+0000 7fbf19600700 1 --2- 172.21.15.62:0/3117920611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf14107600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf040098a0 tx=0x7fbf04009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.672+0000 7fbf19600700 1 -- 172.21.15.62:0/3117920611 shutdown_connections 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.672+0000 7fbf19600700 1 --2- 172.21.15.62:0/3117920611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf14107600 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.672+0000 7fbf19600700 1 -- 172.21.15.62:0/3117920611 >> 172.21.15.62:0/3117920611 conn(0x7fbf14102780 msgr2=0x7fbf14104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.672+0000 7fbf19600700 1 -- 172.21.15.62:0/3117920611 shutdown_connections 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.672+0000 7fbf19600700 1 -- 172.21.15.62:0/3117920611 wait complete. 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.673+0000 7fbf19600700 1 Processor -- start 2024-04-02T14:32:24.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.673+0000 7fbf19600700 1 -- start start 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.673+0000 7fbf19600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf1419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.673+0000 7fbf19600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14062bb0 con 0x7fbf141071e0 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.673+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf1419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.673+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf1419a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59948/0 (socket says 172.21.15.62:59948) 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.674+0000 7fbf12a00700 1 -- 172.21.15.62:0/3009535467 learned_addr learned my addr 172.21.15.62:0/3009535467 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.674+0000 7fbf12a00700 1 -- 172.21.15.62:0/3009535467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1419a680 con 0x7fbf141071e0 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.674+0000 7fbf12a00700 1 --2- 172.21.15.62:0/3009535467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf1419a140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf04000c00 tx=0x7fbf04011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.674+0000 7fbf0b400700 1 -- 172.21.15.62:0/3009535467 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf04003b20 con 0x7fbf141071e0 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.674+0000 7fbf0b400700 1 -- 172.21.15.62:0/3009535467 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fbf04003cc0 con 0x7fbf141071e0 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.674+0000 7fbf19600700 1 -- 172.21.15.62:0/3009535467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1419a910 con 0x7fbf141071e0 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.675+0000 7fbf0b400700 1 -- 172.21.15.62:0/3009535467 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf040249a0 con 0x7fbf141071e0 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.675+0000 7fbf19600700 1 -- 172.21.15.62:0/3009535467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1419d600 con 0x7fbf141071e0 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.675+0000 7fbf0b400700 1 -- 172.21.15.62:0/3009535467 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbf04022020 con 0x7fbf141071e0 2024-04-02T14:32:24.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.675+0000 7fbf0b400700 1 -- 172.21.15.62:0/3009535467 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbf0402dde0 con 0x7fbf141071e0 2024-04-02T14:32:24.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.676+0000 7fbf19600700 1 -- 172.21.15.62:0/3009535467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf14063c20 con 0x7fbf141071e0 2024-04-02T14:32:24.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.678+0000 7fbf0b400700 1 -- 172.21.15.62:0/3009535467 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbf04042dd0 con 0x7fbf141071e0 2024-04-02T14:32:24.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.758+0000 7fbf19600700 1 -- 172.21.15.62:0/3009535467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbf141936a0 con 0x7fbf141071e0 2024-04-02T14:32:24.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.759+0000 7fbf0b400700 1 -- 172.21.15.62:0/3009535467 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fbf04031030 con 0x7fbf141071e0 2024-04-02T14:32:24.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.761+0000 7fbf19600700 1 -- 172.21.15.62:0/3009535467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 msgr2=0x7fbf1419a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:24.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.761+0000 7fbf19600700 1 --2- 172.21.15.62:0/3009535467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf1419a140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf04000c00 tx=0x7fbf04011c80 comp rx=0 tx=0).stop 2024-04-02T14:32:24.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.761+0000 7fbf19600700 1 -- 172.21.15.62:0/3009535467 shutdown_connections 2024-04-02T14:32:24.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.761+0000 7fbf19600700 1 --2- 172.21.15.62:0/3009535467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf141071e0 0x7fbf1419a140 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:24.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.761+0000 7fbf19600700 1 -- 172.21.15.62:0/3009535467 >> 172.21.15.62:0/3009535467 conn(0x7fbf14102780 msgr2=0x7fbf14190760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:24.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.761+0000 7fbf19600700 1 -- 172.21.15.62:0/3009535467 shutdown_connections 2024-04-02T14:32:24.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:23.761+0000 7fbf19600700 1 -- 172.21.15.62:0/3009535467 wait complete. 2024-04-02T14:32:24.514 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (2/15)... 2024-04-02T14:32:25.094 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:24 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:24.762+0000 7f0c37fb0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T14:32:25.094 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:24 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:24.883+0000 7f0c37fb0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T14:32:25.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:25 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:25.145+0000 7f0c37fb0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:32:25.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:25 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:25.394+0000 7f0c37fb0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T14:32:26.094 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:25 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:25.779+0000 7f0c37fb0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:32:26.419 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:26 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:26.102+0000 7f0c37fb0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T14:32:26.772 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:26 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:26.415+0000 7f0c37fb0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T14:32:27.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:27 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3516760108' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T14:32:27.844 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:27 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:27.504+0000 7f0c37fb0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T14:32:28.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:32:28.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-04-02T14:32:28.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:32:28.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T14:32:28.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T14:32:28.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T14:32:28.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T14:32:28.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T14:32:28.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T14:32:28.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T14:32:28.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T14:32:28.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T14:32:28.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T14:32:28.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T14:32:28.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:28.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T14:32:28.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:28.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T14:32:28.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T14:32:28.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:28.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T14:32:28.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T14:32:28.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T14:32:28.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T14:32:28.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T14:32:28.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T14:32:28.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T14:32:28.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:28.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T14:32:28.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:28.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T14:32:28.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:28.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T14:31:58.064994+0000", 2024-04-02T14:32:28.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T14:32:28.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:28.067 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T14:32:28.067 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.059+0000 7f5bc3e00700 1 Processor -- start 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.059+0000 7f5bc3e00700 1 -- start start 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.060+0000 7f5bc3e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc07b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.060+0000 7f5bc3e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc07b770 con 0x7f5bbc07cd40 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.060+0000 7f5bc1400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc07b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.060+0000 7f5bc1400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc07b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59958/0 (socket says 172.21.15.62:59958) 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.060+0000 7f5bc1400700 1 -- 172.21.15.62:0/4119332324 learned_addr learned my addr 172.21.15.62:0/4119332324 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.060+0000 7f5bc1400700 1 -- 172.21.15.62:0/4119332324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc07b8f0 con 0x7f5bbc07cd40 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.061+0000 7f5bc1400700 1 --2- 172.21.15.62:0/4119332324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc07b1a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00098a0 tx=0x7f5bb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee31726f27dda78c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.061+0000 7f5bbbe00700 1 -- 172.21.15.62:0/4119332324 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb0004160 con 0x7f5bbc07cd40 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.061+0000 7f5bbbe00700 1 -- 172.21.15.62:0/4119332324 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f5bb0004300 con 0x7f5bbc07cd40 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.061+0000 7f5bbbe00700 1 -- 172.21.15.62:0/4119332324 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb0011c00 con 0x7f5bbc07cd40 2024-04-02T14:32:28.068 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.062+0000 7f5bc3e00700 1 -- 172.21.15.62:0/4119332324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 msgr2=0x7f5bbc07b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.062+0000 7f5bc3e00700 1 --2- 172.21.15.62:0/4119332324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc07b1a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00098a0 tx=0x7f5bb0009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.062+0000 7f5bc3e00700 1 -- 172.21.15.62:0/4119332324 shutdown_connections 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.062+0000 7f5bc3e00700 1 --2- 172.21.15.62:0/4119332324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc07b1a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.062+0000 7f5bc3e00700 1 -- 172.21.15.62:0/4119332324 >> 172.21.15.62:0/4119332324 conn(0x7f5bbc103890 msgr2=0x7f5bbc105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.062+0000 7f5bc3e00700 1 -- 172.21.15.62:0/4119332324 shutdown_connections 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.062+0000 7f5bc3e00700 1 -- 172.21.15.62:0/4119332324 wait complete. 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.063+0000 7f5bc3e00700 1 Processor -- start 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.063+0000 7f5bc3e00700 1 -- start start 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.064+0000 7f5bc3e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.064+0000 7f5bc3e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc10e900 con 0x7f5bbc07cd40 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.064+0000 7f5bc1400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.064+0000 7f5bc1400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc19e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59964/0 (socket says 172.21.15.62:59964) 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.064+0000 7f5bc1400700 1 -- 172.21.15.62:0/3516760108 learned_addr learned my addr 172.21.15.62:0/3516760108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.064+0000 7f5bc1400700 1 -- 172.21.15.62:0/3516760108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc19ea10 con 0x7f5bbc07cd40 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.065+0000 7f5bc1400700 1 --2- 172.21.15.62:0/3516760108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc19e4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0011bd0 tx=0x7f5bb0011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.065+0000 7f5bba000700 1 -- 172.21.15.62:0/3516760108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb0003b20 con 0x7f5bbc07cd40 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.065+0000 7f5bba000700 1 -- 172.21.15.62:0/3516760108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f5bb0003cc0 con 0x7f5bbc07cd40 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.065+0000 7f5bc3e00700 1 -- 172.21.15.62:0/3516760108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc19eca0 con 0x7f5bbc07cd40 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.065+0000 7f5bba000700 1 -- 172.21.15.62:0/3516760108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb0024930 con 0x7f5bbc07cd40 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.065+0000 7f5bc3e00700 1 -- 172.21.15.62:0/3516760108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc1a1990 con 0x7f5bbc07cd40 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.065+0000 7f5bba000700 1 -- 172.21.15.62:0/3516760108 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5bb0022020 con 0x7f5bbc07cd40 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.065+0000 7f5bba000700 1 -- 172.21.15.62:0/3516760108 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5bb00193c0 con 0x7f5bbc07cd40 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.066+0000 7f5bc3e00700 1 -- 172.21.15.62:0/3516760108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bbc063c20 con 0x7f5bbc07cd40 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.069+0000 7f5bba000700 1 -- 172.21.15.62:0/3516760108 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5bb002ddf0 con 0x7f5bbc07cd40 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.158+0000 7f5bc3e00700 1 -- 172.21.15.62:0/3516760108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5bbc1a1ed0 con 0x7f5bbc07cd40 2024-04-02T14:32:28.070 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.159+0000 7f5bba000700 1 -- 172.21.15.62:0/3516760108 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f5bb001aca0 con 0x7f5bbc07cd40 2024-04-02T14:32:28.071 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.160+0000 7f5bc3e00700 1 -- 172.21.15.62:0/3516760108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 msgr2=0x7f5bbc19e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:28.071 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.160+0000 7f5bc3e00700 1 --2- 172.21.15.62:0/3516760108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc19e4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0011bd0 tx=0x7f5bb0011cb0 comp rx=0 tx=0).stop 2024-04-02T14:32:28.071 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.161+0000 7f5bc3e00700 1 -- 172.21.15.62:0/3516760108 shutdown_connections 2024-04-02T14:32:28.071 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.161+0000 7f5bc3e00700 1 --2- 172.21.15.62:0/3516760108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc07cd40 0x7f5bbc19e4d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:28.071 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.161+0000 7f5bc3e00700 1 -- 172.21.15.62:0/3516760108 >> 172.21.15.62:0/3516760108 conn(0x7f5bbc103890 msgr2=0x7f5bbc104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:28.071 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.161+0000 7f5bc3e00700 1 -- 172.21.15.62:0/3516760108 shutdown_connections 2024-04-02T14:32:28.071 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:27.161+0000 7f5bc3e00700 1 -- 172.21.15.62:0/3516760108 wait complete. 2024-04-02T14:32:28.071 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (3/15)... 2024-04-02T14:32:28.344 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:28.225+0000 7f0c37fb0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T14:32:28.645 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:28.356+0000 7f0c37fb0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T14:32:29.094 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:28.642+0000 7f0c37fb0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T14:32:29.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: Activating manager daemon x 2024-04-02T14:32:29.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: mgrmap e2: x(active, starting, since 0.00626639s) 2024-04-02T14:32:29.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: from='mgr.14100 172.21.15.62:0/3831837391' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T14:32:29.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: from='mgr.14100 172.21.15.62:0/3831837391' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T14:32:29.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: from='mgr.14100 172.21.15.62:0/3831837391' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T14:32:29.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: from='mgr.14100 172.21.15.62:0/3831837391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:32:29.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: from='mgr.14100 172.21.15.62:0/3831837391' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T14:32:29.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: Manager daemon x is now available 2024-04-02T14:32:29.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: from='mgr.14100 172.21.15.62:0/3831837391' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T14:32:29.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: from='mgr.14100 172.21.15.62:0/3831837391' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T14:32:29.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: from='mgr.14100 172.21.15.62:0/3831837391' entity='mgr.x' 2024-04-02T14:32:29.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:28 smithi062 ceph-mon[32116]: from='mgr.14100 172.21.15.62:0/3831837391' entity='mgr.x' 2024-04-02T14:32:30.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:29 smithi062 ceph-mon[32116]: from='mgr.14100 172.21.15.62:0/3831837391' entity='mgr.x' 2024-04-02T14:32:30.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:29 smithi062 ceph-mon[32116]: mgrmap e3: x(active, since 1.02095s) 2024-04-02T14:32:31.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:32:31.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-04-02T14:32:31.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:32:31.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:31.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T14:32:31.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T14:32:31.766 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T14:32:31.766 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T14:32:31.766 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T14:32:31.766 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T14:32:31.766 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T14:32:31.766 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T14:32:31.766 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T14:32:31.766 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:31.766 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T14:32:31.767 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-04-02T14:32:31.768 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T14:32:31.768 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:31.768 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T14:32:31.768 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T14:32:31.768 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T14:31:58.064994+0000", 2024-04-02T14:32:31.768 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T14:32:31.768 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-04-02T14:32:31.768 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T14:32:31.768 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-04-02T14:32:31.768 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.689+0000 7f9325800700 1 Processor -- start 2024-04-02T14:32:31.769 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.689+0000 7f9325800700 1 -- start start 2024-04-02T14:32:31.769 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.689+0000 7f9325800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:31.769 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.689+0000 7f9325800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f93201108c0 con 0x7f932010fed0 2024-04-02T14:32:31.769 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.690+0000 7f931ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:31.769 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.690+0000 7f931ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201102f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60026/0 (socket says 172.21.15.62:60026) 2024-04-02T14:32:31.769 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.690+0000 7f931ea00700 1 -- 172.21.15.62:0/381485301 learned_addr learned my addr 172.21.15.62:0/381485301 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:31.769 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.691+0000 7f931ea00700 1 -- 172.21.15.62:0/381485301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932010ebc0 con 0x7f932010fed0 2024-04-02T14:32:31.769 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.691+0000 7f931ea00700 1 --2- 172.21.15.62:0/381485301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201102f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f93100098a0 tx=0x7f9310009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d00703ffb938ead server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:31.769 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.692+0000 7f931e000700 1 -- 172.21.15.62:0/381485301 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9310004160 con 0x7f932010fed0 2024-04-02T14:32:31.770 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.692+0000 7f931e000700 1 -- 172.21.15.62:0/381485301 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f9310004300 con 0x7f932010fed0 2024-04-02T14:32:31.770 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.692+0000 7f931e000700 1 -- 172.21.15.62:0/381485301 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9310011c00 con 0x7f932010fed0 2024-04-02T14:32:31.770 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.692+0000 7f9325800700 1 -- 172.21.15.62:0/381485301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 msgr2=0x7f93201102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:31.770 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.692+0000 7f9325800700 1 --2- 172.21.15.62:0/381485301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201102f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f93100098a0 tx=0x7f9310009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:31.770 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.693+0000 7f9325800700 1 -- 172.21.15.62:0/381485301 shutdown_connections 2024-04-02T14:32:31.770 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.693+0000 7f9325800700 1 --2- 172.21.15.62:0/381485301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201102f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:31.770 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.693+0000 7f9325800700 1 -- 172.21.15.62:0/381485301 >> 172.21.15.62:0/381485301 conn(0x7f932007dbe0 msgr2=0x7f9320080000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:31.770 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.693+0000 7f9325800700 1 -- 172.21.15.62:0/381485301 shutdown_connections 2024-04-02T14:32:31.770 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.693+0000 7f9325800700 1 -- 172.21.15.62:0/381485301 wait complete. 2024-04-02T14:32:31.770 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.694+0000 7f9325800700 1 Processor -- start 2024-04-02T14:32:31.770 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.694+0000 7f9325800700 1 -- start start 2024-04-02T14:32:31.771 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.694+0000 7f9325800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201a2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:31.771 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.694+0000 7f9325800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9320112e00 con 0x7f932010fed0 2024-04-02T14:32:31.771 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.694+0000 7f931ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:31.771 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.695+0000 7f931ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201a2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60040/0 (socket says 172.21.15.62:60040) 2024-04-02T14:32:31.771 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.695+0000 7f931ea00700 1 -- 172.21.15.62:0/2188648577 learned_addr learned my addr 172.21.15.62:0/2188648577 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:31.771 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.695+0000 7f931ea00700 1 -- 172.21.15.62:0/2188648577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93201a2dc0 con 0x7f932010fed0 2024-04-02T14:32:31.771 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.695+0000 7f931ea00700 1 --2- 172.21.15.62:0/2188648577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201a2880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9310011bd0 tx=0x7f9310011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:31.771 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.696+0000 7f9317400700 1 -- 172.21.15.62:0/2188648577 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9310003b20 con 0x7f932010fed0 2024-04-02T14:32:31.771 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.696+0000 7f9317400700 1 -- 172.21.15.62:0/2188648577 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f9310003cc0 con 0x7f932010fed0 2024-04-02T14:32:31.771 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.696+0000 7f9317400700 1 -- 172.21.15.62:0/2188648577 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9310024930 con 0x7f932010fed0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.696+0000 7f9325800700 1 -- 172.21.15.62:0/2188648577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93201a3050 con 0x7f932010fed0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.696+0000 7f9325800700 1 -- 172.21.15.62:0/2188648577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93201a5d40 con 0x7f932010fed0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.697+0000 7f9317400700 1 -- 172.21.15.62:0/2188648577 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7f9310022020 con 0x7f932010fed0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.697+0000 7f9325800700 1 -- 172.21.15.62:0/2188648577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9320112f80 con 0x7f932010fed0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.697+0000 7f9317400700 1 --2- 172.21.15.62:0/2188648577 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f9304039490 0x7f930403b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.697+0000 7f9317400700 1 -- 172.21.15.62:0/2188648577 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f931004d1f0 con 0x7f932010fed0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.698+0000 7f9317e00700 1 --2- 172.21.15.62:0/2188648577 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f9304039490 0x7f930403b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.698+0000 7f9317e00700 1 --2- 172.21.15.62:0/2188648577 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f9304039490 0x7f930403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9308006fd0 tx=0x7f9308006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.704+0000 7f9317400700 1 -- 172.21.15.62:0/2188648577 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f931001ad50 con 0x7f932010fed0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:30.706+0000 7f9317400700 1 -- 172.21.15.62:0/2188648577 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f931002d3f0 con 0x7f932010fed0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.020+0000 7f9325800700 1 -- 172.21.15.62:0/2188648577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f93201a62f0 con 0x7f932010fed0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.021+0000 7f9317400700 1 -- 172.21.15.62:0/2188648577 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f93100042a0 con 0x7f932010fed0 2024-04-02T14:32:31.772 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.023+0000 7f9325800700 1 -- 172.21.15.62:0/2188648577 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f9304039490 msgr2=0x7f930403b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:31.773 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.023+0000 7f9325800700 1 --2- 172.21.15.62:0/2188648577 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f9304039490 0x7f930403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9308006fd0 tx=0x7f9308006e40 comp rx=0 tx=0).stop 2024-04-02T14:32:31.773 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.023+0000 7f9325800700 1 -- 172.21.15.62:0/2188648577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 msgr2=0x7f93201a2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:31.773 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.023+0000 7f9325800700 1 --2- 172.21.15.62:0/2188648577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201a2880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9310011bd0 tx=0x7f9310011cb0 comp rx=0 tx=0).stop 2024-04-02T14:32:31.773 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.024+0000 7f9325800700 1 -- 172.21.15.62:0/2188648577 shutdown_connections 2024-04-02T14:32:31.773 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.024+0000 7f9325800700 1 --2- 172.21.15.62:0/2188648577 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f9304039490 0x7f930403b950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:31.773 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.024+0000 7f9325800700 1 --2- 172.21.15.62:0/2188648577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932010fed0 0x7f93201a2880 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:31.773 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.024+0000 7f9325800700 1 -- 172.21.15.62:0/2188648577 >> 172.21.15.62:0/2188648577 conn(0x7f932007dbe0 msgr2=0x7f932007e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:31.773 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.024+0000 7f9325800700 1 -- 172.21.15.62:0/2188648577 shutdown_connections 2024-04-02T14:32:31.773 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:31.024+0000 7f9325800700 1 -- 172.21.15.62:0/2188648577 wait complete. 2024-04-02T14:32:31.773 INFO:teuthology.orchestra.run.smithi062.stdout:mgr is available 2024-04-02T14:32:32.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:31 smithi062 ceph-mon[32116]: mgrmap e4: x(active, since 2s) 2024-04-02T14:32:32.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:31 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2188648577' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T14:32:32.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:32 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/88899624' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-04-02T14:32:33.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.362+0000 7fefef200700 1 Processor -- start 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.362+0000 7fefef200700 1 -- start start 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.362+0000 7fefef200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff010a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.363+0000 7fefef200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feff010ad00 con 0x7feff010a310 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.363+0000 7fefede00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff010a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.363+0000 7fefede00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff010a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51628/0 (socket says 172.21.15.62:51628) 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.363+0000 7fefede00700 1 -- 172.21.15.62:0/23061355 learned_addr learned my addr 172.21.15.62:0/23061355 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:33.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.363+0000 7fefede00700 1 -- 172.21.15.62:0/23061355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff010b4c0 con 0x7feff010a310 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.364+0000 7fefede00700 1 --2- 172.21.15.62:0/23061355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff010a730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fefe00098a0 tx=0x7fefe0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8805cd1a4b9de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.364+0000 7fefeca00700 1 -- 172.21.15.62:0/23061355 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe0004160 con 0x7feff010a310 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.364+0000 7fefeca00700 1 -- 172.21.15.62:0/23061355 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fefe0004300 con 0x7feff010a310 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.364+0000 7fefeca00700 1 -- 172.21.15.62:0/23061355 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe0011c00 con 0x7feff010a310 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.365+0000 7fefef200700 1 -- 172.21.15.62:0/23061355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 msgr2=0x7feff010a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.365+0000 7fefef200700 1 --2- 172.21.15.62:0/23061355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff010a730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fefe00098a0 tx=0x7fefe0009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.365+0000 7fefef200700 1 -- 172.21.15.62:0/23061355 shutdown_connections 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.365+0000 7fefef200700 1 --2- 172.21.15.62:0/23061355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff010a730 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.365+0000 7fefef200700 1 -- 172.21.15.62:0/23061355 >> 172.21.15.62:0/23061355 conn(0x7feff007d700 msgr2=0x7feff007db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.365+0000 7fefef200700 1 -- 172.21.15.62:0/23061355 shutdown_connections 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.365+0000 7fefef200700 1 -- 172.21.15.62:0/23061355 wait complete. 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.366+0000 7fefef200700 1 Processor -- start 2024-04-02T14:32:33.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.366+0000 7fefef200700 1 -- start start 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.367+0000 7fefef200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.367+0000 7fefef200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feff010e870 con 0x7feff010a310 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.367+0000 7fefede00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.367+0000 7fefede00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff019e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51638/0 (socket says 172.21.15.62:51638) 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.367+0000 7fefede00700 1 -- 172.21.15.62:0/88899624 learned_addr learned my addr 172.21.15.62:0/88899624 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.367+0000 7fefede00700 1 -- 172.21.15.62:0/88899624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff019e860 con 0x7feff010a310 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.368+0000 7fefede00700 1 --2- 172.21.15.62:0/88899624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff019e320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fefe0000c00 tx=0x7fefe0011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.368+0000 7fefe6a00700 1 -- 172.21.15.62:0/88899624 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe0003b20 con 0x7feff010a310 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.368+0000 7fefe6a00700 1 -- 172.21.15.62:0/88899624 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fefe0003cc0 con 0x7feff010a310 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.368+0000 7fefef200700 1 -- 172.21.15.62:0/88899624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff019eaf0 con 0x7feff010a310 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.368+0000 7fefe6a00700 1 -- 172.21.15.62:0/88899624 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe00249a0 con 0x7feff010a310 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.368+0000 7fefef200700 1 -- 172.21.15.62:0/88899624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff01a17e0 con 0x7feff010a310 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.369+0000 7fefe6a00700 1 -- 172.21.15.62:0/88899624 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fefe0022020 con 0x7feff010a310 2024-04-02T14:32:33.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.369+0000 7fefef200700 1 -- 172.21.15.62:0/88899624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff010e9f0 con 0x7feff010a310 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.370+0000 7fefe6a00700 1 --2- 172.21.15.62:0/88899624 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7fefd40395b0 0x7fefd403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.370+0000 7fefe6a00700 1 -- 172.21.15.62:0/88899624 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fefe004d2f0 con 0x7feff010a310 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.370+0000 7fefed400700 1 --2- 172.21.15.62:0/88899624 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7fefd40395b0 0x7fefd403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.371+0000 7fefed400700 1 --2- 172.21.15.62:0/88899624 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7fefd40395b0 0x7fefd403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fefdc006fd0 tx=0x7fefdc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.375+0000 7fefe6a00700 1 -- 172.21.15.62:0/88899624 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fefe004d050 con 0x7feff010a310 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.604+0000 7fefef200700 1 -- 172.21.15.62:0/88899624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7feff0063c20 con 0x7feff010a310 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.607+0000 7fefe6a00700 1 -- 172.21.15.62:0/88899624 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+400 (secure 0 0 0) 0x7fefe002d3f0 con 0x7feff010a310 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.610+0000 7fefef200700 1 -- 172.21.15.62:0/88899624 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7fefd40395b0 msgr2=0x7fefd403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.610+0000 7fefef200700 1 --2- 172.21.15.62:0/88899624 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7fefd40395b0 0x7fefd403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fefdc006fd0 tx=0x7fefdc006e40 comp rx=0 tx=0).stop 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.610+0000 7fefef200700 1 -- 172.21.15.62:0/88899624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 msgr2=0x7feff019e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.610+0000 7fefef200700 1 --2- 172.21.15.62:0/88899624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff019e320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fefe0000c00 tx=0x7fefe0011c80 comp rx=0 tx=0).stop 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.610+0000 7fefef200700 1 -- 172.21.15.62:0/88899624 shutdown_connections 2024-04-02T14:32:33.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.610+0000 7fefef200700 1 --2- 172.21.15.62:0/88899624 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7fefd40395b0 0x7fefd403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:33.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.610+0000 7fefef200700 1 --2- 172.21.15.62:0/88899624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff010a310 0x7feff019e320 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:33.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.610+0000 7fefef200700 1 -- 172.21.15.62:0/88899624 >> 172.21.15.62:0/88899624 conn(0x7feff007d700 msgr2=0x7feff0107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:33.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.610+0000 7fefef200700 1 -- 172.21.15.62:0/88899624 shutdown_connections 2024-04-02T14:32:33.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:32.611+0000 7fefef200700 1 -- 172.21.15.62:0/88899624 wait complete. 2024-04-02T14:32:33.263 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling cephadm module... 2024-04-02T14:32:34.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:34 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1854008499' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-02T14:32:34.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:34 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: ignoring --setuser ceph since I am not root 2024-04-02T14:32:34.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:34 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: ignoring --setgroup ceph since I am not root 2024-04-02T14:32:35.024 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.876+0000 7f5473e00700 1 Processor -- start 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.877+0000 7f5473e00700 1 -- start start 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.877+0000 7f5473e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547410a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.877+0000 7f5473e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f547410ad60 con 0x7f547410a370 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.877+0000 7f5472a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547410a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.878+0000 7f5472a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547410a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51650/0 (socket says 172.21.15.62:51650) 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.878+0000 7f5472a00700 1 -- 172.21.15.62:0/637464002 learned_addr learned my addr 172.21.15.62:0/637464002 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.878+0000 7f5472a00700 1 -- 172.21.15.62:0/637464002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547410b520 con 0x7f547410a370 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.878+0000 7f5472a00700 1 --2- 172.21.15.62:0/637464002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547410a790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5464009e80 tx=0x7f546400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9eebdcfea271a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.879+0000 7f5472000700 1 -- 172.21.15.62:0/637464002 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5464017070 con 0x7f547410a370 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.879+0000 7f5472000700 1 -- 172.21.15.62:0/637464002 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f546400b8c0 con 0x7f547410a370 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.879+0000 7f5472000700 1 -- 172.21.15.62:0/637464002 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5464004620 con 0x7f547410a370 2024-04-02T14:32:35.025 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.879+0000 7f5473e00700 1 -- 172.21.15.62:0/637464002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 msgr2=0x7f547410a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.879+0000 7f5473e00700 1 --2- 172.21.15.62:0/637464002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547410a790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5464009e80 tx=0x7f546400b330 comp rx=0 tx=0).stop 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.880+0000 7f5473e00700 1 -- 172.21.15.62:0/637464002 shutdown_connections 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.880+0000 7f5473e00700 1 --2- 172.21.15.62:0/637464002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547410a790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.880+0000 7f5473e00700 1 -- 172.21.15.62:0/637464002 >> 172.21.15.62:0/637464002 conn(0x7f547407d840 msgr2=0x7f54741084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.880+0000 7f5473e00700 1 -- 172.21.15.62:0/637464002 shutdown_connections 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.880+0000 7f5473e00700 1 -- 172.21.15.62:0/637464002 wait complete. 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.881+0000 7f5473e00700 1 Processor -- start 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.881+0000 7f5473e00700 1 -- start start 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.881+0000 7f5473e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547419e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.881+0000 7f5473e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f547410e8d0 con 0x7f547410a370 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.881+0000 7f5472a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547419e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.882+0000 7f5472a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547419e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51658/0 (socket says 172.21.15.62:51658) 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.882+0000 7f5472a00700 1 -- 172.21.15.62:0/1854008499 learned_addr learned my addr 172.21.15.62:0/1854008499 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:35.026 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.882+0000 7f5472a00700 1 -- 172.21.15.62:0/1854008499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547419e8a0 con 0x7f547410a370 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.882+0000 7f5472a00700 1 --2- 172.21.15.62:0/1854008499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547419e360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5464009fd0 tx=0x7f5464003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.882+0000 7f5478a00700 1 -- 172.21.15.62:0/1854008499 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5464021020 con 0x7f547410a370 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.883+0000 7f5478a00700 1 -- 172.21.15.62:0/1854008499 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f546400bde0 con 0x7f547410a370 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.883+0000 7f5478a00700 1 -- 172.21.15.62:0/1854008499 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5464017040 con 0x7f547410a370 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.883+0000 7f5473e00700 1 -- 172.21.15.62:0/1854008499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547419eb30 con 0x7f547410a370 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.883+0000 7f5473e00700 1 -- 172.21.15.62:0/1854008499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54741a1820 con 0x7f547410a370 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.884+0000 7f5478a00700 1 -- 172.21.15.62:0/1854008499 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f5464024020 con 0x7f547410a370 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.884+0000 7f5473e00700 1 -- 172.21.15.62:0/1854008499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547410ea50 con 0x7f547410a370 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.884+0000 7f5478a00700 1 --2- 172.21.15.62:0/1854008499 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f54580395b0 0x7f545803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.884+0000 7f5478a00700 1 -- 172.21.15.62:0/1854008499 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f546400f050 con 0x7f547410a370 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.885+0000 7f546a000700 1 --2- 172.21.15.62:0/1854008499 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f54580395b0 0x7f545803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.885+0000 7f546a000700 1 --2- 172.21.15.62:0/1854008499 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f54580395b0 0x7f545803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f545c006fd0 tx=0x7f545c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:35.027 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:33.890+0000 7f5478a00700 1 -- 172.21.15.62:0/1854008499 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f546402b020 con 0x7f547410a370 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.186+0000 7f5473e00700 1 -- 172.21.15.62:0/1854008499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f54741a1e90 con 0x7f547410a370 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.240+0000 7f5478a00700 1 -- 172.21.15.62:0/1854008499 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f546402e350 con 0x7f547410a370 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.243+0000 7f5473e00700 1 -- 172.21.15.62:0/1854008499 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f54580395b0 msgr2=0x7f545803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.243+0000 7f5473e00700 1 --2- 172.21.15.62:0/1854008499 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f54580395b0 0x7f545803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f545c006fd0 tx=0x7f545c006e40 comp rx=0 tx=0).stop 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.243+0000 7f5473e00700 1 -- 172.21.15.62:0/1854008499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 msgr2=0x7f547419e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.243+0000 7f5478a00700 1 -- 172.21.15.62:0/1854008499 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f546401d990 con 0x7f547410a370 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.244+0000 7f5473e00700 1 --2- 172.21.15.62:0/1854008499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547419e360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5464009fd0 tx=0x7f5464003d00 comp rx=0 tx=0).stop 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.244+0000 7f5473e00700 1 -- 172.21.15.62:0/1854008499 shutdown_connections 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.244+0000 7f5473e00700 1 --2- 172.21.15.62:0/1854008499 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f54580395b0 0x7f545803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.245+0000 7f5473e00700 1 --2- 172.21.15.62:0/1854008499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f547410a370 0x7f547419e360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.245+0000 7f5473e00700 1 -- 172.21.15.62:0/1854008499 >> 172.21.15.62:0/1854008499 conn(0x7f547407d840 msgr2=0x7f5474107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.245+0000 7f5473e00700 1 -- 172.21.15.62:0/1854008499 shutdown_connections 2024-04-02T14:32:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:34.245+0000 7f5473e00700 1 -- 172.21.15.62:0/1854008499 wait complete. 2024-04-02T14:32:35.321 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:35 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:35.063+0000 7fa61be8b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T14:32:35.322 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:35 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:35.190+0000 7fa61be8b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T14:32:35.322 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:35 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1854008499' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-02T14:32:35.322 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:35 smithi062 ceph-mon[32116]: mgrmap e5: x(active, since 5s) 2024-04-02T14:32:36.252 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:36 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:36.027+0000 7fa61be8b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T14:32:36.252 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:36 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:36.153+0000 7fa61be8b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T14:32:36.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:36 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/974757085' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T14:32:36.588 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:36 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:36.339+0000 7fa61be8b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T14:32:36.589 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:36 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:36.467+0000 7fa61be8b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T14:32:36.589 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:36 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:36.582+0000 7fa61be8b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T14:32:36.729 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-04-02T14:32:36.809 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-02T14:32:36.809 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T14:32:36.809 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-02T14:32:36.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T14:32:36.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-04-02T14:32:36.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.621+0000 7f55e8600700 1 Processor -- start 2024-04-02T14:32:36.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.622+0000 7f55e8600700 1 -- start start 2024-04-02T14:32:36.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.622+0000 7f55e8600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e01086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:36.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.622+0000 7f55e8600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55e0108ca0 con 0x7f55e01082b0 2024-04-02T14:32:36.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.622+0000 7f55e5c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e01086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:36.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.622+0000 7f55e5c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e01086d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51680/0 (socket says 172.21.15.62:51680) 2024-04-02T14:32:36.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.622+0000 7f55e5c00700 1 -- 172.21.15.62:0/1592089032 learned_addr learned my addr 172.21.15.62:0/1592089032 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:36.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.623+0000 7f55e5c00700 1 -- 172.21.15.62:0/1592089032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e01094b0 con 0x7f55e01082b0 2024-04-02T14:32:36.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.623+0000 7f55e5c00700 1 --2- 172.21.15.62:0/1592089032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e01086d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f55d40098a0 tx=0x7f55d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a542c150ee4cfe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:36.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.623+0000 7f55dfe00700 1 -- 172.21.15.62:0/1592089032 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d4004160 con 0x7f55e01082b0 2024-04-02T14:32:36.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.624+0000 7f55dfe00700 1 -- 172.21.15.62:0/1592089032 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f55d4004300 con 0x7f55e01082b0 2024-04-02T14:32:36.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.624+0000 7f55dfe00700 1 -- 172.21.15.62:0/1592089032 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d4011c00 con 0x7f55e01082b0 2024-04-02T14:32:36.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.624+0000 7f55e8600700 1 -- 172.21.15.62:0/1592089032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 msgr2=0x7f55e01086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:36.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.624+0000 7f55e8600700 1 --2- 172.21.15.62:0/1592089032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e01086d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f55d40098a0 tx=0x7f55d4009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:36.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.624+0000 7f55e8600700 1 -- 172.21.15.62:0/1592089032 shutdown_connections 2024-04-02T14:32:36.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.624+0000 7f55e8600700 1 --2- 172.21.15.62:0/1592089032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e01086d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:36.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.624+0000 7f55e8600700 1 -- 172.21.15.62:0/1592089032 >> 172.21.15.62:0/1592089032 conn(0x7f55e0103890 msgr2=0x7f55e0105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:36.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.625+0000 7f55e8600700 1 -- 172.21.15.62:0/1592089032 shutdown_connections 2024-04-02T14:32:36.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.625+0000 7f55e8600700 1 -- 172.21.15.62:0/1592089032 wait complete. 2024-04-02T14:32:36.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.626+0000 7f55e8600700 1 Processor -- start 2024-04-02T14:32:36.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.626+0000 7f55e8600700 1 -- start start 2024-04-02T14:32:36.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.626+0000 7f55e8600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:36.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.626+0000 7f55e8600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55e010c860 con 0x7f55e01082b0 2024-04-02T14:32:36.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.627+0000 7f55e5c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:36.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.627+0000 7f55e5c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e019c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51692/0 (socket says 172.21.15.62:51692) 2024-04-02T14:32:36.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.627+0000 7f55e5c00700 1 -- 172.21.15.62:0/974757085 learned_addr learned my addr 172.21.15.62:0/974757085 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:36.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.627+0000 7f55e5c00700 1 -- 172.21.15.62:0/974757085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e019c870 con 0x7f55e01082b0 2024-04-02T14:32:36.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.627+0000 7f55e5c00700 1 --2- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e019c330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f55d4000c00 tx=0x7f55d4011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:36.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.628+0000 7f55de000700 1 -- 172.21.15.62:0/974757085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d4003b20 con 0x7f55e01082b0 2024-04-02T14:32:36.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.628+0000 7f55de000700 1 -- 172.21.15.62:0/974757085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f55d4003cc0 con 0x7f55e01082b0 2024-04-02T14:32:36.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.628+0000 7f55e8600700 1 -- 172.21.15.62:0/974757085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e019cb00 con 0x7f55e01082b0 2024-04-02T14:32:36.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.628+0000 7f55de000700 1 -- 172.21.15.62:0/974757085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d40249a0 con 0x7f55e01082b0 2024-04-02T14:32:36.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.628+0000 7f55e8600700 1 -- 172.21.15.62:0/974757085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e019f7f0 con 0x7f55e01082b0 2024-04-02T14:32:36.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.629+0000 7f55e8600700 1 -- 172.21.15.62:0/974757085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e010c9e0 con 0x7f55e01082b0 2024-04-02T14:32:36.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.629+0000 7f55de000700 1 -- 172.21.15.62:0/974757085 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f55d4022020 con 0x7f55e01082b0 2024-04-02T14:32:36.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.630+0000 7f55de000700 1 --2- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f55c8039600 0x7f55c803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:36.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.630+0000 7f55de000700 1 -- 172.21.15.62:0/974757085 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f55d404d360 con 0x7f55e01082b0 2024-04-02T14:32:36.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.630+0000 7f55e5200700 1 -- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f55c8039600 msgr2=0x7f55c803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3623171244 2024-04-02T14:32:36.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.630+0000 7f55e5200700 1 --2- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f55c8039600 0x7f55c803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T14:32:36.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.636+0000 7f55de000700 1 -- 172.21.15.62:0/974757085 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f55d404d0c0 con 0x7f55e01082b0 2024-04-02T14:32:36.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.831+0000 7f55e5200700 1 -- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f55c8039600 msgr2=0x7f55c803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3623171244 2024-04-02T14:32:36.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.831+0000 7f55e5200700 1 --2- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f55c8039600 0x7f55c803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T14:32:36.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.927+0000 7f55e8600700 1 -- 172.21.15.62:0/974757085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f55e019fe40 con 0x7f55e01082b0 2024-04-02T14:32:36.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.928+0000 7f55de000700 1 -- 172.21.15.62:0/974757085 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f55d401a430 con 0x7f55e01082b0 2024-04-02T14:32:36.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.930+0000 7f55e8600700 1 -- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f55c8039600 msgr2=0x7f55c803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T14:32:36.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.930+0000 7f55e8600700 1 --2- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f55c8039600 0x7f55c803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:36.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.931+0000 7f55e8600700 1 -- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 msgr2=0x7f55e019c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:36.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.931+0000 7f55e8600700 1 --2- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e019c330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f55d4000c00 tx=0x7f55d4011c80 comp rx=0 tx=0).stop 2024-04-02T14:32:36.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.931+0000 7f55e8600700 1 -- 172.21.15.62:0/974757085 shutdown_connections 2024-04-02T14:32:36.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.931+0000 7f55e8600700 1 --2- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7f55c8039600 0x7f55c803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:36.817 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.931+0000 7f55e8600700 1 --2- 172.21.15.62:0/974757085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e01082b0 0x7f55e019c330 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:36.817 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.931+0000 7f55e8600700 1 -- 172.21.15.62:0/974757085 >> 172.21.15.62:0/974757085 conn(0x7f55e0103890 msgr2=0x7f55e0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:36.817 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.931+0000 7f55e8600700 1 -- 172.21.15.62:0/974757085 shutdown_connections 2024-04-02T14:32:36.817 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:35.931+0000 7f55e8600700 1 -- 172.21.15.62:0/974757085 wait complete. 2024-04-02T14:32:36.817 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2024-04-02T14:32:36.817 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 5... 2024-04-02T14:32:37.149 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:36 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:36.945+0000 7fa61be8b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T14:32:37.149 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:37 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:37.096+0000 7fa61be8b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T14:32:37.516 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:37 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:37.216+0000 7fa61be8b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T14:32:37.516 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:37 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:37.348+0000 7fa61be8b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T14:32:37.844 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:37 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:37.570+0000 7fa61be8b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T14:32:39.065 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:38 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:38.882+0000 7fa61be8b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T14:32:39.065 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:39 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:39.062+0000 7fa61be8b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T14:32:39.844 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:39 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:39.490+0000 7fa61be8b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T14:32:39.844 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:39 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:39.611+0000 7fa61be8b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T14:32:40.131 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:39 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:39.875+0000 7fa61be8b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:32:40.519 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:40 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:40.127+0000 7fa61be8b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T14:32:40.520 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:40 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:40.516+0000 7fa61be8b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:32:41.095 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:40 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:40.840+0000 7fa61be8b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T14:32:41.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:41 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:41.157+0000 7fa61be8b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T14:32:42.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:42 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:42.226+0000 7fa61be8b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T14:32:43.345 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:42 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:42.945+0000 7fa61be8b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T14:32:43.345 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:43.078+0000 7fa61be8b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T14:32:43.844 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:32:43.370+0000 7fa61be8b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T14:32:43.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-mon[32116]: Active manager daemon x restarted 2024-04-02T14:32:43.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-mon[32116]: Activating manager daemon x 2024-04-02T14:32:43.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-mon[32116]: osdmap e2: 0 total, 0 up, 0 in 2024-04-02T14:32:43.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-mon[32116]: mgrmap e6: x(active, starting, since 0.00654583s) 2024-04-02T14:32:43.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:32:43.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T14:32:43.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T14:32:43.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T14:32:43.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T14:32:43.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-mon[32116]: Manager daemon x is now available 2024-04-02T14:32:43.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:43 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:32:44.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:44 smithi062 ceph-mon[32116]: Found migration_current of "None". Setting to last migration. 2024-04-02T14:32:44.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:44 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:32:44.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:44 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:32:44.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:44 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:32:44.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:44 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T14:32:44.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:44 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T14:32:44.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:44 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:32:44.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:44 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:32:44.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:44 smithi062 ceph-mon[32116]: [02/Apr/2024:14:32:44] ENGINE Bus STARTING 2024-04-02T14:32:44.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:44 smithi062 ceph-mon[32116]: mgrmap e7: x(active, since 1.0156s) 2024-04-02T14:32:45.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-04-02T14:32:45.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-02T14:32:45.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T14:32:45.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-04-02T14:32:45.090 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.304+0000 7febf9800700 1 Processor -- start 2024-04-02T14:32:45.090 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.305+0000 7febf9800700 1 -- start start 2024-04-02T14:32:45.090 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.305+0000 7febf9800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf410b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:45.090 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.305+0000 7febf9800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7febf4063850 con 0x7febf4108e10 2024-04-02T14:32:45.090 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.306+0000 7febf2a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf410b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.306+0000 7febf2a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf410b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51708/0 (socket says 172.21.15.62:51708) 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.306+0000 7febf2a00700 1 -- 172.21.15.62:0/3167575790 learned_addr learned my addr 172.21.15.62:0/3167575790 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.306+0000 7febf2a00700 1 -- 172.21.15.62:0/3167575790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf410b740 con 0x7febf4108e10 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.307+0000 7febf2a00700 1 --2- 172.21.15.62:0/3167575790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf410b200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7febe40098a0 tx=0x7febe4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37d608c8aa99a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.307+0000 7febf1600700 1 -- 172.21.15.62:0/3167575790 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe4004160 con 0x7febf4108e10 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.307+0000 7febf1600700 1 -- 172.21.15.62:0/3167575790 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7febe4004300 con 0x7febf4108e10 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.307+0000 7febf1600700 1 -- 172.21.15.62:0/3167575790 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe4011c00 con 0x7febf4108e10 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.308+0000 7febf9800700 1 -- 172.21.15.62:0/3167575790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 msgr2=0x7febf410b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.308+0000 7febf9800700 1 --2- 172.21.15.62:0/3167575790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf410b200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7febe40098a0 tx=0x7febe4009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.308+0000 7febf9800700 1 -- 172.21.15.62:0/3167575790 shutdown_connections 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.308+0000 7febf9800700 1 --2- 172.21.15.62:0/3167575790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf410b200 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:45.091 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.308+0000 7febf9800700 1 -- 172.21.15.62:0/3167575790 >> 172.21.15.62:0/3167575790 conn(0x7febf4102780 msgr2=0x7febf4104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.308+0000 7febf9800700 1 -- 172.21.15.62:0/3167575790 shutdown_connections 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.308+0000 7febf9800700 1 -- 172.21.15.62:0/3167575790 wait complete. 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.309+0000 7febf9800700 1 Processor -- start 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.309+0000 7febf9800700 1 -- start start 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.309+0000 7febf9800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.309+0000 7febf9800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7febf410c710 con 0x7febf4108e10 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.310+0000 7febf2a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.310+0000 7febf2a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf419e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51716/0 (socket says 172.21.15.62:51716) 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.310+0000 7febf2a00700 1 -- 172.21.15.62:0/2294539829 learned_addr learned my addr 172.21.15.62:0/2294539829 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.310+0000 7febf2a00700 1 -- 172.21.15.62:0/2294539829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf419ea30 con 0x7febf4108e10 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.310+0000 7febf2a00700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf419e4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7febe4011bd0 tx=0x7febe4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:45.092 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.311+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe4003b20 con 0x7febf4108e10 2024-04-02T14:32:45.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.311+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7febe4003cc0 con 0x7febf4108e10 2024-04-02T14:32:45.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.311+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe4024930 con 0x7febf4108e10 2024-04-02T14:32:45.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.311+0000 7febf9800700 1 -- 172.21.15.62:0/2294539829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf419ecc0 con 0x7febf4108e10 2024-04-02T14:32:45.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.311+0000 7febf9800700 1 -- 172.21.15.62:0/2294539829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf41a19b0 con 0x7febf4108e10 2024-04-02T14:32:45.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.312+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7febe4022020 con 0x7febf4108e10 2024-04-02T14:32:45.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.313+0000 7febeb400700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 0x7febd803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:45.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.313+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 --> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7febd803c1d0 con 0x7febd8039600 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.313+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7febe404d3f0 con 0x7febf4108e10 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.313+0000 7febf2000700 1 -- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 msgr2=0x7febd803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3623171244 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.313+0000 7febf2000700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 0x7febd803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.514+0000 7febf2000700 1 -- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 msgr2=0x7febd803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3623171244 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.514+0000 7febf2000700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 0x7febd803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.915+0000 7febf2000700 1 -- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 msgr2=0x7febd803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3623171244 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:37.915+0000 7febf2000700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 0x7febd803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:38.716+0000 7febf2000700 1 -- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 msgr2=0x7febd803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3623171244 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:38.716+0000 7febf2000700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 0x7febd803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:40.318+0000 7febf2000700 1 -- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 msgr2=0x7febd803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3623171244 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:40.318+0000 7febf2000700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 0x7febd803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:43.378+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7febe401a950 con 0x7febf4108e10 2024-04-02T14:32:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:43.378+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 msgr2=0x7febd803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:43.378+0000 7febeb400700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 0x7febd803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.382+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7febe400bb10 con 0x7febf4108e10 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.383+0000 7febeb400700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7febd803cef0 0x7febd803f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.383+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 --> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7febe4017d10 con 0x7febd803cef0 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.384+0000 7febf2000700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7febd803cef0 0x7febd803f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.386+0000 7febf2000700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7febd803cef0 0x7febd803f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7febe0003a10 tx=0x7febe00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.388+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 <== mgr.14120 v2:172.21.15.62:6800/1356430694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7febe4017d10 con 0x7febd803cef0 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.396+0000 7febf9800700 1 -- 172.21.15.62:0/2294539829 --> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7febf4063c20 con 0x7febd803cef0 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.397+0000 7febeb400700 1 -- 172.21.15.62:0/2294539829 <== mgr.14120 v2:172.21.15.62:6800/1356430694 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7febf4063c20 con 0x7febd803cef0 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.397+0000 7febf9800700 1 -- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7febd803cef0 msgr2=0x7febd803f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.397+0000 7febf9800700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7febd803cef0 0x7febd803f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7febe0003a10 tx=0x7febe00092b0 comp rx=0 tx=0).stop 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.397+0000 7febf9800700 1 -- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 msgr2=0x7febf419e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.397+0000 7febf9800700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf419e4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7febe4011bd0 tx=0x7febe4011cb0 comp rx=0 tx=0).stop 2024-04-02T14:32:45.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.398+0000 7febf9800700 1 -- 172.21.15.62:0/2294539829 shutdown_connections 2024-04-02T14:32:45.100 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.398+0000 7febf9800700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7febd803cef0 0x7febd803f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:45.100 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.398+0000 7febf9800700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4108e10 0x7febf419e4f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:45.100 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.398+0000 7febf9800700 1 --2- 172.21.15.62:0/2294539829 >> [v2:172.21.15.62:6800/3623171244,v1:172.21.15.62:6801/3623171244] conn(0x7febd8039600 0x7febd803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:45.100 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.398+0000 7febf9800700 1 -- 172.21.15.62:0/2294539829 >> 172.21.15.62:0/2294539829 conn(0x7febf4102780 msgr2=0x7febf4103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:45.100 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.398+0000 7febf9800700 1 -- 172.21.15.62:0/2294539829 shutdown_connections 2024-04-02T14:32:45.100 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:44.398+0000 7febf9800700 1 -- 172.21.15.62:0/2294539829 wait complete. 2024-04-02T14:32:45.100 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 5 is available 2024-04-02T14:32:45.100 INFO:teuthology.orchestra.run.smithi062.stdout:Setting orchestrator backend to cephadm... 2024-04-02T14:32:45.687 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:45 smithi062 ceph-mon[32116]: [02/Apr/2024:14:32:44] ENGINE Serving on http://172.21.15.62:8765 2024-04-02T14:32:45.687 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:45 smithi062 ceph-mon[32116]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T14:32:45.687 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:45 smithi062 ceph-mon[32116]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T14:32:45.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:45 smithi062 ceph-mon[32116]: [02/Apr/2024:14:32:44] ENGINE Serving on https://172.21.15.62:7150 2024-04-02T14:32:45.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:45 smithi062 ceph-mon[32116]: [02/Apr/2024:14:32:44] ENGINE Bus STARTED 2024-04-02T14:32:45.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:45 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:32:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:46 smithi062 ceph-mon[32116]: mgrmap e8: x(active, since 2s) 2024-04-02T14:32:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:46 smithi062 ceph-mon[32116]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:32:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:46 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:32:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:46 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:32:46.792 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.749+0000 7fcaf0c00700 1 Processor -- start 2024-04-02T14:32:46.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.749+0000 7fcaf0c00700 1 -- start start 2024-04-02T14:32:46.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.749+0000 7fcaf0c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec10f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:46.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.749+0000 7fcaf0c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec10f910 con 0x7fcaec10ef20 2024-04-02T14:32:46.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.750+0000 7fcae9e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec10f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:46.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.750+0000 7fcae9e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec10f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38640/0 (socket says 172.21.15.62:38640) 2024-04-02T14:32:46.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.750+0000 7fcae9e00700 1 -- 172.21.15.62:0/1050996950 learned_addr learned my addr 172.21.15.62:0/1050996950 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.750+0000 7fcae9e00700 1 -- 172.21.15.62:0/1050996950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec10fa90 con 0x7fcaec10ef20 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.751+0000 7fcae9e00700 1 --2- 172.21.15.62:0/1050996950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec10f340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0098a0 tx=0x7fcadc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875bf5342f10d36b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.751+0000 7fcae9400700 1 -- 172.21.15.62:0/1050996950 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcadc004160 con 0x7fcaec10ef20 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.751+0000 7fcae9400700 1 -- 172.21.15.62:0/1050996950 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fcadc004300 con 0x7fcaec10ef20 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.751+0000 7fcae9400700 1 -- 172.21.15.62:0/1050996950 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcadc011c00 con 0x7fcaec10ef20 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.752+0000 7fcaf0c00700 1 -- 172.21.15.62:0/1050996950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 msgr2=0x7fcaec10f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.752+0000 7fcaf0c00700 1 --2- 172.21.15.62:0/1050996950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec10f340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0098a0 tx=0x7fcadc009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.752+0000 7fcaf0c00700 1 -- 172.21.15.62:0/1050996950 shutdown_connections 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.752+0000 7fcaf0c00700 1 --2- 172.21.15.62:0/1050996950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec10f340 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.752+0000 7fcaf0c00700 1 -- 172.21.15.62:0/1050996950 >> 172.21.15.62:0/1050996950 conn(0x7fcaec07d9e0 msgr2=0x7fcaec07fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.752+0000 7fcaf0c00700 1 -- 172.21.15.62:0/1050996950 shutdown_connections 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.752+0000 7fcaf0c00700 1 -- 172.21.15.62:0/1050996950 wait complete. 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.753+0000 7fcaf0c00700 1 Processor -- start 2024-04-02T14:32:46.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.754+0000 7fcaf0c00700 1 -- start start 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.754+0000 7fcaf0c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec1a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.754+0000 7fcaf0c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec110c90 con 0x7fcaec10ef20 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.754+0000 7fcae9e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec1a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.754+0000 7fcae9e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec1a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38656/0 (socket says 172.21.15.62:38656) 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.754+0000 7fcae9e00700 1 -- 172.21.15.62:0/2690197048 learned_addr learned my addr 172.21.15.62:0/2690197048 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.755+0000 7fcae9e00700 1 -- 172.21.15.62:0/2690197048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec1a3040 con 0x7fcaec10ef20 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.755+0000 7fcae9e00700 1 --2- 172.21.15.62:0/2690197048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec1a2b00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcadc011bd0 tx=0x7fcadc011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.755+0000 7fcae2a00700 1 -- 172.21.15.62:0/2690197048 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcadc003970 con 0x7fcaec10ef20 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.755+0000 7fcae2a00700 1 -- 172.21.15.62:0/2690197048 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fcadc003b10 con 0x7fcaec10ef20 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.755+0000 7fcae2a00700 1 -- 172.21.15.62:0/2690197048 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcadc024580 con 0x7fcaec10ef20 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.755+0000 7fcaf0c00700 1 -- 172.21.15.62:0/2690197048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec1a3330 con 0x7fcaec10ef20 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.756+0000 7fcaf0c00700 1 -- 172.21.15.62:0/2690197048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec1a5fc0 con 0x7fcaec10ef20 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.757+0000 7fcae2a00700 1 -- 172.21.15.62:0/2690197048 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fcadc01a9a0 con 0x7fcaec10ef20 2024-04-02T14:32:46.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.757+0000 7fcaf0c00700 1 -- 172.21.15.62:0/2690197048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec063c20 con 0x7fcaec10ef20 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.758+0000 7fcae2a00700 1 --2- 172.21.15.62:0/2690197048 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fcad0039660 0x7fcad003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.758+0000 7fcae2a00700 1 -- 172.21.15.62:0/2690197048 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcadc04d900 con 0x7fcaec10ef20 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.758+0000 7fcae3e00700 1 --2- 172.21.15.62:0/2690197048 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fcad0039660 0x7fcad003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.759+0000 7fcae3e00700 1 --2- 172.21.15.62:0/2690197048 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fcad0039660 0x7fcad003bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcad4006fd0 tx=0x7fcad4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:45.764+0000 7fcae2a00700 1 -- 172.21.15.62:0/2690197048 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcadc01a430 con 0x7fcaec10ef20 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.017+0000 7fcaf0c00700 1 -- 172.21.15.62:0/2690197048 --> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fcaec07fdd0 con 0x7fcad0039660 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.029+0000 7fcae2a00700 1 -- 172.21.15.62:0/2690197048 <== mgr.14120 v2:172.21.15.62:6800/1356430694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcaec07fdd0 con 0x7fcad0039660 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.032+0000 7fcaf0c00700 1 -- 172.21.15.62:0/2690197048 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fcad0039660 msgr2=0x7fcad003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.032+0000 7fcaf0c00700 1 --2- 172.21.15.62:0/2690197048 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fcad0039660 0x7fcad003bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcad4006fd0 tx=0x7fcad4006e40 comp rx=0 tx=0).stop 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.033+0000 7fcaf0c00700 1 -- 172.21.15.62:0/2690197048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 msgr2=0x7fcaec1a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.033+0000 7fcaf0c00700 1 --2- 172.21.15.62:0/2690197048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec1a2b00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcadc011bd0 tx=0x7fcadc011cb0 comp rx=0 tx=0).stop 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.033+0000 7fcaf0c00700 1 -- 172.21.15.62:0/2690197048 shutdown_connections 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.033+0000 7fcaf0c00700 1 --2- 172.21.15.62:0/2690197048 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fcad0039660 0x7fcad003bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:46.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.033+0000 7fcaf0c00700 1 --2- 172.21.15.62:0/2690197048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaec10ef20 0x7fcaec1a2b00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:46.806 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.033+0000 7fcaf0c00700 1 -- 172.21.15.62:0/2690197048 >> 172.21.15.62:0/2690197048 conn(0x7fcaec07d9e0 msgr2=0x7fcaec07f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:46.806 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.033+0000 7fcaf0c00700 1 -- 172.21.15.62:0/2690197048 shutdown_connections 2024-04-02T14:32:46.806 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:46.033+0000 7fcaf0c00700 1 -- 172.21.15.62:0/2690197048 wait complete. 2024-04-02T14:32:48.477 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-02T14:32:48.477 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.367+0000 7f9f4f200700 1 Processor -- start 2024-04-02T14:32:48.477 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.368+0000 7f9f4f200700 1 -- start start 2024-04-02T14:32:48.477 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.368+0000 7f9f4f200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f481086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:48.477 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.368+0000 7f9f4f200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48108ca0 con 0x7f9f481082b0 2024-04-02T14:32:48.477 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.368+0000 7f9f47e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f481086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.369+0000 7f9f47e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f481086d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38672/0 (socket says 172.21.15.62:38672) 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.369+0000 7f9f47e00700 1 -- 172.21.15.62:0/2744695700 learned_addr learned my addr 172.21.15.62:0/2744695700 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.369+0000 7f9f47e00700 1 -- 172.21.15.62:0/2744695700 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f481094b0 con 0x7f9f481082b0 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.369+0000 7f9f47e00700 1 --2- 172.21.15.62:0/2744695700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f481086d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9f380098a0 tx=0x7f9f38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7b43e58d61ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.370+0000 7f9f46a00700 1 -- 172.21.15.62:0/2744695700 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f38004160 con 0x7f9f481082b0 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.370+0000 7f9f46a00700 1 -- 172.21.15.62:0/2744695700 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f9f38004300 con 0x7f9f481082b0 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.370+0000 7f9f46a00700 1 -- 172.21.15.62:0/2744695700 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f38011c00 con 0x7f9f481082b0 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.370+0000 7f9f4f200700 1 -- 172.21.15.62:0/2744695700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 msgr2=0x7f9f481086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.370+0000 7f9f4f200700 1 --2- 172.21.15.62:0/2744695700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f481086d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9f380098a0 tx=0x7f9f38009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.370+0000 7f9f4f200700 1 -- 172.21.15.62:0/2744695700 shutdown_connections 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.370+0000 7f9f4f200700 1 --2- 172.21.15.62:0/2744695700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f481086d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.370+0000 7f9f4f200700 1 -- 172.21.15.62:0/2744695700 >> 172.21.15.62:0/2744695700 conn(0x7f9f48103890 msgr2=0x7f9f48105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:48.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.371+0000 7f9f4f200700 1 -- 172.21.15.62:0/2744695700 shutdown_connections 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.371+0000 7f9f4f200700 1 -- 172.21.15.62:0/2744695700 wait complete. 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.372+0000 7f9f4f200700 1 Processor -- start 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.372+0000 7f9f4f200700 1 -- start start 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.372+0000 7f9f4f200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f4819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.372+0000 7f9f4f200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4810c860 con 0x7f9f481082b0 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.372+0000 7f9f47e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f4819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.372+0000 7f9f47e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f4819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38678/0 (socket says 172.21.15.62:38678) 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.372+0000 7f9f47e00700 1 -- 172.21.15.62:0/460874949 learned_addr learned my addr 172.21.15.62:0/460874949 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.373+0000 7f9f47e00700 1 -- 172.21.15.62:0/460874949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4819ea80 con 0x7f9f481082b0 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.373+0000 7f9f47e00700 1 --2- 172.21.15.62:0/460874949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f4819e540 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f38000c00 tx=0x7f9f38011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.373+0000 7f9f44c00700 1 -- 172.21.15.62:0/460874949 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f38003b20 con 0x7f9f481082b0 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.373+0000 7f9f44c00700 1 -- 172.21.15.62:0/460874949 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f9f38003cc0 con 0x7f9f481082b0 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.374+0000 7f9f44c00700 1 -- 172.21.15.62:0/460874949 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f380249a0 con 0x7f9f481082b0 2024-04-02T14:32:48.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.374+0000 7f9f4f200700 1 -- 172.21.15.62:0/460874949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4819ed10 con 0x7f9f481082b0 2024-04-02T14:32:48.480 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.374+0000 7f9f4f200700 1 -- 172.21.15.62:0/460874949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f481a1a00 con 0x7f9f481082b0 2024-04-02T14:32:48.480 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.375+0000 7f9f4f200700 1 -- 172.21.15.62:0/460874949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4810c9e0 con 0x7f9f481082b0 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.375+0000 7f9f44c00700 1 -- 172.21.15.62:0/460874949 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9f38022020 con 0x7f9f481082b0 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.376+0000 7f9f44c00700 1 --2- 172.21.15.62:0/460874949 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f9f2c039600 0x7f9f2c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.376+0000 7f9f44c00700 1 -- 172.21.15.62:0/460874949 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9f3804d7e0 con 0x7f9f481082b0 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.376+0000 7f9f47400700 1 --2- 172.21.15.62:0/460874949 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f9f2c039600 0x7f9f2c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.377+0000 7f9f47400700 1 --2- 172.21.15.62:0/460874949 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f9f2c039600 0x7f9f2c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f34006fd0 tx=0x7f9f34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.381+0000 7f9f44c00700 1 -- 172.21.15.62:0/460874949 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9f3804d130 con 0x7f9f481082b0 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.609+0000 7f9f4f200700 1 -- 172.21.15.62:0/460874949 --> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9f48104ac0 con 0x7f9f2c039600 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.611+0000 7f9f44c00700 1 -- 172.21.15.62:0/460874949 <== mgr.14120 v2:172.21.15.62:6800/1356430694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9f48104ac0 con 0x7f9f2c039600 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.614+0000 7f9f4f200700 1 -- 172.21.15.62:0/460874949 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f9f2c039600 msgr2=0x7f9f2c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.614+0000 7f9f4f200700 1 --2- 172.21.15.62:0/460874949 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f9f2c039600 0x7f9f2c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f34006fd0 tx=0x7f9f34006e40 comp rx=0 tx=0).stop 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.614+0000 7f9f4f200700 1 -- 172.21.15.62:0/460874949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 msgr2=0x7f9f4819e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:48.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.614+0000 7f9f4f200700 1 --2- 172.21.15.62:0/460874949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f4819e540 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f38000c00 tx=0x7f9f38011c80 comp rx=0 tx=0).stop 2024-04-02T14:32:48.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.614+0000 7f9f4f200700 1 -- 172.21.15.62:0/460874949 shutdown_connections 2024-04-02T14:32:48.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.614+0000 7f9f4f200700 1 --2- 172.21.15.62:0/460874949 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f9f2c039600 0x7f9f2c03bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:48.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.614+0000 7f9f4f200700 1 --2- 172.21.15.62:0/460874949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481082b0 0x7f9f4819e540 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:48.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.614+0000 7f9f4f200700 1 -- 172.21.15.62:0/460874949 >> 172.21.15.62:0/460874949 conn(0x7f9f48103890 msgr2=0x7f9f481043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:48.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.614+0000 7f9f4f200700 1 -- 172.21.15.62:0/460874949 shutdown_connections 2024-04-02T14:32:48.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:47.614+0000 7f9f4f200700 1 -- 172.21.15.62:0/460874949 wait complete. 2024-04-02T14:32:48.482 INFO:teuthology.orchestra.run.smithi062.stdout:Generating ssh key... 2024-04-02T14:32:48.777 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:48 smithi062 ceph-mon[32116]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:32:49.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-mon[32116]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:32:49.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-mon[32116]: Generating ssh key... 2024-04-02T14:32:50.095 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: Generating public/private rsa key pair. 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: Your identification has been saved in /tmp/tmps4_9kigy/key. 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: Your public key has been saved in /tmp/tmps4_9kigy/key.pub. 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: The key fingerprint is: 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: SHA256:qxoAOE4ICk1KncLx11DRhJ4wJpFCOrKUe5hug6bM9mo ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: The key's randomart image is: 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: +---[RSA 3072]----+ 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: |=*+.+..o=. | 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: |O=+= +o. . | 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: |@+o.o.+.. | 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: |==+ . o | 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: |.=.. S | 2024-04-02T14:32:50.096 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: |o .. . | 2024-04-02T14:32:50.097 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: |.= . . | 2024-04-02T14:32:50.097 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: |*E. . . | 2024-04-02T14:32:50.097 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: |++o.... | 2024-04-02T14:32:50.097 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:32:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: +----[SHA256]-----+ 2024-04-02T14:32:50.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.116+0000 7f10dde00700 1 Processor -- start 2024-04-02T14:32:50.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.117+0000 7f10dde00700 1 -- start start 2024-04-02T14:32:50.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.117+0000 7f10dde00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d8109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:50.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.117+0000 7f10dde00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f10d810a2a0 con 0x7f10d81098b0 2024-04-02T14:32:50.428 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.118+0000 7f10d6a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d8109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:50.428 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.118+0000 7f10d6a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d8109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38686/0 (socket says 172.21.15.62:38686) 2024-04-02T14:32:50.428 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.118+0000 7f10d6a00700 1 -- 172.21.15.62:0/3604848621 learned_addr learned my addr 172.21.15.62:0/3604848621 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:50.428 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.118+0000 7f10d6a00700 1 -- 172.21.15.62:0/3604848621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d810a420 con 0x7f10d81098b0 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.119+0000 7f10d6a00700 1 --2- 172.21.15.62:0/3604848621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d8109cd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f10c80098a0 tx=0x7f10c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c251201f2ee67dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.119+0000 7f10d6000700 1 -- 172.21.15.62:0/3604848621 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10c8004160 con 0x7f10d81098b0 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.119+0000 7f10d6000700 1 -- 172.21.15.62:0/3604848621 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f10c8004300 con 0x7f10d81098b0 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.119+0000 7f10d6000700 1 -- 172.21.15.62:0/3604848621 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10c8011c00 con 0x7f10d81098b0 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.119+0000 7f10dde00700 1 -- 172.21.15.62:0/3604848621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 msgr2=0x7f10d8109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.119+0000 7f10dde00700 1 --2- 172.21.15.62:0/3604848621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d8109cd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f10c80098a0 tx=0x7f10c8009bb0 comp rx=0 tx=0).stop 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.120+0000 7f10dde00700 1 -- 172.21.15.62:0/3604848621 shutdown_connections 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.120+0000 7f10dde00700 1 --2- 172.21.15.62:0/3604848621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d8109cd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.120+0000 7f10dde00700 1 -- 172.21.15.62:0/3604848621 >> 172.21.15.62:0/3604848621 conn(0x7f10d8101f10 msgr2=0x7f10d8104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.120+0000 7f10dde00700 1 -- 172.21.15.62:0/3604848621 shutdown_connections 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.120+0000 7f10dde00700 1 -- 172.21.15.62:0/3604848621 wait complete. 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.121+0000 7f10dde00700 1 Processor -- start 2024-04-02T14:32:50.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.121+0000 7f10dde00700 1 -- start start 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.122+0000 7f10dde00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d819e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.122+0000 7f10dde00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f10d8107170 con 0x7f10d81098b0 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.122+0000 7f10d6a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d819e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.122+0000 7f10d6a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d819e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38698/0 (socket says 172.21.15.62:38698) 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.122+0000 7f10d6a00700 1 -- 172.21.15.62:0/3056658826 learned_addr learned my addr 172.21.15.62:0/3056658826 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.122+0000 7f10d6a00700 1 -- 172.21.15.62:0/3056658826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d819ea20 con 0x7f10d81098b0 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.123+0000 7f10d6a00700 1 --2- 172.21.15.62:0/3056658826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d819e4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10c80118c0 tx=0x7f10c80046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.123+0000 7f10dca00700 1 -- 172.21.15.62:0/3056658826 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10c8003af0 con 0x7f10d81098b0 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.123+0000 7f10dca00700 1 -- 172.21.15.62:0/3056658826 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f10c8003c90 con 0x7f10d81098b0 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.123+0000 7f10dca00700 1 -- 172.21.15.62:0/3056658826 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10c80247d0 con 0x7f10d81098b0 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.123+0000 7f10dde00700 1 -- 172.21.15.62:0/3056658826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d819ed10 con 0x7f10d81098b0 2024-04-02T14:32:50.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.123+0000 7f10dde00700 1 -- 172.21.15.62:0/3056658826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d81a19a0 con 0x7f10d81098b0 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.124+0000 7f10dde00700 1 -- 172.21.15.62:0/3056658826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d8063c20 con 0x7f10d81098b0 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.125+0000 7f10dca00700 1 -- 172.21.15.62:0/3056658826 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f10c8022020 con 0x7f10d81098b0 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.125+0000 7f10dca00700 1 --2- 172.21.15.62:0/3056658826 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f10bc039560 0x7f10bc03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.125+0000 7f10dca00700 1 -- 172.21.15.62:0/3056658826 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f10c804d7d0 con 0x7f10d81098b0 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.125+0000 7f10ce000700 1 --2- 172.21.15.62:0/3056658826 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f10bc039560 0x7f10bc03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.126+0000 7f10ce000700 1 --2- 172.21.15.62:0/3056658826 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f10bc039560 0x7f10bc03ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f10c0006fd0 tx=0x7f10c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.130+0000 7f10dca00700 1 -- 172.21.15.62:0/3056658826 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f10c804d120 con 0x7f10d81098b0 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.364+0000 7f10dde00700 1 -- 172.21.15.62:0/3056658826 --> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f10d81a1b70 con 0x7f10bc039560 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.695+0000 7f10dca00700 1 -- 172.21.15.62:0/3056658826 <== mgr.14120 v2:172.21.15.62:6800/1356430694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f10d81a1b70 con 0x7f10bc039560 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.698+0000 7f10dde00700 1 -- 172.21.15.62:0/3056658826 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f10bc039560 msgr2=0x7f10bc03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.698+0000 7f10dde00700 1 --2- 172.21.15.62:0/3056658826 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f10bc039560 0x7f10bc03ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f10c0006fd0 tx=0x7f10c0006e40 comp rx=0 tx=0).stop 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.698+0000 7f10dde00700 1 -- 172.21.15.62:0/3056658826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 msgr2=0x7f10d819e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.698+0000 7f10dde00700 1 --2- 172.21.15.62:0/3056658826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d819e4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10c80118c0 tx=0x7f10c80046a0 comp rx=0 tx=0).stop 2024-04-02T14:32:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.698+0000 7f10dde00700 1 -- 172.21.15.62:0/3056658826 shutdown_connections 2024-04-02T14:32:50.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.698+0000 7f10dde00700 1 --2- 172.21.15.62:0/3056658826 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f10bc039560 0x7f10bc03ba20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:50.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.698+0000 7f10dde00700 1 --2- 172.21.15.62:0/3056658826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d81098b0 0x7f10d819e4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:50.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.698+0000 7f10dde00700 1 -- 172.21.15.62:0/3056658826 >> 172.21.15.62:0/3056658826 conn(0x7f10d8101f10 msgr2=0x7f10d8102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:50.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.698+0000 7f10dde00700 1 -- 172.21.15.62:0/3056658826 shutdown_connections 2024-04-02T14:32:50.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:49.698+0000 7f10dde00700 1 -- 172.21.15.62:0/3056658826 wait complete. 2024-04-02T14:32:50.991 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:50 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:32:50.991 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:50 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:32:51.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:51 smithi062 ceph-mon[32116]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:32:51.986 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsYp7MRg6qIY/ULuJBU7QvMByDRbucEgKtj0FSEfB60hQqwQO03ZWsVd3YgJmE4AhXzmACspll45b+tektFLha7f2Un8a13SanjnHpaewa3gg/3Q/uFfdcUtOUtmmYCjJRQcfe059yeVyPLT20GyLdd2AT9AvAbQbmTuZeRKorlPyahK3voo2rufuzWkmkx8tWkydvVpGVuvgOqjntOGs0nmb4bW+O/NkUHNIoQPGIP42cYuV06qmwZMKxN/QBJfIifa2qs2TMtufzaXs76EJhQ36ETypoEK45bAr2p0PqZqCt99xTZS9YJr338bVMVU1YQZXrAzH5EZ/8J9OyyyrwFgNDuBG5W+ujKB46tLNeHYUcXyO8qU+QdHgIAYKn2WYqKbuGyxc/w5JHbaUf/LZ5CL5Er++/hrlyDyrg5yrOz4uxRLCtTHDh40CujF+SR8FLfWKoxB2a+auRRllzGFj6M4bvqUjBQ9vOkrAhoaAOor3wSNNHaWbfEh7RGiNpEi8= ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:32:51.986 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.048+0000 7f032da00700 1 Processor -- start 2024-04-02T14:32:51.986 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.048+0000 7f032da00700 1 -- start start 2024-04-02T14:32:51.986 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.049+0000 7f032da00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f0328106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.049+0000 7f032da00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f03280638d0 con 0x7f0328106b80 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.050+0000 7f0326a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f0328106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.050+0000 7f0326a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f0328106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38708/0 (socket says 172.21.15.62:38708) 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.050+0000 7f0326a00700 1 -- 172.21.15.62:0/798660051 learned_addr learned my addr 172.21.15.62:0/798660051 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.050+0000 7f0326a00700 1 -- 172.21.15.62:0/798660051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03281074e0 con 0x7f0328106b80 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.050+0000 7f0326a00700 1 --2- 172.21.15.62:0/798660051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f0328106fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0318009e80 tx=0x7f031800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b563ec1e90dc48c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.051+0000 7f0325600700 1 -- 172.21.15.62:0/798660051 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0318017070 con 0x7f0328106b80 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.051+0000 7f0325600700 1 -- 172.21.15.62:0/798660051 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f031800b8c0 con 0x7f0328106b80 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.051+0000 7f0325600700 1 -- 172.21.15.62:0/798660051 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0318004620 con 0x7f0328106b80 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.051+0000 7f032da00700 1 -- 172.21.15.62:0/798660051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 msgr2=0x7f0328106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.051+0000 7f032da00700 1 --2- 172.21.15.62:0/798660051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f0328106fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0318009e80 tx=0x7f031800b330 comp rx=0 tx=0).stop 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.051+0000 7f032da00700 1 -- 172.21.15.62:0/798660051 shutdown_connections 2024-04-02T14:32:51.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.052+0000 7f032da00700 1 --2- 172.21.15.62:0/798660051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f0328106fa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.052+0000 7f032da00700 1 -- 172.21.15.62:0/798660051 >> 172.21.15.62:0/798660051 conn(0x7f0328102780 msgr2=0x7f0328104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.052+0000 7f032da00700 1 -- 172.21.15.62:0/798660051 shutdown_connections 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.052+0000 7f032da00700 1 -- 172.21.15.62:0/798660051 wait complete. 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.053+0000 7f032da00700 1 Processor -- start 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.053+0000 7f032da00700 1 -- start start 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.053+0000 7f032da00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f032819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.053+0000 7f032da00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f03281085d0 con 0x7f0328106b80 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.054+0000 7f0326a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f032819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.054+0000 7f0326a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f032819e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38720/0 (socket says 172.21.15.62:38720) 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.054+0000 7f0326a00700 1 -- 172.21.15.62:0/1870285205 learned_addr learned my addr 172.21.15.62:0/1870285205 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.054+0000 7f0326a00700 1 -- 172.21.15.62:0/1870285205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032819e9d0 con 0x7f0328106b80 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.054+0000 7f0326a00700 1 --2- 172.21.15.62:0/1870285205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f032819e490 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f031800b7b0 tx=0x7f0318003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.055+0000 7f031f400700 1 -- 172.21.15.62:0/1870285205 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0318021020 con 0x7f0328106b80 2024-04-02T14:32:51.988 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.055+0000 7f032da00700 1 -- 172.21.15.62:0/1870285205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032819ec60 con 0x7f0328106b80 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.055+0000 7f031f400700 1 -- 172.21.15.62:0/1870285205 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f031800be00 con 0x7f0328106b80 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.055+0000 7f031f400700 1 -- 172.21.15.62:0/1870285205 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0318017040 con 0x7f0328106b80 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.055+0000 7f032da00700 1 -- 172.21.15.62:0/1870285205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03281a1950 con 0x7f0328106b80 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.056+0000 7f032da00700 1 -- 172.21.15.62:0/1870285205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0328063c20 con 0x7f0328106b80 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.056+0000 7f031f400700 1 -- 172.21.15.62:0/1870285205 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0318024020 con 0x7f0328106b80 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.056+0000 7f031f400700 1 --2- 172.21.15.62:0/1870285205 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f030c039600 0x7f030c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.056+0000 7f031f400700 1 -- 172.21.15.62:0/1870285205 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f031800f050 con 0x7f0328106b80 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.057+0000 7f0326000700 1 --2- 172.21.15.62:0/1870285205 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f030c039600 0x7f030c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.058+0000 7f0326000700 1 --2- 172.21.15.62:0/1870285205 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f030c039600 0x7f030c03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0314006fd0 tx=0x7f0314006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.062+0000 7f031f400700 1 -- 172.21.15.62:0/1870285205 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f031802b020 con 0x7f0328106b80 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.295+0000 7f032da00700 1 -- 172.21.15.62:0/1870285205 --> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f032802e190 con 0x7f030c039600 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.297+0000 7f031f400700 1 -- 172.21.15.62:0/1870285205 <== mgr.14120 v2:172.21.15.62:6800/1356430694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f031400a040 con 0x7f030c039600 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.300+0000 7f032da00700 1 -- 172.21.15.62:0/1870285205 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f030c039600 msgr2=0x7f030c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:51.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.300+0000 7f032da00700 1 --2- 172.21.15.62:0/1870285205 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f030c039600 0x7f030c03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0314006fd0 tx=0x7f0314006e40 comp rx=0 tx=0).stop 2024-04-02T14:32:51.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.300+0000 7f032da00700 1 -- 172.21.15.62:0/1870285205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 msgr2=0x7f032819e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:32:51.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.300+0000 7f032da00700 1 --2- 172.21.15.62:0/1870285205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f032819e490 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f031800b7b0 tx=0x7f0318003d00 comp rx=0 tx=0).stop 2024-04-02T14:32:51.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.300+0000 7f032da00700 1 -- 172.21.15.62:0/1870285205 shutdown_connections 2024-04-02T14:32:51.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.300+0000 7f032da00700 1 --2- 172.21.15.62:0/1870285205 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f030c039600 0x7f030c03bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:51.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.300+0000 7f032da00700 1 --2- 172.21.15.62:0/1870285205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0328106b80 0x7f032819e490 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:32:51.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.300+0000 7f032da00700 1 -- 172.21.15.62:0/1870285205 >> 172.21.15.62:0/1870285205 conn(0x7f0328102780 msgr2=0x7f0328195130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:32:51.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.300+0000 7f032da00700 1 -- 172.21.15.62:0/1870285205 shutdown_connections 2024-04-02T14:32:51.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:51.300+0000 7f032da00700 1 -- 172.21.15.62:0/1870285205 wait complete. 2024-04-02T14:32:51.990 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-02T14:32:51.990 INFO:teuthology.orchestra.run.smithi062.stdout:Adding key to root@localhost authorized_keys... 2024-04-02T14:32:51.991 INFO:teuthology.orchestra.run.smithi062.stdout:Adding host smithi062... 2024-04-02T14:32:53.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:53 smithi062 ceph-mon[32116]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:32:54.724 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:32:54 smithi062 ceph-mon[32116]: Deploying cephadm binary to smithi062 2024-04-02T14:33:00.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Added host 'smithi062' with addr '172.21.15.62' 2024-04-02T14:33:00.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.610+0000 7f4033600700 1 Processor -- start 2024-04-02T14:33:00.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.610+0000 7f4033600700 1 -- start start 2024-04-02T14:33:00.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.611+0000 7f4033600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:00.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.611+0000 7f4033600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f402c063890 con 0x7f402c106bc0 2024-04-02T14:33:00.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.611+0000 7f4030c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:00.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.611+0000 7f4030c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c106fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39626/0 (socket says 172.21.15.62:39626) 2024-04-02T14:33:00.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.611+0000 7f4030c00700 1 -- 172.21.15.62:0/816821528 learned_addr learned my addr 172.21.15.62:0/816821528 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:00.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.612+0000 7f4030c00700 1 -- 172.21.15.62:0/816821528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c107520 con 0x7f402c106bc0 2024-04-02T14:33:00.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.612+0000 7f4030c00700 1 --2- 172.21.15.62:0/816821528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c106fe0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40200098a0 tx=0x7f4020009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3f2fd75fc17632 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:00.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.612+0000 7f402b400700 1 -- 172.21.15.62:0/816821528 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4020004160 con 0x7f402c106bc0 2024-04-02T14:33:00.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.612+0000 7f402b400700 1 -- 172.21.15.62:0/816821528 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f4020004300 con 0x7f402c106bc0 2024-04-02T14:33:00.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.612+0000 7f402b400700 1 -- 172.21.15.62:0/816821528 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4020011c00 con 0x7f402c106bc0 2024-04-02T14:33:00.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.613+0000 7f4033600700 1 -- 172.21.15.62:0/816821528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 msgr2=0x7f402c106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:00.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.613+0000 7f4033600700 1 --2- 172.21.15.62:0/816821528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c106fe0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40200098a0 tx=0x7f4020009bb0 comp rx=0 tx=0).stop 2024-04-02T14:33:00.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.613+0000 7f4033600700 1 -- 172.21.15.62:0/816821528 shutdown_connections 2024-04-02T14:33:00.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.613+0000 7f4033600700 1 --2- 172.21.15.62:0/816821528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c106fe0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:00.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.613+0000 7f4033600700 1 -- 172.21.15.62:0/816821528 >> 172.21.15.62:0/816821528 conn(0x7f402c102780 msgr2=0x7f402c104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:00.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.613+0000 7f4033600700 1 -- 172.21.15.62:0/816821528 shutdown_connections 2024-04-02T14:33:00.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.613+0000 7f4033600700 1 -- 172.21.15.62:0/816821528 wait complete. 2024-04-02T14:33:00.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.614+0000 7f4033600700 1 Processor -- start 2024-04-02T14:33:00.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.615+0000 7f4033600700 1 -- start start 2024-04-02T14:33:00.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.615+0000 7f4033600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:00.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.615+0000 7f4033600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f402c108610 con 0x7f402c106bc0 2024-04-02T14:33:00.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.615+0000 7f4030c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:00.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.615+0000 7f4030c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c19e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39642/0 (socket says 172.21.15.62:39642) 2024-04-02T14:33:00.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.615+0000 7f4030c00700 1 -- 172.21.15.62:0/562221635 learned_addr learned my addr 172.21.15.62:0/562221635 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.616+0000 7f4030c00700 1 -- 172.21.15.62:0/562221635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c19ea10 con 0x7f402c106bc0 2024-04-02T14:33:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.616+0000 7f4030c00700 1 --2- 172.21.15.62:0/562221635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c19e4d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4020011bd0 tx=0x7f4020011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.616+0000 7f4029600700 1 -- 172.21.15.62:0/562221635 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4020003b20 con 0x7f402c106bc0 2024-04-02T14:33:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.616+0000 7f4029600700 1 -- 172.21.15.62:0/562221635 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f4020003cc0 con 0x7f402c106bc0 2024-04-02T14:33:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.616+0000 7f4029600700 1 -- 172.21.15.62:0/562221635 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4020024930 con 0x7f402c106bc0 2024-04-02T14:33:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.616+0000 7f4033600700 1 -- 172.21.15.62:0/562221635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c19eca0 con 0x7f402c106bc0 2024-04-02T14:33:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.617+0000 7f4033600700 1 -- 172.21.15.62:0/562221635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c1a1990 con 0x7f402c106bc0 2024-04-02T14:33:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.618+0000 7f4033600700 1 -- 172.21.15.62:0/562221635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c063c20 con 0x7f402c106bc0 2024-04-02T14:33:00.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.618+0000 7f4029600700 1 -- 172.21.15.62:0/562221635 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4020022020 con 0x7f402c106bc0 2024-04-02T14:33:00.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.618+0000 7f4029600700 1 --2- 172.21.15.62:0/562221635 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f4014039600 0x7f401403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:00.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.618+0000 7f4029600700 1 -- 172.21.15.62:0/562221635 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f402004d7a0 con 0x7f402c106bc0 2024-04-02T14:33:00.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.618+0000 7f402be00700 1 --2- 172.21.15.62:0/562221635 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f4014039600 0x7f401403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:00.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.619+0000 7f402be00700 1 --2- 172.21.15.62:0/562221635 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f4014039600 0x7f401403bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4018006fd0 tx=0x7f4018006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:00.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.624+0000 7f4029600700 1 -- 172.21.15.62:0/562221635 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f402004d130 con 0x7f402c106bc0 2024-04-02T14:33:00.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:52.860+0000 7f4033600700 1 -- 172.21.15.62:0/562221635 --> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}) -- 0x7f402c0008d0 con 0x7f4014039600 2024-04-02T14:33:00.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:59.586+0000 7f4029600700 1 -- 172.21.15.62:0/562221635 <== mgr.14120 v2:172.21.15.62:6800/1356430694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f402c0008d0 con 0x7f4014039600 2024-04-02T14:33:00.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:59.590+0000 7f4033600700 1 -- 172.21.15.62:0/562221635 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f4014039600 msgr2=0x7f401403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:00.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:59.590+0000 7f4033600700 1 --2- 172.21.15.62:0/562221635 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f4014039600 0x7f401403bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4018006fd0 tx=0x7f4018006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:00.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:59.590+0000 7f4033600700 1 -- 172.21.15.62:0/562221635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 msgr2=0x7f402c19e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:00.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:59.590+0000 7f4033600700 1 --2- 172.21.15.62:0/562221635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c19e4d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4020011bd0 tx=0x7f4020011cb0 comp rx=0 tx=0).stop 2024-04-02T14:33:00.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:59.590+0000 7f4033600700 1 -- 172.21.15.62:0/562221635 shutdown_connections 2024-04-02T14:33:00.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:59.590+0000 7f4033600700 1 --2- 172.21.15.62:0/562221635 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f4014039600 0x7f401403bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:00.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:59.590+0000 7f4033600700 1 --2- 172.21.15.62:0/562221635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f402c106bc0 0x7f402c19e4d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:00.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:59.590+0000 7f4033600700 1 -- 172.21.15.62:0/562221635 >> 172.21.15.62:0/562221635 conn(0x7f402c102780 msgr2=0x7f402c195110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:00.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:59.591+0000 7f4033600700 1 -- 172.21.15.62:0/562221635 shutdown_connections 2024-04-02T14:33:00.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:32:59.591+0000 7f4033600700 1 -- 172.21.15.62:0/562221635 wait complete. 2024-04-02T14:33:00.418 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mon service... 2024-04-02T14:33:00.766 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:00 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:33:00.766 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:00 smithi062 ceph-mon[32116]: Added host smithi062 2024-04-02T14:33:00.766 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:00 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:33:02.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:02 smithi062 ceph-mon[32116]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:33:02.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:02 smithi062 ceph-mon[32116]: Saving service mon spec with placement count:5 2024-04-02T14:33:02.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:02 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:33:02.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-02T14:33:02.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.152+0000 7fbb0ee00700 1 Processor -- start 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.152+0000 7fbb0ee00700 1 -- start start 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.153+0000 7fbb0ee00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0810a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.153+0000 7fbb0ee00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0810af00 con 0x7fbb0810a510 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.153+0000 7fbb07e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0810a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.153+0000 7fbb07e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0810a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39644/0 (socket says 172.21.15.62:39644) 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.153+0000 7fbb07e00700 1 -- 172.21.15.62:0/3397309802 learned_addr learned my addr 172.21.15.62:0/3397309802 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.154+0000 7fbb07e00700 1 -- 172.21.15.62:0/3397309802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0810b6c0 con 0x7fbb0810a510 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.154+0000 7fbb07e00700 1 --2- 172.21.15.62:0/3397309802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0810a930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80098a0 tx=0x7fbaf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf65552cce185b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.154+0000 7fbb06a00700 1 -- 172.21.15.62:0/3397309802 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaf8004160 con 0x7fbb0810a510 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.155+0000 7fbb06a00700 1 -- 172.21.15.62:0/3397309802 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fbaf8004300 con 0x7fbb0810a510 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.155+0000 7fbb06a00700 1 -- 172.21.15.62:0/3397309802 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaf8011c00 con 0x7fbb0810a510 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.155+0000 7fbb0ee00700 1 -- 172.21.15.62:0/3397309802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 msgr2=0x7fbb0810a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:02.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.155+0000 7fbb0ee00700 1 --2- 172.21.15.62:0/3397309802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0810a930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80098a0 tx=0x7fbaf8009bb0 comp rx=0 tx=0).stop 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.155+0000 7fbb0ee00700 1 -- 172.21.15.62:0/3397309802 shutdown_connections 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.155+0000 7fbb0ee00700 1 --2- 172.21.15.62:0/3397309802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0810a930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.155+0000 7fbb0ee00700 1 -- 172.21.15.62:0/3397309802 >> 172.21.15.62:0/3397309802 conn(0x7fbb0807d9e0 msgr2=0x7fbb081084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.155+0000 7fbb0ee00700 1 -- 172.21.15.62:0/3397309802 shutdown_connections 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.155+0000 7fbb0ee00700 1 -- 172.21.15.62:0/3397309802 wait complete. 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.156+0000 7fbb0ee00700 1 Processor -- start 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.157+0000 7fbb0ee00700 1 -- start start 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.157+0000 7fbb0ee00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.157+0000 7fbb0ee00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0810ea70 con 0x7fbb0810a510 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.157+0000 7fbb07e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.157+0000 7fbb07e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0819e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39654/0 (socket says 172.21.15.62:39654) 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.157+0000 7fbb07e00700 1 -- 172.21.15.62:0/2582294768 learned_addr learned my addr 172.21.15.62:0/2582294768 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:02.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.158+0000 7fbb07e00700 1 -- 172.21.15.62:0/2582294768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0819ea60 con 0x7fbb0810a510 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.158+0000 7fbb07e00700 1 --2- 172.21.15.62:0/2582294768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0819e520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80118c0 tx=0x7fbaf8011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.158+0000 7fbb04c00700 1 -- 172.21.15.62:0/2582294768 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaf8003af0 con 0x7fbb0810a510 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.158+0000 7fbb04c00700 1 -- 172.21.15.62:0/2582294768 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fbaf8020070 con 0x7fbb0810a510 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.158+0000 7fbb04c00700 1 -- 172.21.15.62:0/2582294768 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaf80195e0 con 0x7fbb0810a510 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.158+0000 7fbb0ee00700 1 -- 172.21.15.62:0/2582294768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0819ecf0 con 0x7fbb0810a510 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.158+0000 7fbb0ee00700 1 -- 172.21.15.62:0/2582294768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb081a19e0 con 0x7fbb0810a510 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.159+0000 7fbb04c00700 1 -- 172.21.15.62:0/2582294768 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbaf801a9a0 con 0x7fbb0810a510 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.159+0000 7fbb0ee00700 1 -- 172.21.15.62:0/2582294768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0810ebf0 con 0x7fbb0810a510 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.160+0000 7fbb04c00700 1 --2- 172.21.15.62:0/2582294768 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fbaec039600 0x7fbaec03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.160+0000 7fbb04c00700 1 -- 172.21.15.62:0/2582294768 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbaf804daf0 con 0x7fbb0810a510 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.161+0000 7fbb07400700 1 --2- 172.21.15.62:0/2582294768 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fbaec039600 0x7fbaec03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.162+0000 7fbb07400700 1 --2- 172.21.15.62:0/2582294768 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fbaec039600 0x7fbaec03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4006fd0 tx=0x7fbaf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.166+0000 7fbb04c00700 1 -- 172.21.15.62:0/2582294768 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbaf800e260 con 0x7fbb0810a510 2024-04-02T14:33:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.420+0000 7fbb0ee00700 1 -- 172.21.15.62:0/2582294768 --> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbb08108300 con 0x7fbaec039600 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.431+0000 7fbb04c00700 1 -- 172.21.15.62:0/2582294768 <== mgr.14120 v2:172.21.15.62:6800/1356430694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbb08108300 con 0x7fbaec039600 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.434+0000 7fbb0ee00700 1 -- 172.21.15.62:0/2582294768 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fbaec039600 msgr2=0x7fbaec03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.434+0000 7fbb0ee00700 1 --2- 172.21.15.62:0/2582294768 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fbaec039600 0x7fbaec03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4006fd0 tx=0x7fbaf4006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.434+0000 7fbb0ee00700 1 -- 172.21.15.62:0/2582294768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 msgr2=0x7fbb0819e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.434+0000 7fbb0ee00700 1 --2- 172.21.15.62:0/2582294768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0819e520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80118c0 tx=0x7fbaf8011c80 comp rx=0 tx=0).stop 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.434+0000 7fbb0ee00700 1 -- 172.21.15.62:0/2582294768 shutdown_connections 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.434+0000 7fbb0ee00700 1 --2- 172.21.15.62:0/2582294768 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fbaec039600 0x7fbaec03bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.434+0000 7fbb0ee00700 1 --2- 172.21.15.62:0/2582294768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0810a510 0x7fbb0819e520 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.434+0000 7fbb0ee00700 1 -- 172.21.15.62:0/2582294768 >> 172.21.15.62:0/2582294768 conn(0x7fbb0807d9e0 msgr2=0x7fbb08107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.434+0000 7fbb0ee00700 1 -- 172.21.15.62:0/2582294768 shutdown_connections 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:01.435+0000 7fbb0ee00700 1 -- 172.21.15.62:0/2582294768 wait complete. 2024-04-02T14:33:02.754 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mgr service... 2024-04-02T14:33:04.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:04 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:33:04.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:04 smithi062 ceph-mon[32116]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:04.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:04 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:33:04.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:04 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:33:04.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-02T14:33:04.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.642+0000 7f2821800700 1 Processor -- start 2024-04-02T14:33:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.642+0000 7f2821800700 1 -- start start 2024-04-02T14:33:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.642+0000 7f2821800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c10a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.642+0000 7f2821800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f281c10af30 con 0x7f281c10a540 2024-04-02T14:33:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.643+0000 7f281aa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c10a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.643+0000 7f281aa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c10a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44294/0 (socket says 172.21.15.62:44294) 2024-04-02T14:33:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.643+0000 7f281aa00700 1 -- 172.21.15.62:0/393069100 learned_addr learned my addr 172.21.15.62:0/393069100 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.644+0000 7f281aa00700 1 -- 172.21.15.62:0/393069100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c10b6f0 con 0x7f281c10a540 2024-04-02T14:33:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.644+0000 7f281aa00700 1 --2- 172.21.15.62:0/393069100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c10a960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f280c0098a0 tx=0x7f280c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a600c7e862883887 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.644+0000 7f2819600700 1 -- 172.21.15.62:0/393069100 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f280c004160 con 0x7f281c10a540 2024-04-02T14:33:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.644+0000 7f2819600700 1 -- 172.21.15.62:0/393069100 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f280c004300 con 0x7f281c10a540 2024-04-02T14:33:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.644+0000 7f2819600700 1 -- 172.21.15.62:0/393069100 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f280c011bb0 con 0x7f281c10a540 2024-04-02T14:33:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.645+0000 7f2821800700 1 -- 172.21.15.62:0/393069100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 msgr2=0x7f281c10a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.645+0000 7f2821800700 1 --2- 172.21.15.62:0/393069100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c10a960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f280c0098a0 tx=0x7f280c009bb0 comp rx=0 tx=0).stop 2024-04-02T14:33:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.645+0000 7f2821800700 1 -- 172.21.15.62:0/393069100 shutdown_connections 2024-04-02T14:33:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.645+0000 7f2821800700 1 --2- 172.21.15.62:0/393069100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c10a960 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.645+0000 7f2821800700 1 -- 172.21.15.62:0/393069100 >> 172.21.15.62:0/393069100 conn(0x7f281c07d9b0 msgr2=0x7f281c1084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.645+0000 7f2821800700 1 -- 172.21.15.62:0/393069100 shutdown_connections 2024-04-02T14:33:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.645+0000 7f2821800700 1 -- 172.21.15.62:0/393069100 wait complete. 2024-04-02T14:33:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.646+0000 7f2821800700 1 Processor -- start 2024-04-02T14:33:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.647+0000 7f2821800700 1 -- start start 2024-04-02T14:33:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.647+0000 7f2821800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.647+0000 7f2821800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f281c10eaa0 con 0x7f281c10a540 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.647+0000 7f281aa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.647+0000 7f281aa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c19e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44302/0 (socket says 172.21.15.62:44302) 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.647+0000 7f281aa00700 1 -- 172.21.15.62:0/2469528534 learned_addr learned my addr 172.21.15.62:0/2469528534 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.648+0000 7f281aa00700 1 -- 172.21.15.62:0/2469528534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c19ea50 con 0x7f281c10a540 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.648+0000 7f281aa00700 1 --2- 172.21.15.62:0/2469528534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c19e510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f280c011690 tx=0x7f280c011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.648+0000 7f2813400700 1 -- 172.21.15.62:0/2469528534 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f280c0039c0 con 0x7f281c10a540 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.648+0000 7f2813400700 1 -- 172.21.15.62:0/2469528534 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f280c003b60 con 0x7f281c10a540 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.648+0000 7f2821800700 1 -- 172.21.15.62:0/2469528534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c19ece0 con 0x7f281c10a540 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.648+0000 7f2813400700 1 -- 172.21.15.62:0/2469528534 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f280c0197d0 con 0x7f281c10a540 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.649+0000 7f2821800700 1 -- 172.21.15.62:0/2469528534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c19f140 con 0x7f281c10a540 2024-04-02T14:33:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.650+0000 7f2821800700 1 -- 172.21.15.62:0/2469528534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281c063c20 con 0x7f281c10a540 2024-04-02T14:33:04.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.650+0000 7f2813400700 1 -- 172.21.15.62:0/2469528534 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f280c01abe0 con 0x7f281c10a540 2024-04-02T14:33:04.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.651+0000 7f2813400700 1 --2- 172.21.15.62:0/2469528534 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f2800039560 0x7f280003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:04.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.651+0000 7f2813400700 1 -- 172.21.15.62:0/2469528534 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f280c04dda0 con 0x7f281c10a540 2024-04-02T14:33:04.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.651+0000 7f281a000700 1 --2- 172.21.15.62:0/2469528534 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f2800039560 0x7f280003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:04.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.652+0000 7f281a000700 1 --2- 172.21.15.62:0/2469528534 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f2800039560 0x7f280003ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2808006fd0 tx=0x7f2808006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:04.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.657+0000 7f2813400700 1 -- 172.21.15.62:0/2469528534 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f280c00b3e0 con 0x7f281c10a540 2024-04-02T14:33:04.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.918+0000 7f2821800700 1 -- 172.21.15.62:0/2469528534 --> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f281c1082d0 con 0x7f2800039560 2024-04-02T14:33:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.926+0000 7f2813400700 1 -- 172.21.15.62:0/2469528534 <== mgr.14120 v2:172.21.15.62:6800/1356430694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f281c1082d0 con 0x7f2800039560 2024-04-02T14:33:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.930+0000 7f2821800700 1 -- 172.21.15.62:0/2469528534 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f2800039560 msgr2=0x7f280003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.930+0000 7f2821800700 1 --2- 172.21.15.62:0/2469528534 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f2800039560 0x7f280003ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2808006fd0 tx=0x7f2808006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.930+0000 7f2821800700 1 -- 172.21.15.62:0/2469528534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 msgr2=0x7f281c19e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.930+0000 7f2821800700 1 --2- 172.21.15.62:0/2469528534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c19e510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f280c011690 tx=0x7f280c011c30 comp rx=0 tx=0).stop 2024-04-02T14:33:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.930+0000 7f2821800700 1 -- 172.21.15.62:0/2469528534 shutdown_connections 2024-04-02T14:33:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.930+0000 7f2821800700 1 --2- 172.21.15.62:0/2469528534 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f2800039560 0x7f280003ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.930+0000 7f2821800700 1 --2- 172.21.15.62:0/2469528534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c10a540 0x7f281c19e510 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.930+0000 7f2821800700 1 -- 172.21.15.62:0/2469528534 >> 172.21.15.62:0/2469528534 conn(0x7f281c07d9b0 msgr2=0x7f281c107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.930+0000 7f2821800700 1 -- 172.21.15.62:0/2469528534 shutdown_connections 2024-04-02T14:33:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:03.930+0000 7f2821800700 1 -- 172.21.15.62:0/2469528534 wait complete. 2024-04-02T14:33:05.245 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:05 smithi062 ceph-mon[32116]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:33:05.245 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:05 smithi062 ceph-mon[32116]: Saving service mgr spec with placement count:2 2024-04-02T14:33:06.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:06 smithi062 ceph-mon[32116]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:06.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:06 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/847380204' entity='client.admin' 2024-04-02T14:33:06.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.311+0000 7f51f2800700 1 Processor -- start 2024-04-02T14:33:06.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.311+0000 7f51f2800700 1 -- start start 2024-04-02T14:33:06.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.311+0000 7f51f2800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec10a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:06.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.311+0000 7f51f2800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec10af30 con 0x7f51ec10a540 2024-04-02T14:33:06.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.312+0000 7f51eb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec10a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:06.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.312+0000 7f51eb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec10a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44316/0 (socket says 172.21.15.62:44316) 2024-04-02T14:33:06.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.312+0000 7f51eb400700 1 -- 172.21.15.62:0/524001321 learned_addr learned my addr 172.21.15.62:0/524001321 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:06.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.312+0000 7f51eb400700 1 -- 172.21.15.62:0/524001321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec10b6f0 con 0x7f51ec10a540 2024-04-02T14:33:06.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.313+0000 7f51eb400700 1 --2- 172.21.15.62:0/524001321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec10a960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009e80 tx=0x7f51dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d55e765903f8c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.313+0000 7f51eaa00700 1 -- 172.21.15.62:0/524001321 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51dc017070 con 0x7f51ec10a540 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.313+0000 7f51eaa00700 1 -- 172.21.15.62:0/524001321 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f51dc00b8c0 con 0x7f51ec10a540 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.313+0000 7f51eaa00700 1 -- 172.21.15.62:0/524001321 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51dc004580 con 0x7f51ec10a540 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.314+0000 7f51f2800700 1 -- 172.21.15.62:0/524001321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 msgr2=0x7f51ec10a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.314+0000 7f51f2800700 1 --2- 172.21.15.62:0/524001321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec10a960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009e80 tx=0x7f51dc00b330 comp rx=0 tx=0).stop 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.314+0000 7f51f2800700 1 -- 172.21.15.62:0/524001321 shutdown_connections 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.314+0000 7f51f2800700 1 --2- 172.21.15.62:0/524001321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec10a960 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.314+0000 7f51f2800700 1 -- 172.21.15.62:0/524001321 >> 172.21.15.62:0/524001321 conn(0x7f51ec105b20 msgr2=0x7f51ec107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.314+0000 7f51f2800700 1 -- 172.21.15.62:0/524001321 shutdown_connections 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.314+0000 7f51f2800700 1 -- 172.21.15.62:0/524001321 wait complete. 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.315+0000 7f51f2800700 1 Processor -- start 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.315+0000 7f51f2800700 1 -- start start 2024-04-02T14:33:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.315+0000 7f51f2800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec19e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:06.321 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.315+0000 7f51f2800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec10eaa0 con 0x7f51ec10a540 2024-04-02T14:33:06.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.316+0000 7f51eb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec19e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:06.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.316+0000 7f51eb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec19e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44320/0 (socket says 172.21.15.62:44320) 2024-04-02T14:33:06.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.316+0000 7f51eb400700 1 -- 172.21.15.62:0/847380204 learned_addr learned my addr 172.21.15.62:0/847380204 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:06.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.316+0000 7f51eb400700 1 -- 172.21.15.62:0/847380204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec19ea00 con 0x7f51ec10a540 2024-04-02T14:33:06.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.316+0000 7f51eb400700 1 --2- 172.21.15.62:0/847380204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec19e4c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f51dc000f80 tx=0x7f51dc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:06.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.317+0000 7f51e8c00700 1 -- 172.21.15.62:0/847380204 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51dc021020 con 0x7f51ec10a540 2024-04-02T14:33:06.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.317+0000 7f51e8c00700 1 -- 172.21.15.62:0/847380204 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f51dc00bde0 con 0x7f51ec10a540 2024-04-02T14:33:06.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.317+0000 7f51f2800700 1 -- 172.21.15.62:0/847380204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec19ec90 con 0x7f51ec10a540 2024-04-02T14:33:06.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.317+0000 7f51e8c00700 1 -- 172.21.15.62:0/847380204 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51dc017040 con 0x7f51ec10a540 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.317+0000 7f51f2800700 1 -- 172.21.15.62:0/847380204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec1a1980 con 0x7f51ec10a540 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.318+0000 7f51f2800700 1 -- 172.21.15.62:0/847380204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec10ec20 con 0x7f51ec10a540 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.318+0000 7f51e8c00700 1 -- 172.21.15.62:0/847380204 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f51dc024020 con 0x7f51ec10a540 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.319+0000 7f51e8c00700 1 --2- 172.21.15.62:0/847380204 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f51d0039600 0x7f51d003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.319+0000 7f51e8c00700 1 -- 172.21.15.62:0/847380204 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f51dc00f050 con 0x7f51ec10a540 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.319+0000 7f51e2a00700 1 --2- 172.21.15.62:0/847380204 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f51d0039600 0x7f51d003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.320+0000 7f51e2a00700 1 --2- 172.21.15.62:0/847380204 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f51d0039600 0x7f51d003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f51d4006fd0 tx=0x7f51d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.324+0000 7f51e8c00700 1 -- 172.21.15.62:0/847380204 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f51dc02b020 con 0x7f51ec10a540 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.560+0000 7f51f2800700 1 -- 172.21.15.62:0/847380204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f51ec063c20 con 0x7f51ec10a540 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.563+0000 7f51e8c00700 1 -- 172.21.15.62:0/847380204 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f51dc02e350 con 0x7f51ec10a540 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.567+0000 7f51f2800700 1 -- 172.21.15.62:0/847380204 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f51d0039600 msgr2=0x7f51d003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.567+0000 7f51f2800700 1 --2- 172.21.15.62:0/847380204 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f51d0039600 0x7f51d003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f51d4006fd0 tx=0x7f51d4006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.567+0000 7f51f2800700 1 -- 172.21.15.62:0/847380204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 msgr2=0x7f51ec19e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:06.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.567+0000 7f51f2800700 1 --2- 172.21.15.62:0/847380204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec19e4c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f51dc000f80 tx=0x7f51dc003d00 comp rx=0 tx=0).stop 2024-04-02T14:33:06.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.567+0000 7f51f2800700 1 -- 172.21.15.62:0/847380204 shutdown_connections 2024-04-02T14:33:06.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.567+0000 7f51f2800700 1 --2- 172.21.15.62:0/847380204 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f51d0039600 0x7f51d003bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:06.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.567+0000 7f51f2800700 1 --2- 172.21.15.62:0/847380204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec10a540 0x7f51ec19e4c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:06.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.567+0000 7f51f2800700 1 -- 172.21.15.62:0/847380204 >> 172.21.15.62:0/847380204 conn(0x7f51ec105b20 msgr2=0x7f51ec1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:06.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.567+0000 7f51f2800700 1 -- 172.21.15.62:0/847380204 shutdown_connections 2024-04-02T14:33:06.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:05.567+0000 7f51f2800700 1 -- 172.21.15.62:0/847380204 wait complete. 2024-04-02T14:33:07.822 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:07 smithi062 ceph-mon[32116]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:07.823 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:07 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/468303132' entity='client.admin' 2024-04-02T14:33:08.184 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.169+0000 7f0c5a400700 1 Processor -- start 2024-04-02T14:33:08.184 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.169+0000 7f0c5a400700 1 -- start start 2024-04-02T14:33:08.184 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.170+0000 7f0c5a400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5410a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:08.184 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.170+0000 7f0c5a400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5410af30 con 0x7f0c5410a540 2024-04-02T14:33:08.184 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.170+0000 7f0c53400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5410a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:08.184 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.170+0000 7f0c53400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5410a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44328/0 (socket says 172.21.15.62:44328) 2024-04-02T14:33:08.184 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.170+0000 7f0c53400700 1 -- 172.21.15.62:0/370891189 learned_addr learned my addr 172.21.15.62:0/370891189 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:08.185 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.171+0000 7f0c53400700 1 -- 172.21.15.62:0/370891189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5410b6a0 con 0x7f0c5410a540 2024-04-02T14:33:08.185 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.171+0000 7f0c53400700 1 --2- 172.21.15.62:0/370891189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5410a960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0c440098a0 tx=0x7f0c44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1c50d53d9020e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:08.185 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.171+0000 7f0c52000700 1 -- 172.21.15.62:0/370891189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c44004160 con 0x7f0c5410a540 2024-04-02T14:33:08.185 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.171+0000 7f0c52000700 1 -- 172.21.15.62:0/370891189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0c44004300 con 0x7f0c5410a540 2024-04-02T14:33:08.185 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.172+0000 7f0c52000700 1 -- 172.21.15.62:0/370891189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c44011bb0 con 0x7f0c5410a540 2024-04-02T14:33:08.185 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.172+0000 7f0c5a400700 1 -- 172.21.15.62:0/370891189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 msgr2=0x7f0c5410a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:08.185 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.172+0000 7f0c5a400700 1 --2- 172.21.15.62:0/370891189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5410a960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0c440098a0 tx=0x7f0c44009bb0 comp rx=0 tx=0).stop 2024-04-02T14:33:08.186 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.172+0000 7f0c5a400700 1 -- 172.21.15.62:0/370891189 shutdown_connections 2024-04-02T14:33:08.186 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.172+0000 7f0c5a400700 1 --2- 172.21.15.62:0/370891189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5410a960 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:08.186 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.172+0000 7f0c5a400700 1 -- 172.21.15.62:0/370891189 >> 172.21.15.62:0/370891189 conn(0x7f0c54105b20 msgr2=0x7f0c54107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:08.187 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.172+0000 7f0c5a400700 1 -- 172.21.15.62:0/370891189 shutdown_connections 2024-04-02T14:33:08.187 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.172+0000 7f0c5a400700 1 -- 172.21.15.62:0/370891189 wait complete. 2024-04-02T14:33:08.187 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.174+0000 7f0c5a400700 1 Processor -- start 2024-04-02T14:33:08.187 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.174+0000 7f0c5a400700 1 -- start start 2024-04-02T14:33:08.187 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.174+0000 7f0c5a400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:08.187 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.174+0000 7f0c5a400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5410ea50 con 0x7f0c5410a540 2024-04-02T14:33:08.187 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.174+0000 7f0c53400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:08.187 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.174+0000 7f0c53400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5419e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44342/0 (socket says 172.21.15.62:44342) 2024-04-02T14:33:08.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.174+0000 7f0c53400700 1 -- 172.21.15.62:0/468303132 learned_addr learned my addr 172.21.15.62:0/468303132 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:08.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.175+0000 7f0c53400700 1 -- 172.21.15.62:0/468303132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5419ea20 con 0x7f0c5410a540 2024-04-02T14:33:08.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.175+0000 7f0c53400700 1 --2- 172.21.15.62:0/468303132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5419e4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c44000c00 tx=0x7f0c44004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:08.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.175+0000 7f0c4be00700 1 -- 172.21.15.62:0/468303132 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c44011e60 con 0x7f0c5410a540 2024-04-02T14:33:08.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.175+0000 7f0c4be00700 1 -- 172.21.15.62:0/468303132 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0c44003af0 con 0x7f0c5410a540 2024-04-02T14:33:08.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.175+0000 7f0c4be00700 1 -- 172.21.15.62:0/468303132 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c440246a0 con 0x7f0c5410a540 2024-04-02T14:33:08.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.176+0000 7f0c5a400700 1 -- 172.21.15.62:0/468303132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5419ecb0 con 0x7f0c5410a540 2024-04-02T14:33:08.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.176+0000 7f0c5a400700 1 -- 172.21.15.62:0/468303132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c5419f110 con 0x7f0c5410a540 2024-04-02T14:33:08.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.177+0000 7f0c5a400700 1 -- 172.21.15.62:0/468303132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c54063c20 con 0x7f0c5410a540 2024-04-02T14:33:08.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.177+0000 7f0c4be00700 1 -- 172.21.15.62:0/468303132 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0c44022020 con 0x7f0c5410a540 2024-04-02T14:33:08.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.177+0000 7f0c4be00700 1 --2- 172.21.15.62:0/468303132 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f0c38039600 0x7f0c3803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:08.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.178+0000 7f0c4be00700 1 -- 172.21.15.62:0/468303132 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0c4404d820 con 0x7f0c5410a540 2024-04-02T14:33:08.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.178+0000 7f0c52a00700 1 --2- 172.21.15.62:0/468303132 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f0c38039600 0x7f0c3803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:08.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.179+0000 7f0c52a00700 1 --2- 172.21.15.62:0/468303132 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f0c38039600 0x7f0c3803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0c40006fd0 tx=0x7f0c40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:08.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.183+0000 7f0c4be00700 1 -- 172.21.15.62:0/468303132 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0c440212c0 con 0x7f0c5410a540 2024-04-02T14:33:08.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.417+0000 7f0c5a400700 1 -- 172.21.15.62:0/468303132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f0c541a1be0 con 0x7f0c5410a540 2024-04-02T14:33:08.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.421+0000 7f0c4be00700 1 -- 172.21.15.62:0/468303132 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f0c4404d170 con 0x7f0c5410a540 2024-04-02T14:33:08.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.424+0000 7f0c5a400700 1 -- 172.21.15.62:0/468303132 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f0c38039600 msgr2=0x7f0c3803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:08.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.425+0000 7f0c5a400700 1 --2- 172.21.15.62:0/468303132 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f0c38039600 0x7f0c3803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0c40006fd0 tx=0x7f0c40006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:08.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.425+0000 7f0c5a400700 1 -- 172.21.15.62:0/468303132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 msgr2=0x7f0c5419e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:08.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.425+0000 7f0c5a400700 1 --2- 172.21.15.62:0/468303132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5419e4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c44000c00 tx=0x7f0c44004620 comp rx=0 tx=0).stop 2024-04-02T14:33:08.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.425+0000 7f0c5a400700 1 -- 172.21.15.62:0/468303132 shutdown_connections 2024-04-02T14:33:08.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.425+0000 7f0c5a400700 1 --2- 172.21.15.62:0/468303132 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f0c38039600 0x7f0c3803bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:08.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.425+0000 7f0c5a400700 1 --2- 172.21.15.62:0/468303132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c5410a540 0x7f0c5419e4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:08.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.425+0000 7f0c5a400700 1 -- 172.21.15.62:0/468303132 >> 172.21.15.62:0/468303132 conn(0x7f0c54105b20 msgr2=0x7f0c54106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:08.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.425+0000 7f0c5a400700 1 -- 172.21.15.62:0/468303132 shutdown_connections 2024-04-02T14:33:08.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:07.425+0000 7f0c5a400700 1 -- 172.21.15.62:0/468303132 wait complete. 2024-04-02T14:33:08.191 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling the dashboard module... 2024-04-02T14:33:09.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:08 smithi062 ceph-mon[32116]: from='mgr.14120 172.21.15.62:0/173339544' entity='mgr.x' 2024-04-02T14:33:10.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:09 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2090985707' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-02T14:33:10.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:09 smithi062 ceph-mon[32116]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:10.345 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:10 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: ignoring --setuser ceph since I am not root 2024-04-02T14:33:10.345 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:10 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: ignoring --setgroup ceph since I am not root 2024-04-02T14:33:10.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.769+0000 7fe045200700 1 Processor -- start 2024-04-02T14:33:10.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.769+0000 7fe045200700 1 -- start start 2024-04-02T14:33:10.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.769+0000 7fe045200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04010b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:10.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.769+0000 7fe045200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe040063910 con 0x7fe040108dd0 2024-04-02T14:33:10.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.770+0000 7fe03de00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04010b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.770+0000 7fe03de00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04010b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44350/0 (socket says 172.21.15.62:44350) 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.770+0000 7fe03de00700 1 -- 172.21.15.62:0/1921398611 learned_addr learned my addr 172.21.15.62:0/1921398611 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.770+0000 7fe03de00700 1 -- 172.21.15.62:0/1921398611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04010b700 con 0x7fe040108dd0 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.771+0000 7fe03de00700 1 --2- 172.21.15.62:0/1921398611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04010b1c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe030009e80 tx=0x7fe03000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5087508d85570a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.771+0000 7fe03ca00700 1 -- 172.21.15.62:0/1921398611 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe030017070 con 0x7fe040108dd0 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.771+0000 7fe03ca00700 1 -- 172.21.15.62:0/1921398611 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe03000b8c0 con 0x7fe040108dd0 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.771+0000 7fe03ca00700 1 -- 172.21.15.62:0/1921398611 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe030004580 con 0x7fe040108dd0 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.772+0000 7fe045200700 1 -- 172.21.15.62:0/1921398611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 msgr2=0x7fe04010b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.772+0000 7fe045200700 1 --2- 172.21.15.62:0/1921398611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04010b1c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe030009e80 tx=0x7fe03000b330 comp rx=0 tx=0).stop 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.772+0000 7fe045200700 1 -- 172.21.15.62:0/1921398611 shutdown_connections 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.772+0000 7fe045200700 1 --2- 172.21.15.62:0/1921398611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04010b1c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.772+0000 7fe045200700 1 -- 172.21.15.62:0/1921398611 >> 172.21.15.62:0/1921398611 conn(0x7fe040102840 msgr2=0x7fe040104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.772+0000 7fe045200700 1 -- 172.21.15.62:0/1921398611 shutdown_connections 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.772+0000 7fe045200700 1 -- 172.21.15.62:0/1921398611 wait complete. 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.773+0000 7fe045200700 1 Processor -- start 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.773+0000 7fe045200700 1 -- start start 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.774+0000 7fe045200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.774+0000 7fe045200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe04010c7d0 con 0x7fe040108dd0 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.774+0000 7fe03de00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.774+0000 7fe03de00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04019e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44356/0 (socket says 172.21.15.62:44356) 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.774+0000 7fe03de00700 1 -- 172.21.15.62:0/2090985707 learned_addr learned my addr 172.21.15.62:0/2090985707 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.774+0000 7fe03de00700 1 -- 172.21.15.62:0/2090985707 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04019ea60 con 0x7fe040108dd0 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.775+0000 7fe03de00700 1 --2- 172.21.15.62:0/2090985707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04019e520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe030009fd0 tx=0x7fe030004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.775+0000 7fe036a00700 1 -- 172.21.15.62:0/2090985707 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe030021020 con 0x7fe040108dd0 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.775+0000 7fe036a00700 1 -- 172.21.15.62:0/2090985707 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe030003d00 con 0x7fe040108dd0 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.775+0000 7fe036a00700 1 -- 172.21.15.62:0/2090985707 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe030017040 con 0x7fe040108dd0 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.775+0000 7fe045200700 1 -- 172.21.15.62:0/2090985707 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04019ecf0 con 0x7fe040108dd0 2024-04-02T14:33:10.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.775+0000 7fe045200700 1 -- 172.21.15.62:0/2090985707 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0401a19e0 con 0x7fe040108dd0 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.776+0000 7fe036a00700 1 -- 172.21.15.62:0/2090985707 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe030024020 con 0x7fe040108dd0 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.776+0000 7fe045200700 1 -- 172.21.15.62:0/2090985707 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe040063c20 con 0x7fe040108dd0 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.777+0000 7fe036a00700 1 --2- 172.21.15.62:0/2090985707 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fe0240395b0 0x7fe02403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.777+0000 7fe036a00700 1 -- 172.21.15.62:0/2090985707 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe03000f050 con 0x7fe040108dd0 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.777+0000 7fe03d400700 1 --2- 172.21.15.62:0/2090985707 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fe0240395b0 0x7fe02403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.778+0000 7fe03d400700 1 --2- 172.21.15.62:0/2090985707 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fe0240395b0 0x7fe02403ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe02c006fd0 tx=0x7fe02c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:08.783+0000 7fe036a00700 1 -- 172.21.15.62:0/2090985707 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe03002b020 con 0x7fe040108dd0 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.085+0000 7fe045200700 1 -- 172.21.15.62:0/2090985707 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fe0401a1fb0 con 0x7fe040108dd0 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.965+0000 7fe036a00700 1 -- 172.21.15.62:0/2090985707 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fe03002e350 con 0x7fe040108dd0 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.969+0000 7fe036a00700 1 -- 172.21.15.62:0/2090985707 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fe03001d470 con 0x7fe040108dd0 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.970+0000 7fe045200700 1 -- 172.21.15.62:0/2090985707 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fe0240395b0 msgr2=0x7fe02403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.970+0000 7fe045200700 1 --2- 172.21.15.62:0/2090985707 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fe0240395b0 0x7fe02403ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe02c006fd0 tx=0x7fe02c006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.970+0000 7fe045200700 1 -- 172.21.15.62:0/2090985707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 msgr2=0x7fe04019e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:10.817 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.970+0000 7fe045200700 1 --2- 172.21.15.62:0/2090985707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04019e520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe030009fd0 tx=0x7fe030004980 comp rx=0 tx=0).stop 2024-04-02T14:33:10.817 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.971+0000 7fe045200700 1 -- 172.21.15.62:0/2090985707 shutdown_connections 2024-04-02T14:33:10.817 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.971+0000 7fe045200700 1 --2- 172.21.15.62:0/2090985707 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7fe0240395b0 0x7fe02403ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:10.817 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.971+0000 7fe045200700 1 --2- 172.21.15.62:0/2090985707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe040108dd0 0x7fe04019e520 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:10.817 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.971+0000 7fe045200700 1 -- 172.21.15.62:0/2090985707 >> 172.21.15.62:0/2090985707 conn(0x7fe040102840 msgr2=0x7fe040103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:10.817 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.972+0000 7fe045200700 1 -- 172.21.15.62:0/2090985707 shutdown_connections 2024-04-02T14:33:10.817 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:09.972+0000 7fe045200700 1 -- 172.21.15.62:0/2090985707 wait complete. 2024-04-02T14:33:10.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:10 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:10.839+0000 7fc5e800d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T14:33:11.236 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:10 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:10.964+0000 7fc5e800d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T14:33:11.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:10 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2090985707' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-02T14:33:11.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:10 smithi062 ceph-mon[32116]: mgrmap e9: x(active, since 26s) 2024-04-02T14:33:11.930 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:11 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:11.802+0000 7fc5e800d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T14:33:12.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:11 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3510230261' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T14:33:12.242 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:11 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:11.928+0000 7fc5e800d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T14:33:12.242 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:12 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:12.111+0000 7fc5e800d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T14:33:12.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-04-02T14:33:12.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-02T14:33:12.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T14:33:12.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-02T14:33:12.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T14:33:12.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-04-02T14:33:12.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.391+0000 7f5169a00700 1 Processor -- start 2024-04-02T14:33:12.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.391+0000 7f5169a00700 1 -- start start 2024-04-02T14:33:12.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.391+0000 7f5169a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516407b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:12.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.391+0000 7f5169a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f516407b770 con 0x7f516407cd40 2024-04-02T14:33:12.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.392+0000 7f5162a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516407b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:12.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.392+0000 7f5162a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516407b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44382/0 (socket says 172.21.15.62:44382) 2024-04-02T14:33:12.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.392+0000 7f5162a00700 1 -- 172.21.15.62:0/1167140432 learned_addr learned my addr 172.21.15.62:0/1167140432 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:12.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.392+0000 7f5162a00700 1 -- 172.21.15.62:0/1167140432 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516407b8f0 con 0x7f516407cd40 2024-04-02T14:33:12.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.393+0000 7f5162a00700 1 --2- 172.21.15.62:0/1167140432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516407b1a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51540098a0 tx=0x7f5154009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c2d34914702471 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:12.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.393+0000 7f5161600700 1 -- 172.21.15.62:0/1167140432 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5154004160 con 0x7f516407cd40 2024-04-02T14:33:12.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.393+0000 7f5161600700 1 -- 172.21.15.62:0/1167140432 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5154004300 con 0x7f516407cd40 2024-04-02T14:33:12.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.393+0000 7f5161600700 1 -- 172.21.15.62:0/1167140432 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5154011bb0 con 0x7f516407cd40 2024-04-02T14:33:12.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.394+0000 7f5169a00700 1 -- 172.21.15.62:0/1167140432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 msgr2=0x7f516407b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:12.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.394+0000 7f5169a00700 1 --2- 172.21.15.62:0/1167140432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516407b1a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51540098a0 tx=0x7f5154009bb0 comp rx=0 tx=0).stop 2024-04-02T14:33:12.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.394+0000 7f5169a00700 1 -- 172.21.15.62:0/1167140432 shutdown_connections 2024-04-02T14:33:12.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.394+0000 7f5169a00700 1 --2- 172.21.15.62:0/1167140432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516407b1a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:12.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.394+0000 7f5169a00700 1 -- 172.21.15.62:0/1167140432 >> 172.21.15.62:0/1167140432 conn(0x7f5164103890 msgr2=0x7f5164105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:12.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.394+0000 7f5169a00700 1 -- 172.21.15.62:0/1167140432 shutdown_connections 2024-04-02T14:33:12.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.394+0000 7f5169a00700 1 -- 172.21.15.62:0/1167140432 wait complete. 2024-04-02T14:33:12.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.395+0000 7f5169a00700 1 Processor -- start 2024-04-02T14:33:12.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.395+0000 7f5169a00700 1 -- start start 2024-04-02T14:33:12.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.396+0000 7f5169a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.396+0000 7f5169a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f516410e950 con 0x7f516407cd40 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.396+0000 7f5162a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.396+0000 7f5162a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44388/0 (socket says 172.21.15.62:44388) 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.396+0000 7f5162a00700 1 -- 172.21.15.62:0/3510230261 learned_addr learned my addr 172.21.15.62:0/3510230261 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.396+0000 7f5162a00700 1 -- 172.21.15.62:0/3510230261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516419e990 con 0x7f516407cd40 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.397+0000 7f5162a00700 1 --2- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516419e450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f5154011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.397+0000 7f515b400700 1 -- 172.21.15.62:0/3510230261 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5154011e60 con 0x7f516407cd40 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.397+0000 7f515b400700 1 -- 172.21.15.62:0/3510230261 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5154003b20 con 0x7f516407cd40 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.397+0000 7f5169a00700 1 -- 172.21.15.62:0/3510230261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516419ec20 con 0x7f516407cd40 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.397+0000 7f515b400700 1 -- 172.21.15.62:0/3510230261 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51540247a0 con 0x7f516407cd40 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.397+0000 7f5169a00700 1 -- 172.21.15.62:0/3510230261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516419f080 con 0x7f516407cd40 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.398+0000 7f515b400700 1 -- 172.21.15.62:0/3510230261 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f5154022020 con 0x7f516407cd40 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.398+0000 7f5169a00700 1 -- 172.21.15.62:0/3510230261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5164063c20 con 0x7f516407cd40 2024-04-02T14:33:12.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.399+0000 7f515b400700 1 --2- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f5148039600 0x7f514803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.399+0000 7f515b400700 1 -- 172.21.15.62:0/3510230261 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f515404d790 con 0x7f516407cd40 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.399+0000 7f5162000700 1 -- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f5148039600 msgr2=0x7f514803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1356430694 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.399+0000 7f5162000700 1 --2- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f5148039600 0x7f514803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.409+0000 7f515b400700 1 -- 172.21.15.62:0/3510230261 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f515400b3e0 con 0x7f516407cd40 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.600+0000 7f5162000700 1 -- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f5148039600 msgr2=0x7f514803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1356430694 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.600+0000 7f5162000700 1 --2- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f5148039600 0x7f514803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.703+0000 7f5169a00700 1 -- 172.21.15.62:0/3510230261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f51641a1a30 con 0x7f516407cd40 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.704+0000 7f515b400700 1 -- 172.21.15.62:0/3510230261 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f515401aaa0 con 0x7f516407cd40 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.707+0000 7f5169a00700 1 -- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f5148039600 msgr2=0x7f514803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.707+0000 7f5169a00700 1 --2- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f5148039600 0x7f514803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.707+0000 7f5169a00700 1 -- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 msgr2=0x7f516419e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.707+0000 7f5169a00700 1 --2- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516419e450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f5154011ca0 comp rx=0 tx=0).stop 2024-04-02T14:33:12.424 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.707+0000 7f5169a00700 1 -- 172.21.15.62:0/3510230261 shutdown_connections 2024-04-02T14:33:12.425 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.707+0000 7f5169a00700 1 --2- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f5148039600 0x7f514803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:12.425 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.707+0000 7f5169a00700 1 --2- 172.21.15.62:0/3510230261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f516407cd40 0x7f516419e450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:12.425 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.707+0000 7f5169a00700 1 -- 172.21.15.62:0/3510230261 >> 172.21.15.62:0/3510230261 conn(0x7f5164103890 msgr2=0x7f51641043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:12.425 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.707+0000 7f5169a00700 1 -- 172.21.15.62:0/3510230261 shutdown_connections 2024-04-02T14:33:12.425 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:11.707+0000 7f5169a00700 1 -- 172.21.15.62:0/3510230261 wait complete. 2024-04-02T14:33:12.425 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2024-04-02T14:33:12.425 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 9... 2024-04-02T14:33:12.572 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:12 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:12.240+0000 7fc5e800d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T14:33:12.572 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:12 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:12.356+0000 7fc5e800d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T14:33:12.844 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:12 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:12.705+0000 7fc5e800d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T14:33:13.266 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:12 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:12.852+0000 7fc5e800d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T14:33:13.267 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:12 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:12.970+0000 7fc5e800d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T14:33:13.267 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:13 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:13.102+0000 7fc5e800d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T14:33:13.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:13 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:13.327+0000 7fc5e800d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T14:33:14.773 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:14 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:14.670+0000 7fc5e800d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T14:33:15.094 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:14 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:14.855+0000 7fc5e800d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T14:33:15.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:15 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:15.292+0000 7fc5e800d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T14:33:15.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:15 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:15.416+0000 7fc5e800d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T14:33:15.944 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:15 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:15.689+0000 7fc5e800d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:33:15.944 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:15 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:15.942+0000 7fc5e800d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T14:33:16.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:16 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:16.335+0000 7fc5e800d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:33:16.981 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:16 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:16.661+0000 7fc5e800d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T14:33:17.344 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:16 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:16.979+0000 7fc5e800d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T14:33:18.344 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:18 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:18.054+0000 7fc5e800d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T14:33:19.094 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:18 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:18.777+0000 7fc5e800d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T14:33:19.094 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:18 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:18.913+0000 7fc5e800d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T14:33:19.594 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:33:19 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:33:19.207+0000 7fc5e800d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T14:33:19.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:19 smithi062 ceph-mon[32116]: Active manager daemon x restarted 2024-04-02T14:33:19.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:19 smithi062 ceph-mon[32116]: Activating manager daemon x 2024-04-02T14:33:19.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:19 smithi062 ceph-mon[32116]: osdmap e3: 0 total, 0 up, 0 in 2024-04-02T14:33:19.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:19 smithi062 ceph-mon[32116]: mgrmap e10: x(active, starting, since 0.00794512s) 2024-04-02T14:33:19.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:33:19.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T14:33:19.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T14:33:19.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T14:33:19.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T14:33:19.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:19 smithi062 ceph-mon[32116]: Manager daemon x is now available 2024-04-02T14:33:20.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:20 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:33:20.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:20 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T14:33:20.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:20 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T14:33:20.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:20 smithi062 ceph-mon[32116]: [02/Apr/2024:14:33:19] ENGINE Bus STARTING 2024-04-02T14:33:20.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:20 smithi062 ceph-mon[32116]: [02/Apr/2024:14:33:19] ENGINE Serving on https://172.21.15.62:7150 2024-04-02T14:33:20.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:20 smithi062 ceph-mon[32116]: [02/Apr/2024:14:33:20] ENGINE Serving on http://172.21.15.62:8765 2024-04-02T14:33:20.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:20 smithi062 ceph-mon[32116]: [02/Apr/2024:14:33:20] ENGINE Bus STARTED 2024-04-02T14:33:20.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:20 smithi062 ceph-mon[32116]: mgrmap e11: x(active, since 1.00908s) 2024-04-02T14:33:20.977 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-04-02T14:33:20.978 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-02T14:33:20.979 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T14:33:20.979 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-04-02T14:33:20.979 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.055+0000 7f5839600700 1 Processor -- start 2024-04-02T14:33:20.979 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.055+0000 7f5839600700 1 -- start start 2024-04-02T14:33:20.979 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.056+0000 7f5839600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f5834108730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:20.980 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.056+0000 7f5839600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5834108d00 con 0x7f5834108310 2024-04-02T14:33:20.980 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.056+0000 7f5832a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f5834108730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:20.983 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.056+0000 7f5832a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f5834108730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35760/0 (socket says 172.21.15.62:35760) 2024-04-02T14:33:20.983 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.056+0000 7f5832a00700 1 -- 172.21.15.62:0/3144910261 learned_addr learned my addr 172.21.15.62:0/3144910261 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:20.983 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.057+0000 7f5832a00700 1 -- 172.21.15.62:0/3144910261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5834109510 con 0x7f5834108310 2024-04-02T14:33:20.983 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.057+0000 7f5832a00700 1 --2- 172.21.15.62:0/3144910261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f5834108730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f58240098a0 tx=0x7f5824009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6c20d87811ab6e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:20.983 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.057+0000 7f5831600700 1 -- 172.21.15.62:0/3144910261 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5824004160 con 0x7f5834108310 2024-04-02T14:33:20.983 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.058+0000 7f5831600700 1 -- 172.21.15.62:0/3144910261 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5824004300 con 0x7f5834108310 2024-04-02T14:33:20.983 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.058+0000 7f5831600700 1 -- 172.21.15.62:0/3144910261 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5824011bb0 con 0x7f5834108310 2024-04-02T14:33:20.983 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.058+0000 7f5839600700 1 -- 172.21.15.62:0/3144910261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 msgr2=0x7f5834108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:20.984 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.058+0000 7f5839600700 1 --2- 172.21.15.62:0/3144910261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f5834108730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f58240098a0 tx=0x7f5824009bb0 comp rx=0 tx=0).stop 2024-04-02T14:33:20.984 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.058+0000 7f5839600700 1 -- 172.21.15.62:0/3144910261 shutdown_connections 2024-04-02T14:33:20.984 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.058+0000 7f5839600700 1 --2- 172.21.15.62:0/3144910261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f5834108730 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:20.984 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.058+0000 7f5839600700 1 -- 172.21.15.62:0/3144910261 >> 172.21.15.62:0/3144910261 conn(0x7f58341038f0 msgr2=0x7f5834105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:20.984 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.059+0000 7f5839600700 1 -- 172.21.15.62:0/3144910261 shutdown_connections 2024-04-02T14:33:20.984 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.059+0000 7f5839600700 1 -- 172.21.15.62:0/3144910261 wait complete. 2024-04-02T14:33:20.984 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.060+0000 7f5839600700 1 Processor -- start 2024-04-02T14:33:20.984 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.060+0000 7f5839600700 1 -- start start 2024-04-02T14:33:20.984 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.060+0000 7f5839600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f583419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:20.984 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.060+0000 7f5839600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f583410c8c0 con 0x7f5834108310 2024-04-02T14:33:20.984 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.061+0000 7f5832a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f583419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:20.985 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.061+0000 7f5832a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f583419e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35772/0 (socket says 172.21.15.62:35772) 2024-04-02T14:33:20.985 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.061+0000 7f5832a00700 1 -- 172.21.15.62:0/3132956445 learned_addr learned my addr 172.21.15.62:0/3132956445 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:20.985 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.061+0000 7f5832a00700 1 -- 172.21.15.62:0/3132956445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583419eaa0 con 0x7f5834108310 2024-04-02T14:33:20.985 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.061+0000 7f5832a00700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f583419e560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5824011bf0 tx=0x7f5824011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:20.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.062+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5824003930 con 0x7f5834108310 2024-04-02T14:33:20.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.062+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5824003ad0 con 0x7f5834108310 2024-04-02T14:33:20.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.062+0000 7f5839600700 1 -- 172.21.15.62:0/3132956445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583419ed30 con 0x7f5834108310 2024-04-02T14:33:20.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.062+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5824019760 con 0x7f5834108310 2024-04-02T14:33:20.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.062+0000 7f5839600700 1 -- 172.21.15.62:0/3132956445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f583419f190 con 0x7f5834108310 2024-04-02T14:33:20.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.063+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f582401ab50 con 0x7f5834108310 2024-04-02T14:33:20.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.063+0000 7f582b400700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 0x7f581803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:20.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.063+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 --> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f581803c180 con 0x7f58180395b0 2024-04-02T14:33:20.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.063+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f582404dce0 con 0x7f5834108310 2024-04-02T14:33:20.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.064+0000 7f5832000700 1 -- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 msgr2=0x7f581803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1356430694 2024-04-02T14:33:20.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.064+0000 7f5832000700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 0x7f581803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T14:33:20.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.265+0000 7f5832000700 1 -- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 msgr2=0x7f581803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1356430694 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.265+0000 7f5832000700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 0x7f581803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.666+0000 7f5832000700 1 -- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 msgr2=0x7f581803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1356430694 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:13.666+0000 7f5832000700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 0x7f581803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:14.467+0000 7f5832000700 1 -- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 msgr2=0x7f581803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1356430694 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:14.467+0000 7f5832000700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 0x7f581803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:16.069+0000 7f5832000700 1 -- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 msgr2=0x7f581803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1356430694 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:16.069+0000 7f5832000700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 0x7f581803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:19.216+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f582401a430 con 0x7f5834108310 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:19.216+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 msgr2=0x7f581803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:19.216+0000 7f582b400700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 0x7f581803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.217+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f582404f9f0 con 0x7f5834108310 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.218+0000 7f582b400700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f581803cfe0 0x7f581803f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:20.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.218+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5824008ea0 con 0x7f581803cfe0 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.219+0000 7f5832000700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f581803cfe0 0x7f581803f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.220+0000 7f5832000700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f581803cfe0 0x7f581803f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5820003a10 tx=0x7f58200092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.221+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f5824008ea0 con 0x7f581803cfe0 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.229+0000 7f5839600700 1 -- 172.21.15.62:0/3132956445 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f583410ca40 con 0x7f581803cfe0 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.229+0000 7f582b400700 1 -- 172.21.15.62:0/3132956445 <== mgr.14152 v2:172.21.15.62:6800/1995176952 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f583410ca40 con 0x7f581803cfe0 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.230+0000 7f5839600700 1 -- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f581803cfe0 msgr2=0x7f581803f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.230+0000 7f5839600700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f581803cfe0 0x7f581803f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5820003a10 tx=0x7f58200092b0 comp rx=0 tx=0).stop 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.230+0000 7f5839600700 1 -- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 msgr2=0x7f583419e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.230+0000 7f5839600700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f583419e560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5824011bf0 tx=0x7f5824011cd0 comp rx=0 tx=0).stop 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.230+0000 7f5839600700 1 -- 172.21.15.62:0/3132956445 shutdown_connections 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.230+0000 7f5839600700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5834108310 0x7f583419e560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.230+0000 7f5839600700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f581803cfe0 0x7f581803f3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.230+0000 7f5839600700 1 --2- 172.21.15.62:0/3132956445 >> [v2:172.21.15.62:6800/1356430694,v1:172.21.15.62:6801/1356430694] conn(0x7f58180395b0 0x7f581803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:20.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.230+0000 7f5839600700 1 -- 172.21.15.62:0/3132956445 >> 172.21.15.62:0/3132956445 conn(0x7f58341038f0 msgr2=0x7f5834104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:20.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.230+0000 7f5839600700 1 -- 172.21.15.62:0/3132956445 shutdown_connections 2024-04-02T14:33:20.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:20.230+0000 7f5839600700 1 -- 172.21.15.62:0/3132956445 wait complete. 2024-04-02T14:33:20.993 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 9 is available 2024-04-02T14:33:20.993 INFO:teuthology.orchestra.run.smithi062.stdout:Generating a dashboard self-signed certificate... 2024-04-02T14:33:21.572 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:21 smithi062 ceph-mon[32116]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T14:33:21.572 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:21 smithi062 ceph-mon[32116]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T14:33:22.567 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:22 smithi062 ceph-mon[32116]: mgrmap e12: x(active, since 2s) 2024-04-02T14:33:22.567 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:22 smithi062 ceph-mon[32116]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:33:22.567 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:22.567 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:22.915 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-02T14:33:22.916 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.735+0000 7fdb3e400700 1 Processor -- start 2024-04-02T14:33:22.916 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.735+0000 7fdb3e400700 1 -- start start 2024-04-02T14:33:22.916 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.736+0000 7fdb3e400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3807c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:22.916 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.736+0000 7fdb3e400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3807c870 con 0x7fdb3807de30 2024-04-02T14:33:22.916 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.736+0000 7fdb37400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:22.916 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.736+0000 7fdb37400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3807c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35872/0 (socket says 172.21.15.62:35872) 2024-04-02T14:33:22.916 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.736+0000 7fdb37400700 1 -- 172.21.15.62:0/3935230927 learned_addr learned my addr 172.21.15.62:0/3935230927 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:22.916 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.737+0000 7fdb37400700 1 -- 172.21.15.62:0/3935230927 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3807c9f0 con 0x7fdb3807de30 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.737+0000 7fdb37400700 1 --2- 172.21.15.62:0/3935230927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3807c330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb280098a0 tx=0x7fdb28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1edb27ff110913c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.737+0000 7fdb36a00700 1 -- 172.21.15.62:0/3935230927 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb28004160 con 0x7fdb3807de30 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.737+0000 7fdb36a00700 1 -- 172.21.15.62:0/3935230927 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdb28004300 con 0x7fdb3807de30 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.737+0000 7fdb36a00700 1 -- 172.21.15.62:0/3935230927 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb28011bb0 con 0x7fdb3807de30 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.738+0000 7fdb3e400700 1 -- 172.21.15.62:0/3935230927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 msgr2=0x7fdb3807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.738+0000 7fdb3e400700 1 --2- 172.21.15.62:0/3935230927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3807c330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb280098a0 tx=0x7fdb28009bb0 comp rx=0 tx=0).stop 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.738+0000 7fdb3e400700 1 -- 172.21.15.62:0/3935230927 shutdown_connections 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.738+0000 7fdb3e400700 1 --2- 172.21.15.62:0/3935230927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3807c330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.738+0000 7fdb3e400700 1 -- 172.21.15.62:0/3935230927 >> 172.21.15.62:0/3935230927 conn(0x7fdb38103980 msgr2=0x7fdb38105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.738+0000 7fdb3e400700 1 -- 172.21.15.62:0/3935230927 shutdown_connections 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.738+0000 7fdb3e400700 1 -- 172.21.15.62:0/3935230927 wait complete. 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.739+0000 7fdb3e400700 1 Processor -- start 2024-04-02T14:33:22.917 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.739+0000 7fdb3e400700 1 -- start start 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.740+0000 7fdb3e400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.740+0000 7fdb3e400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3810e960 con 0x7fdb3807de30 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.740+0000 7fdb37400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.740+0000 7fdb37400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3819e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35884/0 (socket says 172.21.15.62:35884) 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.740+0000 7fdb37400700 1 -- 172.21.15.62:0/2664765905 learned_addr learned my addr 172.21.15.62:0/2664765905 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.740+0000 7fdb37400700 1 -- 172.21.15.62:0/2664765905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3819ea00 con 0x7fdb3807de30 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.740+0000 7fdb37400700 1 --2- 172.21.15.62:0/2664765905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3819e4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdb28000c00 tx=0x7fdb28011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.741+0000 7fdb34c00700 1 -- 172.21.15.62:0/2664765905 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb28011e60 con 0x7fdb3807de30 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.741+0000 7fdb34c00700 1 -- 172.21.15.62:0/2664765905 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdb28020070 con 0x7fdb3807de30 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.741+0000 7fdb34c00700 1 -- 172.21.15.62:0/2664765905 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb28019550 con 0x7fdb3807de30 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.741+0000 7fdb3e400700 1 -- 172.21.15.62:0/2664765905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3819ec90 con 0x7fdb3807de30 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.741+0000 7fdb3e400700 1 -- 172.21.15.62:0/2664765905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3819f0f0 con 0x7fdb3807de30 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.742+0000 7fdb3e400700 1 -- 172.21.15.62:0/2664765905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb38063c20 con 0x7fdb3807de30 2024-04-02T14:33:22.918 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.742+0000 7fdb34c00700 1 -- 172.21.15.62:0/2664765905 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fdb28003aa0 con 0x7fdb3807de30 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.743+0000 7fdb34c00700 1 --2- 172.21.15.62:0/2664765905 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb1c039660 0x7fdb1c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.743+0000 7fdb34c00700 1 -- 172.21.15.62:0/2664765905 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdb2804dba0 con 0x7fdb3807de30 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.743+0000 7fdb2fe00700 1 --2- 172.21.15.62:0/2664765905 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb1c039660 0x7fdb1c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.744+0000 7fdb2fe00700 1 --2- 172.21.15.62:0/2664765905 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb1c039660 0x7fdb1c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdb20006fd0 tx=0x7fdb20006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.749+0000 7fdb34c00700 1 -- 172.21.15.62:0/2664765905 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdb2800e260 con 0x7fdb3807de30 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:21.989+0000 7fdb3e400700 1 -- 172.21.15.62:0/2664765905 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fdb38104b40 con 0x7fdb1c039660 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:22.205+0000 7fdb34c00700 1 -- 172.21.15.62:0/2664765905 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fdb38104b40 con 0x7fdb1c039660 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:22.208+0000 7fdb3e400700 1 -- 172.21.15.62:0/2664765905 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb1c039660 msgr2=0x7fdb1c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:22.208+0000 7fdb3e400700 1 --2- 172.21.15.62:0/2664765905 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb1c039660 0x7fdb1c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdb20006fd0 tx=0x7fdb20006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:22.208+0000 7fdb3e400700 1 -- 172.21.15.62:0/2664765905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 msgr2=0x7fdb3819e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:22.208+0000 7fdb3e400700 1 --2- 172.21.15.62:0/2664765905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3819e4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdb28000c00 tx=0x7fdb28011c30 comp rx=0 tx=0).stop 2024-04-02T14:33:22.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:22.208+0000 7fdb3e400700 1 -- 172.21.15.62:0/2664765905 shutdown_connections 2024-04-02T14:33:22.920 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:22.208+0000 7fdb3e400700 1 --2- 172.21.15.62:0/2664765905 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb1c039660 0x7fdb1c03bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:22.920 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:22.208+0000 7fdb3e400700 1 --2- 172.21.15.62:0/2664765905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb3807de30 0x7fdb3819e4c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:22.920 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:22.208+0000 7fdb3e400700 1 -- 172.21.15.62:0/2664765905 >> 172.21.15.62:0/2664765905 conn(0x7fdb38103980 msgr2=0x7fdb38104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:22.920 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:22.209+0000 7fdb3e400700 1 -- 172.21.15.62:0/2664765905 shutdown_connections 2024-04-02T14:33:22.920 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:22.209+0000 7fdb3e400700 1 -- 172.21.15.62:0/2664765905 wait complete. 2024-04-02T14:33:22.920 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial admin user... 2024-04-02T14:33:23.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:23 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:24.827 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$VduSBTwosM.IFlmIbe8EdOafXIqSMCmCaedW6MQ9.2HcU/vImVl2m", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712068404, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-02T14:33:24.827 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.548+0000 7fa547600700 1 Processor -- start 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.548+0000 7fa547600700 1 -- start start 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.549+0000 7fa547600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa540106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.549+0000 7fa547600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa5400638d0 con 0x7fa540106b80 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.549+0000 7fa544c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa540106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.549+0000 7fa544c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa540106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50168/0 (socket says 172.21.15.62:50168) 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.549+0000 7fa544c00700 1 -- 172.21.15.62:0/1102283322 learned_addr learned my addr 172.21.15.62:0/1102283322 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.549+0000 7fa544c00700 1 -- 172.21.15.62:0/1102283322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5401074e0 con 0x7fa540106b80 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.550+0000 7fa544c00700 1 --2- 172.21.15.62:0/1102283322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa540106fa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5340098a0 tx=0x7fa534009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27713950d70f6e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.550+0000 7fa53f400700 1 -- 172.21.15.62:0/1102283322 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa534004160 con 0x7fa540106b80 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.550+0000 7fa53f400700 1 -- 172.21.15.62:0/1102283322 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa534004300 con 0x7fa540106b80 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.550+0000 7fa53f400700 1 -- 172.21.15.62:0/1102283322 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa534011bb0 con 0x7fa540106b80 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.551+0000 7fa547600700 1 -- 172.21.15.62:0/1102283322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 msgr2=0x7fa540106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:24.828 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.551+0000 7fa547600700 1 --2- 172.21.15.62:0/1102283322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa540106fa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5340098a0 tx=0x7fa534009bb0 comp rx=0 tx=0).stop 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.551+0000 7fa547600700 1 -- 172.21.15.62:0/1102283322 shutdown_connections 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.551+0000 7fa547600700 1 --2- 172.21.15.62:0/1102283322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa540106fa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.551+0000 7fa547600700 1 -- 172.21.15.62:0/1102283322 >> 172.21.15.62:0/1102283322 conn(0x7fa540102780 msgr2=0x7fa540104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.551+0000 7fa547600700 1 -- 172.21.15.62:0/1102283322 shutdown_connections 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.551+0000 7fa547600700 1 -- 172.21.15.62:0/1102283322 wait complete. 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.552+0000 7fa547600700 1 Processor -- start 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.552+0000 7fa547600700 1 -- start start 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.553+0000 7fa547600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa54019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.553+0000 7fa547600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa540108610 con 0x7fa540106b80 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.553+0000 7fa544c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa54019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.553+0000 7fa544c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa54019e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50174/0 (socket says 172.21.15.62:50174) 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.553+0000 7fa544c00700 1 -- 172.21.15.62:0/1867395216 learned_addr learned my addr 172.21.15.62:0/1867395216 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.553+0000 7fa544c00700 1 -- 172.21.15.62:0/1867395216 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54019e9f0 con 0x7fa540106b80 2024-04-02T14:33:24.829 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.554+0000 7fa544c00700 1 --2- 172.21.15.62:0/1867395216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa54019e4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa534000c00 tx=0x7fa534011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:24.830 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.554+0000 7fa53d600700 1 -- 172.21.15.62:0/1867395216 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa534011e60 con 0x7fa540106b80 2024-04-02T14:33:24.830 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.554+0000 7fa53d600700 1 -- 172.21.15.62:0/1867395216 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa534003b20 con 0x7fa540106b80 2024-04-02T14:33:24.830 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.554+0000 7fa53d600700 1 -- 172.21.15.62:0/1867395216 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5340247a0 con 0x7fa540106b80 2024-04-02T14:33:24.830 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.554+0000 7fa547600700 1 -- 172.21.15.62:0/1867395216 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54019ec80 con 0x7fa540106b80 2024-04-02T14:33:24.830 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.554+0000 7fa547600700 1 -- 172.21.15.62:0/1867395216 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54019f0e0 con 0x7fa540106b80 2024-04-02T14:33:24.830 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.555+0000 7fa53d600700 1 -- 172.21.15.62:0/1867395216 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fa534022020 con 0x7fa540106b80 2024-04-02T14:33:24.830 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.556+0000 7fa547600700 1 -- 172.21.15.62:0/1867395216 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa540063c20 con 0x7fa540106b80 2024-04-02T14:33:24.830 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.556+0000 7fa53d600700 1 --2- 172.21.15.62:0/1867395216 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa528039660 0x7fa52803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:24.830 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.556+0000 7fa53d600700 1 -- 172.21.15.62:0/1867395216 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa53404d780 con 0x7fa540106b80 2024-04-02T14:33:24.830 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.557+0000 7fa53fe00700 1 --2- 172.21.15.62:0/1867395216 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa528039660 0x7fa52803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:24.834 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.557+0000 7fa53fe00700 1 --2- 172.21.15.62:0/1867395216 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa528039660 0x7fa52803bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa52c006fd0 tx=0x7fa52c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.562+0000 7fa53d600700 1 -- 172.21.15.62:0/1867395216 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa53400b3e0 con 0x7fa540106b80 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:23.814+0000 7fa547600700 1 -- 172.21.15.62:0/1867395216 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa540198090 con 0x7fa528039660 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:24.103+0000 7fa53d600700 1 -- 172.21.15.62:0/1867395216 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa540198090 con 0x7fa528039660 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:24.106+0000 7fa547600700 1 -- 172.21.15.62:0/1867395216 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa528039660 msgr2=0x7fa52803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:24.106+0000 7fa547600700 1 --2- 172.21.15.62:0/1867395216 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa528039660 0x7fa52803bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa52c006fd0 tx=0x7fa52c006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:24.106+0000 7fa547600700 1 -- 172.21.15.62:0/1867395216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 msgr2=0x7fa54019e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:24.106+0000 7fa547600700 1 --2- 172.21.15.62:0/1867395216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa54019e4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa534000c00 tx=0x7fa534011ca0 comp rx=0 tx=0).stop 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:24.107+0000 7fa547600700 1 -- 172.21.15.62:0/1867395216 shutdown_connections 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:24.107+0000 7fa547600700 1 --2- 172.21.15.62:0/1867395216 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa528039660 0x7fa52803bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:24.107+0000 7fa547600700 1 --2- 172.21.15.62:0/1867395216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa540106b80 0x7fa54019e4b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:24.107+0000 7fa547600700 1 -- 172.21.15.62:0/1867395216 >> 172.21.15.62:0/1867395216 conn(0x7fa540102780 msgr2=0x7fa5401951d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:24.107+0000 7fa547600700 1 -- 172.21.15.62:0/1867395216 shutdown_connections 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:24.107+0000 7fa547600700 1 -- 172.21.15.62:0/1867395216 wait complete. 2024-04-02T14:33:24.835 INFO:teuthology.orchestra.run.smithi062.stdout:Fetching dashboard port number... 2024-04-02T14:33:25.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:25 smithi062 ceph-mon[32116]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:33:25.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:25 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:25.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:25 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:26.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:26 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1402991668' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-02T14:33:26.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 8443 2024-04-02T14:33:26.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.415+0000 7fdc64000700 1 Processor -- start 2024-04-02T14:33:26.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.415+0000 7fdc64000700 1 -- start start 2024-04-02T14:33:26.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.415+0000 7fdc64000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c108740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:26.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.415+0000 7fdc64000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c108d10 con 0x7fdc5c108320 2024-04-02T14:33:26.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.416+0000 7fdc61600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c108740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:26.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.416+0000 7fdc61600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c108740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50186/0 (socket says 172.21.15.62:50186) 2024-04-02T14:33:26.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.416+0000 7fdc61600700 1 -- 172.21.15.62:0/3916108149 learned_addr learned my addr 172.21.15.62:0/3916108149 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:26.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.417+0000 7fdc61600700 1 -- 172.21.15.62:0/3916108149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c109520 con 0x7fdc5c108320 2024-04-02T14:33:26.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.417+0000 7fdc61600700 1 --2- 172.21.15.62:0/3916108149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c108740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc500098a0 tx=0x7fdc50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d31eeec3dea7db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:26.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.417+0000 7fdc5be00700 1 -- 172.21.15.62:0/3916108149 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc50004160 con 0x7fdc5c108320 2024-04-02T14:33:26.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.417+0000 7fdc5be00700 1 -- 172.21.15.62:0/3916108149 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc50004300 con 0x7fdc5c108320 2024-04-02T14:33:26.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.417+0000 7fdc5be00700 1 -- 172.21.15.62:0/3916108149 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc50011bb0 con 0x7fdc5c108320 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.418+0000 7fdc64000700 1 -- 172.21.15.62:0/3916108149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 msgr2=0x7fdc5c108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.418+0000 7fdc64000700 1 --2- 172.21.15.62:0/3916108149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c108740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc500098a0 tx=0x7fdc50009bb0 comp rx=0 tx=0).stop 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.419+0000 7fdc64000700 1 -- 172.21.15.62:0/3916108149 shutdown_connections 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.419+0000 7fdc64000700 1 --2- 172.21.15.62:0/3916108149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c108740 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.419+0000 7fdc64000700 1 -- 172.21.15.62:0/3916108149 >> 172.21.15.62:0/3916108149 conn(0x7fdc5c1038e0 msgr2=0x7fdc5c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.419+0000 7fdc64000700 1 -- 172.21.15.62:0/3916108149 shutdown_connections 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.419+0000 7fdc64000700 1 -- 172.21.15.62:0/3916108149 wait complete. 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.420+0000 7fdc64000700 1 Processor -- start 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.420+0000 7fdc64000700 1 -- start start 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.420+0000 7fdc64000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.421+0000 7fdc64000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c10c8d0 con 0x7fdc5c108320 2024-04-02T14:33:26.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.421+0000 7fdc61600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.421+0000 7fdc61600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c19c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50196/0 (socket says 172.21.15.62:50196) 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.421+0000 7fdc61600700 1 -- 172.21.15.62:0/1402991668 learned_addr learned my addr 172.21.15.62:0/1402991668 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.421+0000 7fdc61600700 1 -- 172.21.15.62:0/1402991668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c19c820 con 0x7fdc5c108320 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.422+0000 7fdc61600700 1 --2- 172.21.15.62:0/1402991668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c19c2e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc50000c00 tx=0x7fdc50004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.422+0000 7fdc5a000700 1 -- 172.21.15.62:0/1402991668 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc50011e60 con 0x7fdc5c108320 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.422+0000 7fdc5a000700 1 -- 172.21.15.62:0/1402991668 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc50003af0 con 0x7fdc5c108320 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.422+0000 7fdc64000700 1 -- 172.21.15.62:0/1402991668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5c19cab0 con 0x7fdc5c108320 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.422+0000 7fdc5a000700 1 -- 172.21.15.62:0/1402991668 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc500246a0 con 0x7fdc5c108320 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.422+0000 7fdc64000700 1 -- 172.21.15.62:0/1402991668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5c19cf10 con 0x7fdc5c108320 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.423+0000 7fdc5a000700 1 -- 172.21.15.62:0/1402991668 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fdc50022020 con 0x7fdc5c108320 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.424+0000 7fdc64000700 1 -- 172.21.15.62:0/1402991668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5c063c20 con 0x7fdc5c108320 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.424+0000 7fdc5a000700 1 --2- 172.21.15.62:0/1402991668 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdc44039660 0x7fdc4403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.424+0000 7fdc5a000700 1 -- 172.21.15.62:0/1402991668 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdc5004d950 con 0x7fdc5c108320 2024-04-02T14:33:26.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.424+0000 7fdc60c00700 1 --2- 172.21.15.62:0/1402991668 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdc44039660 0x7fdc4403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.425+0000 7fdc60c00700 1 --2- 172.21.15.62:0/1402991668 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdc44039660 0x7fdc4403bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdc48006fd0 tx=0x7fdc48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.432+0000 7fdc5a000700 1 -- 172.21.15.62:0/1402991668 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdc500212c0 con 0x7fdc5c108320 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.668+0000 7fdc64000700 1 -- 172.21.15.62:0/1402991668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fdc5c19f9a0 con 0x7fdc5c108320 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.669+0000 7fdc5a000700 1 -- 172.21.15.62:0/1402991668 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fdc5000bcb0 con 0x7fdc5c108320 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.672+0000 7fdc64000700 1 -- 172.21.15.62:0/1402991668 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdc44039660 msgr2=0x7fdc4403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.672+0000 7fdc64000700 1 --2- 172.21.15.62:0/1402991668 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdc44039660 0x7fdc4403bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdc48006fd0 tx=0x7fdc48006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.672+0000 7fdc64000700 1 -- 172.21.15.62:0/1402991668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 msgr2=0x7fdc5c19c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.672+0000 7fdc64000700 1 --2- 172.21.15.62:0/1402991668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c19c2e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc50000c00 tx=0x7fdc50004620 comp rx=0 tx=0).stop 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.673+0000 7fdc64000700 1 -- 172.21.15.62:0/1402991668 shutdown_connections 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.673+0000 7fdc64000700 1 --2- 172.21.15.62:0/1402991668 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdc44039660 0x7fdc4403bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.673+0000 7fdc64000700 1 --2- 172.21.15.62:0/1402991668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c108320 0x7fdc5c19c2e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.673+0000 7fdc64000700 1 -- 172.21.15.62:0/1402991668 >> 172.21.15.62:0/1402991668 conn(0x7fdc5c1038e0 msgr2=0x7fdc5c104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.673+0000 7fdc64000700 1 -- 172.21.15.62:0/1402991668 shutdown_connections 2024-04-02T14:33:26.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:25.673+0000 7fdc64000700 1 -- 172.21.15.62:0/1402991668 wait complete. 2024-04-02T14:33:26.801 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T14:33:26.801 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-04-02T14:33:26.829 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T14:33:26.829 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-04-02T14:33:26.829 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-04-02T14:33:26.829 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-02T14:33:26.836 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph Dashboard is now available at: 2024-04-02T14:33:26.836 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:33:26.836 INFO:teuthology.orchestra.run.smithi062.stdout: URL: https://smithi062.front.sepia.ceph.com:8443/ 2024-04-02T14:33:26.837 INFO:teuthology.orchestra.run.smithi062.stdout: User: admin 2024-04-02T14:33:26.837 INFO:teuthology.orchestra.run.smithi062.stdout: Password: pqz0bl45ac 2024-04-02T14:33:26.837 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:33:26.837 INFO:teuthology.orchestra.run.smithi062.stdout:Saving cluster configuration to /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config directory 2024-04-02T14:33:26.840 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling autotune for osd_memory_target 2024-04-02T14:33:27.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:27 smithi062 ceph-mon[32116]: mgrmap e13: x(active, since 6s) 2024-04-02T14:33:28.831 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.816+0000 7f663f800700 1 Processor -- start 2024-04-02T14:33:28.836 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.817+0000 7f663f800700 1 -- start start 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.817+0000 7f663f800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663810b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.817+0000 7f663f800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6638063910 con 0x7f6638108de0 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.818+0000 7f663ce00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663810b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.818+0000 7f663ce00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663810b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50212/0 (socket says 172.21.15.62:50212) 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.818+0000 7f663ce00700 1 -- 172.21.15.62:0/1731740175 learned_addr learned my addr 172.21.15.62:0/1731740175 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.818+0000 7f663ce00700 1 -- 172.21.15.62:0/1731740175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663810b710 con 0x7f6638108de0 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.819+0000 7f663ce00700 1 --2- 172.21.15.62:0/1731740175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663810b1d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f662c0098a0 tx=0x7f662c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d34df745fca4a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.819+0000 7f6637400700 1 -- 172.21.15.62:0/1731740175 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f662c004160 con 0x7f6638108de0 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.819+0000 7f6637400700 1 -- 172.21.15.62:0/1731740175 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f662c004300 con 0x7f6638108de0 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.819+0000 7f6637400700 1 -- 172.21.15.62:0/1731740175 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f662c011bb0 con 0x7f6638108de0 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.820+0000 7f663f800700 1 -- 172.21.15.62:0/1731740175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 msgr2=0x7f663810b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:28.837 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.820+0000 7f663f800700 1 --2- 172.21.15.62:0/1731740175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663810b1d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f662c0098a0 tx=0x7f662c009bb0 comp rx=0 tx=0).stop 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.820+0000 7f663f800700 1 -- 172.21.15.62:0/1731740175 shutdown_connections 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.820+0000 7f663f800700 1 --2- 172.21.15.62:0/1731740175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663810b1d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.820+0000 7f663f800700 1 -- 172.21.15.62:0/1731740175 >> 172.21.15.62:0/1731740175 conn(0x7f6638102810 msgr2=0x7f6638104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.820+0000 7f663f800700 1 -- 172.21.15.62:0/1731740175 shutdown_connections 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.820+0000 7f663f800700 1 -- 172.21.15.62:0/1731740175 wait complete. 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.821+0000 7f663f800700 1 Processor -- start 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.822+0000 7f663f800700 1 -- start start 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.822+0000 7f663f800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.822+0000 7f663f800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f663810c7e0 con 0x7f6638108de0 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.822+0000 7f663ce00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.822+0000 7f663ce00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663819c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50224/0 (socket says 172.21.15.62:50224) 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.822+0000 7f663ce00700 1 -- 172.21.15.62:0/2958690406 learned_addr learned my addr 172.21.15.62:0/2958690406 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.823+0000 7f663ce00700 1 -- 172.21.15.62:0/2958690406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663819c7b0 con 0x7f6638108de0 2024-04-02T14:33:28.838 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.823+0000 7f663ce00700 1 --2- 172.21.15.62:0/2958690406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663819c270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f662c000c00 tx=0x7f662c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.823+0000 7f6635600700 1 -- 172.21.15.62:0/2958690406 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f662c011e60 con 0x7f6638108de0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.823+0000 7f6635600700 1 -- 172.21.15.62:0/2958690406 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f662c003b20 con 0x7f6638108de0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.823+0000 7f6635600700 1 -- 172.21.15.62:0/2958690406 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f662c0247a0 con 0x7f6638108de0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.823+0000 7f663f800700 1 -- 172.21.15.62:0/2958690406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663819ca40 con 0x7f6638108de0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.824+0000 7f663f800700 1 -- 172.21.15.62:0/2958690406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663819cea0 con 0x7f6638108de0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.825+0000 7f663f800700 1 -- 172.21.15.62:0/2958690406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6638063c20 con 0x7f6638108de0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.825+0000 7f6635600700 1 -- 172.21.15.62:0/2958690406 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f662c022020 con 0x7f6638108de0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.825+0000 7f6635600700 1 --2- 172.21.15.62:0/2958690406 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6620039700 0x7f662003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.825+0000 7f6635600700 1 -- 172.21.15.62:0/2958690406 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f662c04d940 con 0x7f6638108de0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.826+0000 7f6637e00700 1 --2- 172.21.15.62:0/2958690406 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6620039700 0x7f662003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.826+0000 7f6637e00700 1 --2- 172.21.15.62:0/2958690406 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6620039700 0x7f662003bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6624006fd0 tx=0x7f6624006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:27.831+0000 7f6635600700 1 -- 172.21.15.62:0/2958690406 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f662c00b3e0 con 0x7f6638108de0 2024-04-02T14:33:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.061+0000 7f663f800700 1 -- 172.21.15.62:0/2958690406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f663819f8d0 con 0x7f6638108de0 2024-04-02T14:33:28.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.062+0000 7f6635600700 1 -- 172.21.15.62:0/2958690406 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f662c02ddb0 con 0x7f6638108de0 2024-04-02T14:33:28.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.065+0000 7f663f800700 1 -- 172.21.15.62:0/2958690406 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6620039700 msgr2=0x7f662003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:28.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.065+0000 7f663f800700 1 --2- 172.21.15.62:0/2958690406 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6620039700 0x7f662003bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6624006fd0 tx=0x7f6624006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:28.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.065+0000 7f663f800700 1 -- 172.21.15.62:0/2958690406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 msgr2=0x7f663819c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:28.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.065+0000 7f663f800700 1 --2- 172.21.15.62:0/2958690406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663819c270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f662c000c00 tx=0x7f662c011ca0 comp rx=0 tx=0).stop 2024-04-02T14:33:28.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.065+0000 7f663f800700 1 -- 172.21.15.62:0/2958690406 shutdown_connections 2024-04-02T14:33:28.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.065+0000 7f663f800700 1 --2- 172.21.15.62:0/2958690406 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6620039700 0x7f662003bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:28.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.065+0000 7f663f800700 1 --2- 172.21.15.62:0/2958690406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6638108de0 0x7f663819c270 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:28.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.065+0000 7f663f800700 1 -- 172.21.15.62:0/2958690406 >> 172.21.15.62:0/2958690406 conn(0x7f6638102810 msgr2=0x7f6638103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:28.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.066+0000 7f663f800700 1 -- 172.21.15.62:0/2958690406 shutdown_connections 2024-04-02T14:33:28.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:28.066+0000 7f663f800700 1 -- 172.21.15.62:0/2958690406 wait complete. 2024-04-02T14:33:30.520 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.487+0000 7fc617200700 1 Processor -- start 2024-04-02T14:33:30.520 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.487+0000 7fc617200700 1 -- start start 2024-04-02T14:33:30.521 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.488+0000 7fc617200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61807c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:30.521 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.488+0000 7fc617200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc61807c900 con 0x7fc61807de80 2024-04-02T14:33:30.521 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.488+0000 7fc615e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:30.521 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.488+0000 7fc615e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61807c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50236/0 (socket says 172.21.15.62:50236) 2024-04-02T14:33:30.521 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.488+0000 7fc615e00700 1 -- 172.21.15.62:0/1682994526 learned_addr learned my addr 172.21.15.62:0/1682994526 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:30.522 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.489+0000 7fc615e00700 1 -- 172.21.15.62:0/1682994526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61807ca80 con 0x7fc61807de80 2024-04-02T14:33:30.522 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.489+0000 7fc615e00700 1 --2- 172.21.15.62:0/1682994526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61807c330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6080098a0 tx=0x7fc608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc35efa0814baae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:30.523 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.489+0000 7fc614a00700 1 -- 172.21.15.62:0/1682994526 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc608004160 con 0x7fc61807de80 2024-04-02T14:33:30.523 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.489+0000 7fc614a00700 1 -- 172.21.15.62:0/1682994526 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc608004300 con 0x7fc61807de80 2024-04-02T14:33:30.523 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.489+0000 7fc614a00700 1 -- 172.21.15.62:0/1682994526 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc608011bb0 con 0x7fc61807de80 2024-04-02T14:33:30.523 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.490+0000 7fc617200700 1 -- 172.21.15.62:0/1682994526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 msgr2=0x7fc61807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:30.523 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.490+0000 7fc617200700 1 --2- 172.21.15.62:0/1682994526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61807c330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6080098a0 tx=0x7fc608009bb0 comp rx=0 tx=0).stop 2024-04-02T14:33:30.524 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.490+0000 7fc617200700 1 -- 172.21.15.62:0/1682994526 shutdown_connections 2024-04-02T14:33:30.524 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.490+0000 7fc617200700 1 --2- 172.21.15.62:0/1682994526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61807c330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:30.524 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.490+0000 7fc617200700 1 -- 172.21.15.62:0/1682994526 >> 172.21.15.62:0/1682994526 conn(0x7fc618103720 msgr2=0x7fc618105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:30.524 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.490+0000 7fc617200700 1 -- 172.21.15.62:0/1682994526 shutdown_connections 2024-04-02T14:33:30.525 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.490+0000 7fc617200700 1 -- 172.21.15.62:0/1682994526 wait complete. 2024-04-02T14:33:30.525 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.491+0000 7fc617200700 1 Processor -- start 2024-04-02T14:33:30.525 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.492+0000 7fc617200700 1 -- start start 2024-04-02T14:33:30.526 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.492+0000 7fc617200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:30.526 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.492+0000 7fc617200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc61810e7c0 con 0x7fc61807de80 2024-04-02T14:33:30.526 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.492+0000 7fc615e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:30.526 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.492+0000 7fc615e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61819e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50240/0 (socket says 172.21.15.62:50240) 2024-04-02T14:33:30.527 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.492+0000 7fc615e00700 1 -- 172.21.15.62:0/2429424057 learned_addr learned my addr 172.21.15.62:0/2429424057 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:30.527 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.493+0000 7fc615e00700 1 -- 172.21.15.62:0/2429424057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61819e850 con 0x7fc61807de80 2024-04-02T14:33:30.527 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.493+0000 7fc615e00700 1 --2- 172.21.15.62:0/2429424057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61819e310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc608000c00 tx=0x7fc608011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:30.527 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.493+0000 7fc60ea00700 1 -- 172.21.15.62:0/2429424057 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc608011e60 con 0x7fc61807de80 2024-04-02T14:33:30.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.493+0000 7fc60ea00700 1 -- 172.21.15.62:0/2429424057 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc608020070 con 0x7fc61807de80 2024-04-02T14:33:30.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.493+0000 7fc617200700 1 -- 172.21.15.62:0/2429424057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61819eae0 con 0x7fc61807de80 2024-04-02T14:33:30.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.493+0000 7fc60ea00700 1 -- 172.21.15.62:0/2429424057 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc608019550 con 0x7fc61807de80 2024-04-02T14:33:30.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.494+0000 7fc617200700 1 -- 172.21.15.62:0/2429424057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61819ef40 con 0x7fc61807de80 2024-04-02T14:33:30.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.495+0000 7fc60ea00700 1 -- 172.21.15.62:0/2429424057 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc608003aa0 con 0x7fc61807de80 2024-04-02T14:33:30.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.495+0000 7fc617200700 1 -- 172.21.15.62:0/2429424057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc618063c20 con 0x7fc61807de80 2024-04-02T14:33:30.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.495+0000 7fc60ea00700 1 --2- 172.21.15.62:0/2429424057 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc5fc039700 0x7fc5fc03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.495+0000 7fc60ea00700 1 -- 172.21.15.62:0/2429424057 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc60804dc80 con 0x7fc61807de80 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.495+0000 7fc615400700 1 --2- 172.21.15.62:0/2429424057 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc5fc039700 0x7fc5fc03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.496+0000 7fc615400700 1 --2- 172.21.15.62:0/2429424057 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc5fc039700 0x7fc5fc03bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc604006fd0 tx=0x7fc604006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.501+0000 7fc60ea00700 1 -- 172.21.15.62:0/2429424057 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc608018350 con 0x7fc61807de80 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.848+0000 7fc617200700 1 -- 172.21.15.62:0/2429424057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc6181a1910 con 0x7fc61807de80 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.854+0000 7fc60ea00700 1 -- 172.21.15.62:0/2429424057 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fc60800e260 con 0x7fc61807de80 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.857+0000 7fc617200700 1 -- 172.21.15.62:0/2429424057 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc5fc039700 msgr2=0x7fc5fc03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.857+0000 7fc617200700 1 --2- 172.21.15.62:0/2429424057 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc5fc039700 0x7fc5fc03bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc604006fd0 tx=0x7fc604006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.857+0000 7fc617200700 1 -- 172.21.15.62:0/2429424057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 msgr2=0x7fc61819e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.857+0000 7fc617200700 1 --2- 172.21.15.62:0/2429424057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61819e310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc608000c00 tx=0x7fc608011c30 comp rx=0 tx=0).stop 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.857+0000 7fc617200700 1 -- 172.21.15.62:0/2429424057 shutdown_connections 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.857+0000 7fc617200700 1 --2- 172.21.15.62:0/2429424057 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc5fc039700 0x7fc5fc03bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.857+0000 7fc617200700 1 --2- 172.21.15.62:0/2429424057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc61807de80 0x7fc61819e310 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:30.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.857+0000 7fc617200700 1 -- 172.21.15.62:0/2429424057 >> 172.21.15.62:0/2429424057 conn(0x7fc618103720 msgr2=0x7fc618104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:30.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.857+0000 7fc617200700 1 -- 172.21.15.62:0/2429424057 shutdown_connections 2024-04-02T14:33:30.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-04-02T14:33:29.857+0000 7fc617200700 1 -- 172.21.15.62:0/2429424057 wait complete. 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout:Or, if you are only running a single cluster on this host: 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout: ceph telemetry on 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout:For more information see: 2024-04-02T14:33:30.533 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:33:30.534 INFO:teuthology.orchestra.run.smithi062.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-02T14:33:30.534 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:33:30.534 INFO:teuthology.orchestra.run.smithi062.stdout:Bootstrap complete. 2024-04-02T14:33:30.653 INFO:tasks.cephadm:Fetching config... 2024-04-02T14:33:30.654 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:33:30.654 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-02T14:33:30.707 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-02T14:33:30.708 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:33:30.708 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-02T14:33:30.748 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-02T14:33:30.748 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:33:30.748 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/keyring of=/dev/stdout 2024-04-02T14:33:30.883 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-02T14:33:30.883 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:33:30.883 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-02T14:33:30.934 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-02T14:33:30.934 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsYp7MRg6qIY/ULuJBU7QvMByDRbucEgKtj0FSEfB60hQqwQO03ZWsVd3YgJmE4AhXzmACspll45b+tektFLha7f2Un8a13SanjnHpaewa3gg/3Q/uFfdcUtOUtmmYCjJRQcfe059yeVyPLT20GyLdd2AT9AvAbQbmTuZeRKorlPyahK3voo2rufuzWkmkx8tWkydvVpGVuvgOqjntOGs0nmb4bW+O/NkUHNIoQPGIP42cYuV06qmwZMKxN/QBJfIifa2qs2TMtufzaXs76EJhQ36ETypoEK45bAr2p0PqZqCt99xTZS9YJr338bVMVU1YQZXrAzH5EZ/8J9OyyyrwFgNDuBG5W+ujKB46tLNeHYUcXyO8qU+QdHgIAYKn2WYqKbuGyxc/w5JHbaUf/LZ5CL5Er++/hrlyDyrg5yrOz4uxRLCtTHDh40CujF+SR8FLfWKoxB2a+auRRllzGFj6M4bvqUjBQ9vOkrAhoaAOor3wSNNHaWbfEh7RGiNpEi8= ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T14:33:31.070 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsYp7MRg6qIY/ULuJBU7QvMByDRbucEgKtj0FSEfB60hQqwQO03ZWsVd3YgJmE4AhXzmACspll45b+tektFLha7f2Un8a13SanjnHpaewa3gg/3Q/uFfdcUtOUtmmYCjJRQcfe059yeVyPLT20GyLdd2AT9AvAbQbmTuZeRKorlPyahK3voo2rufuzWkmkx8tWkydvVpGVuvgOqjntOGs0nmb4bW+O/NkUHNIoQPGIP42cYuV06qmwZMKxN/QBJfIifa2qs2TMtufzaXs76EJhQ36ETypoEK45bAr2p0PqZqCt99xTZS9YJr338bVMVU1YQZXrAzH5EZ/8J9OyyyrwFgNDuBG5W+ujKB46tLNeHYUcXyO8qU+QdHgIAYKn2WYqKbuGyxc/w5JHbaUf/LZ5CL5Er++/hrlyDyrg5yrOz4uxRLCtTHDh40CujF+SR8FLfWKoxB2a+auRRllzGFj6M4bvqUjBQ9vOkrAhoaAOor3wSNNHaWbfEh7RGiNpEi8= ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:33:31.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:30 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2429424057' entity='client.admin' 2024-04-02T14:33:31.117 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsYp7MRg6qIY/ULuJBU7QvMByDRbucEgKtj0FSEfB60hQqwQO03ZWsVd3YgJmE4AhXzmACspll45b+tektFLha7f2Un8a13SanjnHpaewa3gg/3Q/uFfdcUtOUtmmYCjJRQcfe059yeVyPLT20GyLdd2AT9AvAbQbmTuZeRKorlPyahK3voo2rufuzWkmkx8tWkydvVpGVuvgOqjntOGs0nmb4bW+O/NkUHNIoQPGIP42cYuV06qmwZMKxN/QBJfIifa2qs2TMtufzaXs76EJhQ36ETypoEK45bAr2p0PqZqCt99xTZS9YJr338bVMVU1YQZXrAzH5EZ/8J9OyyyrwFgNDuBG5W+ujKB46tLNeHYUcXyO8qU+QdHgIAYKn2WYqKbuGyxc/w5JHbaUf/LZ5CL5Er++/hrlyDyrg5yrOz4uxRLCtTHDh40CujF+SR8FLfWKoxB2a+auRRllzGFj6M4bvqUjBQ9vOkrAhoaAOor3wSNNHaWbfEh7RGiNpEi8= ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T14:33:31.164 INFO:teuthology.orchestra.run.smithi113.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsYp7MRg6qIY/ULuJBU7QvMByDRbucEgKtj0FSEfB60hQqwQO03ZWsVd3YgJmE4AhXzmACspll45b+tektFLha7f2Un8a13SanjnHpaewa3gg/3Q/uFfdcUtOUtmmYCjJRQcfe059yeVyPLT20GyLdd2AT9AvAbQbmTuZeRKorlPyahK3voo2rufuzWkmkx8tWkydvVpGVuvgOqjntOGs0nmb4bW+O/NkUHNIoQPGIP42cYuV06qmwZMKxN/QBJfIifa2qs2TMtufzaXs76EJhQ36ETypoEK45bAr2p0PqZqCt99xTZS9YJr338bVMVU1YQZXrAzH5EZ/8J9OyyyrwFgNDuBG5W+ujKB46tLNeHYUcXyO8qU+QdHgIAYKn2WYqKbuGyxc/w5JHbaUf/LZ5CL5Er++/hrlyDyrg5yrOz4uxRLCtTHDh40CujF+SR8FLfWKoxB2a+auRRllzGFj6M4bvqUjBQ9vOkrAhoaAOor3wSNNHaWbfEh7RGiNpEi8= ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:33:31.178 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsYp7MRg6qIY/ULuJBU7QvMByDRbucEgKtj0FSEfB60hQqwQO03ZWsVd3YgJmE4AhXzmACspll45b+tektFLha7f2Un8a13SanjnHpaewa3gg/3Q/uFfdcUtOUtmmYCjJRQcfe059yeVyPLT20GyLdd2AT9AvAbQbmTuZeRKorlPyahK3voo2rufuzWkmkx8tWkydvVpGVuvgOqjntOGs0nmb4bW+O/NkUHNIoQPGIP42cYuV06qmwZMKxN/QBJfIifa2qs2TMtufzaXs76EJhQ36ETypoEK45bAr2p0PqZqCt99xTZS9YJr338bVMVU1YQZXrAzH5EZ/8J9OyyyrwFgNDuBG5W+ujKB46tLNeHYUcXyO8qU+QdHgIAYKn2WYqKbuGyxc/w5JHbaUf/LZ5CL5Er++/hrlyDyrg5yrOz4uxRLCtTHDh40CujF+SR8FLfWKoxB2a+auRRllzGFj6M4bvqUjBQ9vOkrAhoaAOor3wSNNHaWbfEh7RGiNpEi8= ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T14:33:31.228 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsYp7MRg6qIY/ULuJBU7QvMByDRbucEgKtj0FSEfB60hQqwQO03ZWsVd3YgJmE4AhXzmACspll45b+tektFLha7f2Un8a13SanjnHpaewa3gg/3Q/uFfdcUtOUtmmYCjJRQcfe059yeVyPLT20GyLdd2AT9AvAbQbmTuZeRKorlPyahK3voo2rufuzWkmkx8tWkydvVpGVuvgOqjntOGs0nmb4bW+O/NkUHNIoQPGIP42cYuV06qmwZMKxN/QBJfIifa2qs2TMtufzaXs76EJhQ36ETypoEK45bAr2p0PqZqCt99xTZS9YJr338bVMVU1YQZXrAzH5EZ/8J9OyyyrwFgNDuBG5W+ujKB46tLNeHYUcXyO8qU+QdHgIAYKn2WYqKbuGyxc/w5JHbaUf/LZ5CL5Er++/hrlyDyrg5yrOz4uxRLCtTHDh40CujF+SR8FLfWKoxB2a+auRRllzGFj6M4bvqUjBQ9vOkrAhoaAOor3wSNNHaWbfEh7RGiNpEi8= ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:33:31.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-02T14:33:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:33:34.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.386+0000 7fcccc200700 1 -- 172.21.15.62:0/1805857284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccc4102b10 msgr2=0x7fccc4104f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:34.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.386+0000 7fcccc200700 1 --2- 172.21.15.62:0/1805857284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccc4102b10 0x7fccc4104f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009910 tx=0x7fccb8009c20 comp rx=0 tx=0).stop 2024-04-02T14:33:34.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.386+0000 7fcccc200700 1 -- 172.21.15.62:0/1805857284 shutdown_connections 2024-04-02T14:33:34.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.386+0000 7fcccc200700 1 --2- 172.21.15.62:0/1805857284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccc4102b10 0x7fccc4104f00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:34.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.386+0000 7fcccc200700 1 -- 172.21.15.62:0/1805857284 >> 172.21.15.62:0/1805857284 conn(0x7fccc40fc4c0 msgr2=0x7fccc40fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:34.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.386+0000 7fcccc200700 1 -- 172.21.15.62:0/1805857284 shutdown_connections 2024-04-02T14:33:34.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.386+0000 7fcccc200700 1 -- 172.21.15.62:0/1805857284 wait complete. 2024-04-02T14:33:34.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.387+0000 7fcccc200700 1 Processor -- start 2024-04-02T14:33:34.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.388+0000 7fcccc200700 1 -- start start 2024-04-02T14:33:34.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.388+0000 7fcccc200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccc4102b10 0x7fccc4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:34.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.388+0000 7fcccc200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4108200 con 0x7fccc4102b10 2024-04-02T14:33:34.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.388+0000 7fccc9800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccc4102b10 0x7fccc4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:34.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.388+0000 7fccc9800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccc4102b10 0x7fccc4199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37676/0 (socket says 172.21.15.62:37676) 2024-04-02T14:33:34.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.388+0000 7fccc9800700 1 -- 172.21.15.62:0/3014759916 learned_addr learned my addr 172.21.15.62:0/3014759916 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:34.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.389+0000 7fccc9800700 1 -- 172.21.15.62:0/3014759916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc4199e90 con 0x7fccc4102b10 2024-04-02T14:33:34.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.389+0000 7fccc9800700 1 --2- 172.21.15.62:0/3014759916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccc4102b10 0x7fccc4199950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fccb8005950 tx=0x7fccb8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:34.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.389+0000 7fccc2000700 1 -- 172.21.15.62:0/3014759916 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccb8027070 con 0x7fccc4102b10 2024-04-02T14:33:34.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.389+0000 7fccc2000700 1 -- 172.21.15.62:0/3014759916 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fccb8022470 con 0x7fccc4102b10 2024-04-02T14:33:34.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.390+0000 7fcccc200700 1 -- 172.21.15.62:0/3014759916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc419a120 con 0x7fccc4102b10 2024-04-02T14:33:34.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.390+0000 7fccc2000700 1 -- 172.21.15.62:0/3014759916 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccb800f590 con 0x7fccc4102b10 2024-04-02T14:33:34.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.390+0000 7fcccc200700 1 -- 172.21.15.62:0/3014759916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc419ce10 con 0x7fccc4102b10 2024-04-02T14:33:34.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.391+0000 7fcccc200700 1 -- 172.21.15.62:0/3014759916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc4108980 con 0x7fccc4102b10 2024-04-02T14:33:34.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.391+0000 7fccc2000700 1 -- 172.21.15.62:0/3014759916 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fccb802a070 con 0x7fccc4102b10 2024-04-02T14:33:34.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.392+0000 7fccc2000700 1 --2- 172.21.15.62:0/3014759916 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fccac0396b0 0x7fccac03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:34.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.392+0000 7fccc2000700 1 -- 172.21.15.62:0/3014759916 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fccb801e030 con 0x7fccc4102b10 2024-04-02T14:33:34.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.392+0000 7fccc8e00700 1 --2- 172.21.15.62:0/3014759916 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fccac0396b0 0x7fccac03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:34.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.393+0000 7fccc8e00700 1 --2- 172.21.15.62:0/3014759916 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fccac0396b0 0x7fccac03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fccb0006fd0 tx=0x7fccb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:34.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.398+0000 7fccc2000700 1 -- 172.21.15.62:0/3014759916 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fccb8013070 con 0x7fccc4102b10 2024-04-02T14:33:34.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.635+0000 7fcccc200700 1 -- 172.21.15.62:0/3014759916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fccc402de40 con 0x7fccc4102b10 2024-04-02T14:33:34.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.638+0000 7fccc2000700 1 -- 172.21.15.62:0/3014759916 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fccb8020de0 con 0x7fccc4102b10 2024-04-02T14:33:34.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.641+0000 7fcccc200700 1 -- 172.21.15.62:0/3014759916 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fccac0396b0 msgr2=0x7fccac03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:34.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.641+0000 7fcccc200700 1 --2- 172.21.15.62:0/3014759916 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fccac0396b0 0x7fccac03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fccb0006fd0 tx=0x7fccb0006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:34.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.641+0000 7fcccc200700 1 -- 172.21.15.62:0/3014759916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccc4102b10 msgr2=0x7fccc4199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:34.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.641+0000 7fcccc200700 1 --2- 172.21.15.62:0/3014759916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccc4102b10 0x7fccc4199950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fccb8005950 tx=0x7fccb8004740 comp rx=0 tx=0).stop 2024-04-02T14:33:34.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.642+0000 7fcccc200700 1 -- 172.21.15.62:0/3014759916 shutdown_connections 2024-04-02T14:33:34.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.642+0000 7fcccc200700 1 --2- 172.21.15.62:0/3014759916 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fccac0396b0 0x7fccac03bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:34.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.642+0000 7fcccc200700 1 --2- 172.21.15.62:0/3014759916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccc4102b10 0x7fccc4199950 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:34.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.642+0000 7fcccc200700 1 -- 172.21.15.62:0/3014759916 >> 172.21.15.62:0/3014759916 conn(0x7fccc40fc4c0 msgr2=0x7fccc40fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:34.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.642+0000 7fcccc200700 1 -- 172.21.15.62:0/3014759916 shutdown_connections 2024-04-02T14:33:34.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:34.642+0000 7fcccc200700 1 -- 172.21.15.62:0/3014759916 wait complete. 2024-04-02T14:33:35.459 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-02T14:33:35.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-02T14:33:35.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:35 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:35.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:35 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:35.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:35 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:33:35.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:35 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:35.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:35 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:33:35.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:35 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:35.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:35 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3014759916' entity='client.admin' 2024-04-02T14:33:35.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:35 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:33:35.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:35 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:35.911 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:33:37.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.915+0000 7f3da7200700 1 -- 172.21.15.62:0/3486090975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8104070 msgr2=0x7f3da8104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:37.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.915+0000 7f3da7200700 1 --2- 172.21.15.62:0/3486090975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8104070 0x7f3da8104490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009910 tx=0x7f3d98009c20 comp rx=0 tx=0).stop 2024-04-02T14:33:37.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.915+0000 7f3da7200700 1 -- 172.21.15.62:0/3486090975 shutdown_connections 2024-04-02T14:33:37.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.915+0000 7f3da7200700 1 --2- 172.21.15.62:0/3486090975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8104070 0x7f3da8104490 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:37.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.915+0000 7f3da7200700 1 -- 172.21.15.62:0/3486090975 >> 172.21.15.62:0/3486090975 conn(0x7f3da80ff630 msgr2=0x7f3da8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:37.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.915+0000 7f3da7200700 1 -- 172.21.15.62:0/3486090975 shutdown_connections 2024-04-02T14:33:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.915+0000 7f3da7200700 1 -- 172.21.15.62:0/3486090975 wait complete. 2024-04-02T14:33:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.916+0000 7f3da7200700 1 Processor -- start 2024-04-02T14:33:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.917+0000 7f3da7200700 1 -- start start 2024-04-02T14:33:37.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.917+0000 7f3da7200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8104070 0x7f3da81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:37.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.917+0000 7f3da7200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3da810a240 con 0x7f3da8104070 2024-04-02T14:33:37.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.917+0000 7f3da5e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8104070 0x7f3da81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:37.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.917+0000 7f3da5e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8104070 0x7f3da81997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37700/0 (socket says 172.21.15.62:37700) 2024-04-02T14:33:37.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.917+0000 7f3da5e00700 1 -- 172.21.15.62:0/3935136212 learned_addr learned my addr 172.21.15.62:0/3935136212 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.918+0000 7f3da5e00700 1 -- 172.21.15.62:0/3935136212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da8199ce0 con 0x7f3da8104070 2024-04-02T14:33:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.918+0000 7f3da5e00700 1 --2- 172.21.15.62:0/3935136212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8104070 0x7f3da81997a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009ed0 tx=0x7f3d98004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.918+0000 7f3d9ea00700 1 -- 172.21.15.62:0/3935136212 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d98027070 con 0x7f3da8104070 2024-04-02T14:33:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.918+0000 7f3d9ea00700 1 -- 172.21.15.62:0/3935136212 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3d98022470 con 0x7f3da8104070 2024-04-02T14:33:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.918+0000 7f3d9ea00700 1 -- 172.21.15.62:0/3935136212 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d9800f5d0 con 0x7f3da8104070 2024-04-02T14:33:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.918+0000 7f3da7200700 1 -- 172.21.15.62:0/3935136212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da8199f70 con 0x7f3da8104070 2024-04-02T14:33:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.919+0000 7f3da7200700 1 -- 172.21.15.62:0/3935136212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da819cc60 con 0x7f3da8104070 2024-04-02T14:33:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.920+0000 7f3d9ea00700 1 -- 172.21.15.62:0/3935136212 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3d9802a070 con 0x7f3da8104070 2024-04-02T14:33:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.920+0000 7f3da7200700 1 -- 172.21.15.62:0/3935136212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da810a9c0 con 0x7f3da8104070 2024-04-02T14:33:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.920+0000 7f3d9ea00700 1 --2- 172.21.15.62:0/3935136212 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3d8c0396b0 0x7f3d8c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.920+0000 7f3d9ea00700 1 -- 172.21.15.62:0/3935136212 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3d9801e030 con 0x7f3da8104070 2024-04-02T14:33:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.921+0000 7f3d9fe00700 1 --2- 172.21.15.62:0/3935136212 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3d8c0396b0 0x7f3d8c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.921+0000 7f3d9fe00700 1 --2- 172.21.15.62:0/3935136212 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3d8c0396b0 0x7f3d8c03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006fd0 tx=0x7f3d90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:37.926+0000 7f3d9ea00700 1 -- 172.21.15.62:0/3935136212 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3d98013070 con 0x7f3da8104070 2024-04-02T14:33:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.160+0000 7f3da7200700 1 -- 172.21.15.62:0/3935136212 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f3da802de40 con 0x7f3d8c0396b0 2024-04-02T14:33:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.168+0000 7f3d9ea00700 1 -- 172.21.15.62:0/3935136212 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3da802de40 con 0x7f3d8c0396b0 2024-04-02T14:33:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.171+0000 7f3da7200700 1 -- 172.21.15.62:0/3935136212 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3d8c0396b0 msgr2=0x7f3d8c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.171+0000 7f3da7200700 1 --2- 172.21.15.62:0/3935136212 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3d8c0396b0 0x7f3d8c03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006fd0 tx=0x7f3d90006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:38.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.171+0000 7f3da7200700 1 -- 172.21.15.62:0/3935136212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8104070 msgr2=0x7f3da81997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:38.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.171+0000 7f3da7200700 1 --2- 172.21.15.62:0/3935136212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8104070 0x7f3da81997a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009ed0 tx=0x7f3d98004690 comp rx=0 tx=0).stop 2024-04-02T14:33:38.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.172+0000 7f3da7200700 1 -- 172.21.15.62:0/3935136212 shutdown_connections 2024-04-02T14:33:38.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.172+0000 7f3da7200700 1 --2- 172.21.15.62:0/3935136212 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3d8c0396b0 0x7f3d8c03bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:38.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.172+0000 7f3da7200700 1 --2- 172.21.15.62:0/3935136212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8104070 0x7f3da81997a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:38.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.172+0000 7f3da7200700 1 -- 172.21.15.62:0/3935136212 >> 172.21.15.62:0/3935136212 conn(0x7f3da80ff630 msgr2=0x7f3da80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:38.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.172+0000 7f3da7200700 1 -- 172.21.15.62:0/3935136212 shutdown_connections 2024-04-02T14:33:38.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:38.172+0000 7f3da7200700 1 -- 172.21.15.62:0/3935136212 wait complete. 2024-04-02T14:33:39.035 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi113 2024-04-02T14:33:39.035 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-02T14:33:39.035 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/etc/ceph/ceph.conf 2024-04-02T14:33:39.054 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-02T14:33:39.054 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:33:39.114 INFO:tasks.cephadm:Adding host smithi113 to orchestrator... 2024-04-02T14:33:39.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi113 2024-04-02T14:33:39.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:39 smithi062 ceph-mon[32116]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:33:39.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:39.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:33:39.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:33:39.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:33:39.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:39 smithi062 ceph-mon[32116]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-02T14:33:39.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:39 smithi062 ceph-mon[32116]: Updating smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:33:39.538 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:33:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:40 smithi062 ceph-mon[32116]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:40 smithi062 ceph-mon[32116]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:33:41.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.745+0000 7fec2fc00700 1 -- 172.21.15.62:0/3993562229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec28074990 msgr2=0x7fec28074d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:41.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.745+0000 7fec2fc00700 1 --2- 172.21.15.62:0/3993562229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec28074990 0x7fec28074d70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fec18009910 tx=0x7fec18009c20 comp rx=0 tx=0).stop 2024-04-02T14:33:41.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.745+0000 7fec2fc00700 1 -- 172.21.15.62:0/3993562229 shutdown_connections 2024-04-02T14:33:41.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.745+0000 7fec2fc00700 1 --2- 172.21.15.62:0/3993562229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec28074990 0x7fec28074d70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:41.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.745+0000 7fec2fc00700 1 -- 172.21.15.62:0/3993562229 >> 172.21.15.62:0/3993562229 conn(0x7fec280fdb00 msgr2=0x7fec280fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:41.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.745+0000 7fec2fc00700 1 -- 172.21.15.62:0/3993562229 shutdown_connections 2024-04-02T14:33:41.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.745+0000 7fec2fc00700 1 -- 172.21.15.62:0/3993562229 wait complete. 2024-04-02T14:33:41.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.746+0000 7fec2fc00700 1 Processor -- start 2024-04-02T14:33:41.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.746+0000 7fec2fc00700 1 -- start start 2024-04-02T14:33:41.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.747+0000 7fec2fc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec28074990 0x7fec2819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:41.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.747+0000 7fec2fc00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fec28112da0 con 0x7fec28074990 2024-04-02T14:33:41.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.747+0000 7fec2d200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec28074990 0x7fec2819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:41.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.747+0000 7fec2d200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec28074990 0x7fec2819e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37708/0 (socket says 172.21.15.62:37708) 2024-04-02T14:33:41.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.747+0000 7fec2d200700 1 -- 172.21.15.62:0/3135804613 learned_addr learned my addr 172.21.15.62:0/3135804613 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:41.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.748+0000 7fec2d200700 1 -- 172.21.15.62:0/3135804613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2819e7a0 con 0x7fec28074990 2024-04-02T14:33:41.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.748+0000 7fec2d200700 1 --2- 172.21.15.62:0/3135804613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec28074990 0x7fec2819e260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fec18006010 tx=0x7fec18004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:41.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.748+0000 7fec26000700 1 -- 172.21.15.62:0/3135804613 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec18026070 con 0x7fec28074990 2024-04-02T14:33:41.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.748+0000 7fec26000700 1 -- 172.21.15.62:0/3135804613 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fec180044a0 con 0x7fec28074990 2024-04-02T14:33:41.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.748+0000 7fec2fc00700 1 -- 172.21.15.62:0/3135804613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2819ea30 con 0x7fec28074990 2024-04-02T14:33:41.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.748+0000 7fec26000700 1 -- 172.21.15.62:0/3135804613 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec18003ba0 con 0x7fec28074990 2024-04-02T14:33:41.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.748+0000 7fec2fc00700 1 -- 172.21.15.62:0/3135804613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2819ee90 con 0x7fec28074990 2024-04-02T14:33:41.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.749+0000 7fec26000700 1 -- 172.21.15.62:0/3135804613 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fec18029070 con 0x7fec28074990 2024-04-02T14:33:41.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.749+0000 7fec2fc00700 1 -- 172.21.15.62:0/3135804613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec280760d0 con 0x7fec28074990 2024-04-02T14:33:41.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.750+0000 7fec26000700 1 --2- 172.21.15.62:0/3135804613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fec0c0396b0 0x7fec0c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:41.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.750+0000 7fec26000700 1 -- 172.21.15.62:0/3135804613 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fec1801e070 con 0x7fec28074990 2024-04-02T14:33:41.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.750+0000 7fec1fe00700 1 --2- 172.21.15.62:0/3135804613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fec0c0396b0 0x7fec0c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:41.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.751+0000 7fec1fe00700 1 --2- 172.21.15.62:0/3135804613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fec0c0396b0 0x7fec0c03bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec10006fd0 tx=0x7fec10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:41.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.756+0000 7fec26000700 1 -- 172.21.15.62:0/3135804613 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fec18013070 con 0x7fec28074990 2024-04-02T14:33:41.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:41.997+0000 7fec2fc00700 1 -- 172.21.15.62:0/3135804613 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi113", "target": ["mon-mgr", ""]}) -- 0x7fec2819f5c0 con 0x7fec0c0396b0 2024-04-02T14:33:42.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:41 smithi062 ceph-mon[32116]: Updating smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:33:42.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:41 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:42.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:41 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:42.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:41 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:43.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:42 smithi062 ceph-mon[32116]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:43.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:42 smithi062 ceph-mon[32116]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi113", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:33:44.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:44 smithi062 ceph-mon[32116]: Deploying cephadm binary to smithi113 2024-04-02T14:33:45.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:45 smithi062 ceph-mon[32116]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:46.918 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi113' with addr '172.21.15.113' 2024-04-02T14:33:46.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:46.915+0000 7fec26000700 1 -- 172.21.15.62:0/3135804613 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fec2819f5c0 con 0x7fec0c0396b0 2024-04-02T14:33:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:46.918+0000 7fec2fc00700 1 -- 172.21.15.62:0/3135804613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fec0c0396b0 msgr2=0x7fec0c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:46.918+0000 7fec2fc00700 1 --2- 172.21.15.62:0/3135804613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fec0c0396b0 0x7fec0c03bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec10006fd0 tx=0x7fec10006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:46.918+0000 7fec2fc00700 1 -- 172.21.15.62:0/3135804613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec28074990 msgr2=0x7fec2819e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:46.918+0000 7fec2fc00700 1 --2- 172.21.15.62:0/3135804613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec28074990 0x7fec2819e260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fec18006010 tx=0x7fec18004cc0 comp rx=0 tx=0).stop 2024-04-02T14:33:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:46.918+0000 7fec2fc00700 1 -- 172.21.15.62:0/3135804613 shutdown_connections 2024-04-02T14:33:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:46.918+0000 7fec2fc00700 1 --2- 172.21.15.62:0/3135804613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fec0c0396b0 0x7fec0c03bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:46.918+0000 7fec2fc00700 1 --2- 172.21.15.62:0/3135804613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec28074990 0x7fec2819e260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:46.918+0000 7fec2fc00700 1 -- 172.21.15.62:0/3135804613 >> 172.21.15.62:0/3135804613 conn(0x7fec280fdb00 msgr2=0x7fec280ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:46.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:46.919+0000 7fec2fc00700 1 -- 172.21.15.62:0/3135804613 shutdown_connections 2024-04-02T14:33:46.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:46.919+0000 7fec2fc00700 1 -- 172.21.15.62:0/3135804613 wait complete. 2024-04-02T14:33:47.315 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:47 smithi062 ceph-mon[32116]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:47.315 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:47.315 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:33:47.716 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T14:33:48.179 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:33:48.318 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:48 smithi062 ceph-mon[32116]: Added host smithi113 2024-04-02T14:33:48.318 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:48 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:48.318 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:48 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:49.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:49 smithi062 ceph-mon[32116]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.265+0000 7fa266400700 1 -- 172.21.15.62:0/2062640711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260103180 msgr2=0x7fa260103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.265+0000 7fa266400700 1 --2- 172.21.15.62:0/2062640711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260103180 0x7fa260103560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa250009910 tx=0x7fa250009c20 comp rx=0 tx=0).stop 2024-04-02T14:33:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.265+0000 7fa266400700 1 -- 172.21.15.62:0/2062640711 shutdown_connections 2024-04-02T14:33:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.265+0000 7fa266400700 1 --2- 172.21.15.62:0/2062640711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260103180 0x7fa260103560 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.265+0000 7fa266400700 1 -- 172.21.15.62:0/2062640711 >> 172.21.15.62:0/2062640711 conn(0x7fa260077b30 msgr2=0x7fa2600fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.265+0000 7fa266400700 1 -- 172.21.15.62:0/2062640711 shutdown_connections 2024-04-02T14:33:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.265+0000 7fa266400700 1 -- 172.21.15.62:0/2062640711 wait complete. 2024-04-02T14:33:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.267+0000 7fa266400700 1 Processor -- start 2024-04-02T14:33:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.267+0000 7fa266400700 1 -- start start 2024-04-02T14:33:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.267+0000 7fa266400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260103180 0x7fa260199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.267+0000 7fa266400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa26010c7b0 con 0x7fa260103180 2024-04-02T14:33:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.267+0000 7fa25f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260103180 0x7fa260199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.268+0000 7fa25f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260103180 0x7fa260199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45676/0 (socket says 172.21.15.62:45676) 2024-04-02T14:33:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.268+0000 7fa25f400700 1 -- 172.21.15.62:0/334967678 learned_addr learned my addr 172.21.15.62:0/334967678 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.268+0000 7fa25f400700 1 -- 172.21.15.62:0/334967678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26019a390 con 0x7fa260103180 2024-04-02T14:33:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.268+0000 7fa25f400700 1 --2- 172.21.15.62:0/334967678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260103180 0x7fa260199e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa250004c10 tx=0x7fa250004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.269+0000 7fa25cc00700 1 -- 172.21.15.62:0/334967678 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa250026070 con 0x7fa260103180 2024-04-02T14:33:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.269+0000 7fa25cc00700 1 -- 172.21.15.62:0/334967678 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa250004370 con 0x7fa260103180 2024-04-02T14:33:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.269+0000 7fa25cc00700 1 -- 172.21.15.62:0/334967678 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa250003bf0 con 0x7fa260103180 2024-04-02T14:33:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.269+0000 7fa266400700 1 -- 172.21.15.62:0/334967678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26019a620 con 0x7fa260103180 2024-04-02T14:33:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.269+0000 7fa266400700 1 -- 172.21.15.62:0/334967678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa26019aa80 con 0x7fa260103180 2024-04-02T14:33:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.270+0000 7fa25cc00700 1 -- 172.21.15.62:0/334967678 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa250029070 con 0x7fa260103180 2024-04-02T14:33:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.271+0000 7fa266400700 1 -- 172.21.15.62:0/334967678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26006a2c0 con 0x7fa260103180 2024-04-02T14:33:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.271+0000 7fa25cc00700 1 --2- 172.21.15.62:0/334967678 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400396b0 0x7fa24003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.271+0000 7fa25cc00700 1 -- 172.21.15.62:0/334967678 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa25001e070 con 0x7fa260103180 2024-04-02T14:33:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.271+0000 7fa256a00700 1 --2- 172.21.15.62:0/334967678 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400396b0 0x7fa24003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.272+0000 7fa256a00700 1 --2- 172.21.15.62:0/334967678 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400396b0 0x7fa24003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa248006fd0 tx=0x7fa248006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:50.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.277+0000 7fa25cc00700 1 -- 172.21.15.62:0/334967678 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa250013070 con 0x7fa260103180 2024-04-02T14:33:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.518+0000 7fa266400700 1 -- 172.21.15.62:0/334967678 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa26019b220 con 0x7fa2400396b0 2024-04-02T14:33:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.521+0000 7fa25cc00700 1 -- 172.21.15.62:0/334967678 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fa26019b220 con 0x7fa2400396b0 2024-04-02T14:33:50.523 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:33:50.524 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.113", "hostname": "smithi113", "labels": [], "status": ""}] 2024-04-02T14:33:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.524+0000 7fa266400700 1 -- 172.21.15.62:0/334967678 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400396b0 msgr2=0x7fa24003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.524+0000 7fa266400700 1 --2- 172.21.15.62:0/334967678 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400396b0 0x7fa24003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa248006fd0 tx=0x7fa248006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.524+0000 7fa266400700 1 -- 172.21.15.62:0/334967678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260103180 msgr2=0x7fa260199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.524+0000 7fa266400700 1 --2- 172.21.15.62:0/334967678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260103180 0x7fa260199e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa250004c10 tx=0x7fa250004cf0 comp rx=0 tx=0).stop 2024-04-02T14:33:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.524+0000 7fa266400700 1 -- 172.21.15.62:0/334967678 shutdown_connections 2024-04-02T14:33:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.524+0000 7fa266400700 1 --2- 172.21.15.62:0/334967678 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400396b0 0x7fa24003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.524+0000 7fa266400700 1 --2- 172.21.15.62:0/334967678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260103180 0x7fa260199e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.524+0000 7fa266400700 1 -- 172.21.15.62:0/334967678 >> 172.21.15.62:0/334967678 conn(0x7fa260077b30 msgr2=0x7fa260108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:50.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.525+0000 7fa266400700 1 -- 172.21.15.62:0/334967678 shutdown_connections 2024-04-02T14:33:50.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:50.525+0000 7fa266400700 1 -- 172.21.15.62:0/334967678 wait complete. 2024-04-02T14:33:50.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:50 smithi062 ceph-mon[32116]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:50.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:50 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:51.326 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-04-02T14:33:51.326 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T14:33:51.326 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-04-02T14:33:51.345 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T14:33:51.345 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:33:51.402 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-04-02T14:33:51.402 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi204 2024-04-02T14:33:51.871 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:33:52.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:51 smithi062 ceph-mon[32116]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:33:52.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:52 smithi062 ceph-mon[32116]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:52.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:52.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:52.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:52.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:52.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi113", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:33:52.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:33:52.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:33:53.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:53 smithi062 ceph-mon[32116]: Updating smithi113:/etc/ceph/ceph.conf 2024-04-02T14:33:53.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:53 smithi062 ceph-mon[32116]: Updating smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:33:53.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:53 smithi062 ceph-mon[32116]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:33:54.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.054+0000 7f6c58800700 1 -- 172.21.15.62:0/2140677527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c50102220 msgr2=0x7f6c50102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:54.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.054+0000 7f6c58800700 1 --2- 172.21.15.62:0/2140677527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c50102220 0x7f6c50102600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009910 tx=0x7f6c44009c20 comp rx=0 tx=0).stop 2024-04-02T14:33:54.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.054+0000 7f6c58800700 1 -- 172.21.15.62:0/2140677527 shutdown_connections 2024-04-02T14:33:54.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.054+0000 7f6c58800700 1 --2- 172.21.15.62:0/2140677527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c50102220 0x7f6c50102600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:54.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.054+0000 7f6c58800700 1 -- 172.21.15.62:0/2140677527 >> 172.21.15.62:0/2140677527 conn(0x7f6c500fdad0 msgr2=0x7f6c500ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:54.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.054+0000 7f6c58800700 1 -- 172.21.15.62:0/2140677527 shutdown_connections 2024-04-02T14:33:54.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.054+0000 7f6c58800700 1 -- 172.21.15.62:0/2140677527 wait complete. 2024-04-02T14:33:54.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.055+0000 7f6c58800700 1 Processor -- start 2024-04-02T14:33:54.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.055+0000 7f6c58800700 1 -- start start 2024-04-02T14:33:54.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.056+0000 7f6c58800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c50102220 0x7f6c50199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:54.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.056+0000 7f6c58800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5010c730 con 0x7f6c50102220 2024-04-02T14:33:54.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.056+0000 7f6c55e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c50102220 0x7f6c50199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:54.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.056+0000 7f6c55e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c50102220 0x7f6c50199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39732/0 (socket says 172.21.15.62:39732) 2024-04-02T14:33:54.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.056+0000 7f6c55e00700 1 -- 172.21.15.62:0/4232411114 learned_addr learned my addr 172.21.15.62:0/4232411114 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:33:54.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.056+0000 7f6c55e00700 1 -- 172.21.15.62:0/4232411114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5019a2f0 con 0x7f6c50102220 2024-04-02T14:33:54.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.057+0000 7f6c55e00700 1 --2- 172.21.15.62:0/4232411114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c50102220 0x7f6c50199db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6c44006010 tx=0x7f6c44004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:54.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.057+0000 7f6c4aa00700 1 -- 172.21.15.62:0/4232411114 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c44026070 con 0x7f6c50102220 2024-04-02T14:33:54.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.057+0000 7f6c4aa00700 1 -- 172.21.15.62:0/4232411114 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6c440044a0 con 0x7f6c50102220 2024-04-02T14:33:54.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.057+0000 7f6c4aa00700 1 -- 172.21.15.62:0/4232411114 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c44003a70 con 0x7f6c50102220 2024-04-02T14:33:54.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.057+0000 7f6c58800700 1 -- 172.21.15.62:0/4232411114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5019a5e0 con 0x7f6c50102220 2024-04-02T14:33:54.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.057+0000 7f6c58800700 1 -- 172.21.15.62:0/4232411114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5019e680 con 0x7f6c50102220 2024-04-02T14:33:54.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.058+0000 7f6c4aa00700 1 -- 172.21.15.62:0/4232411114 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6c44029070 con 0x7f6c50102220 2024-04-02T14:33:54.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.058+0000 7f6c58800700 1 -- 172.21.15.62:0/4232411114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c50103700 con 0x7f6c50102220 2024-04-02T14:33:54.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.059+0000 7f6c4aa00700 1 --2- 172.21.15.62:0/4232411114 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6c38039650 0x7f6c3803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:33:54.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.059+0000 7f6c4aa00700 1 -- 172.21.15.62:0/4232411114 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c4401e070 con 0x7f6c50102220 2024-04-02T14:33:54.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.060+0000 7f6c55400700 1 --2- 172.21.15.62:0/4232411114 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6c38039650 0x7f6c3803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:33:54.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.060+0000 7f6c55400700 1 --2- 172.21.15.62:0/4232411114 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6c38039650 0x7f6c3803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6c40006fd0 tx=0x7f6c40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:33:54.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.065+0000 7f6c4aa00700 1 -- 172.21.15.62:0/4232411114 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6c44013070 con 0x7f6c50102220 2024-04-02T14:33:54.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:54.304+0000 7f6c58800700 1 -- 172.21.15.62:0/4232411114 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) -- 0x7f6c5019b110 con 0x7f6c38039650 2024-04-02T14:33:55.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:54 smithi062 ceph-mon[32116]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:55.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:54 smithi062 ceph-mon[32116]: Updating smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:33:55.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:54 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:55.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:54 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:55.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:54 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:33:56.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:55 smithi062 ceph-mon[32116]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:33:56.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:55 smithi062 ceph-mon[32116]: Deploying cephadm binary to smithi204 2024-04-02T14:33:57.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:56 smithi062 ceph-mon[32116]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:59.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:33:58 smithi062 ceph-mon[32116]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:33:59.268 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-04-02T14:33:59.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:59.266+0000 7f6c4aa00700 1 -- 172.21.15.62:0/4232411114 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6c5019b110 con 0x7f6c38039650 2024-04-02T14:33:59.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:59.270+0000 7f6c58800700 1 -- 172.21.15.62:0/4232411114 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6c38039650 msgr2=0x7f6c3803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:59.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:59.270+0000 7f6c58800700 1 --2- 172.21.15.62:0/4232411114 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6c38039650 0x7f6c3803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6c40006fd0 tx=0x7f6c40006e40 comp rx=0 tx=0).stop 2024-04-02T14:33:59.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:59.270+0000 7f6c58800700 1 -- 172.21.15.62:0/4232411114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c50102220 msgr2=0x7f6c50199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:33:59.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:59.270+0000 7f6c58800700 1 --2- 172.21.15.62:0/4232411114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c50102220 0x7f6c50199db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6c44006010 tx=0x7f6c44004cc0 comp rx=0 tx=0).stop 2024-04-02T14:33:59.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:59.271+0000 7f6c58800700 1 -- 172.21.15.62:0/4232411114 shutdown_connections 2024-04-02T14:33:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:59.271+0000 7f6c58800700 1 --2- 172.21.15.62:0/4232411114 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6c38039650 0x7f6c3803bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:59.271+0000 7f6c58800700 1 --2- 172.21.15.62:0/4232411114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c50102220 0x7f6c50199db0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:33:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:59.271+0000 7f6c58800700 1 -- 172.21.15.62:0/4232411114 >> 172.21.15.62:0/4232411114 conn(0x7f6c500fdad0 msgr2=0x7f6c500761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:33:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:59.271+0000 7f6c58800700 1 -- 172.21.15.62:0/4232411114 shutdown_connections 2024-04-02T14:33:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:33:59.271+0000 7f6c58800700 1 -- 172.21.15.62:0/4232411114 wait complete. 2024-04-02T14:34:00.025 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T14:34:00.478 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:34:00.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:00 smithi062 ceph-mon[32116]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:00.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:00.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:00 smithi062 ceph-mon[32116]: Added host smithi204 2024-04-02T14:34:00.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:00.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:01.697 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:02.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.573+0000 7fddc6e00700 1 -- 172.21.15.62:0/3176634538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddc0074910 msgr2=0x7fddc0074cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:02.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.573+0000 7fddc6e00700 1 --2- 172.21.15.62:0/3176634538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddc0074910 0x7fddc0074cf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fddb4009910 tx=0x7fddb4009c20 comp rx=0 tx=0).stop 2024-04-02T14:34:02.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.573+0000 7fddc6e00700 1 -- 172.21.15.62:0/3176634538 shutdown_connections 2024-04-02T14:34:02.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.573+0000 7fddc6e00700 1 --2- 172.21.15.62:0/3176634538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddc0074910 0x7fddc0074cf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:02.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.573+0000 7fddc6e00700 1 -- 172.21.15.62:0/3176634538 >> 172.21.15.62:0/3176634538 conn(0x7fddc00fd920 msgr2=0x7fddc00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:02.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.574+0000 7fddc6e00700 1 -- 172.21.15.62:0/3176634538 shutdown_connections 2024-04-02T14:34:02.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.574+0000 7fddc6e00700 1 -- 172.21.15.62:0/3176634538 wait complete. 2024-04-02T14:34:02.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.575+0000 7fddc6e00700 1 Processor -- start 2024-04-02T14:34:02.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.575+0000 7fddc6e00700 1 -- start start 2024-04-02T14:34:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.575+0000 7fddc6e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddc0074910 0x7fddc019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.575+0000 7fddc6e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fddc010e7f0 con 0x7fddc0074910 2024-04-02T14:34:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.576+0000 7fddc5a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddc0074910 0x7fddc019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:02.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.576+0000 7fddc5a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddc0074910 0x7fddc019f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48736/0 (socket says 172.21.15.62:48736) 2024-04-02T14:34:02.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.576+0000 7fddc5a00700 1 -- 172.21.15.62:0/2298129375 learned_addr learned my addr 172.21.15.62:0/2298129375 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:34:02.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.576+0000 7fddc5a00700 1 -- 172.21.15.62:0/2298129375 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc019fdc0 con 0x7fddc0074910 2024-04-02T14:34:02.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.576+0000 7fddc5a00700 1 --2- 172.21.15.62:0/2298129375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddc0074910 0x7fddc019f880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fddb4006010 tx=0x7fddb4004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:02.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.577+0000 7fddbe000700 1 -- 172.21.15.62:0/2298129375 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddb4026070 con 0x7fddc0074910 2024-04-02T14:34:02.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.577+0000 7fddbe000700 1 -- 172.21.15.62:0/2298129375 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fddb40044a0 con 0x7fddc0074910 2024-04-02T14:34:02.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.577+0000 7fddbe000700 1 -- 172.21.15.62:0/2298129375 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddb4003ba0 con 0x7fddc0074910 2024-04-02T14:34:02.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.577+0000 7fddc6e00700 1 -- 172.21.15.62:0/2298129375 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc01a0050 con 0x7fddc0074910 2024-04-02T14:34:02.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.577+0000 7fddc6e00700 1 -- 172.21.15.62:0/2298129375 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc0199a40 con 0x7fddc0074910 2024-04-02T14:34:02.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.578+0000 7fddc6e00700 1 -- 172.21.15.62:0/2298129375 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc0076050 con 0x7fddc0074910 2024-04-02T14:34:02.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.578+0000 7fddbe000700 1 -- 172.21.15.62:0/2298129375 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fddb4029070 con 0x7fddc0074910 2024-04-02T14:34:02.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.579+0000 7fddbe000700 1 --2- 172.21.15.62:0/2298129375 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdda80396b0 0x7fdda803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:02.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.579+0000 7fddbe000700 1 -- 172.21.15.62:0/2298129375 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fddb401e070 con 0x7fddc0074910 2024-04-02T14:34:02.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.579+0000 7fddc5000700 1 --2- 172.21.15.62:0/2298129375 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdda80396b0 0x7fdda803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:02.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.580+0000 7fddc5000700 1 --2- 172.21.15.62:0/2298129375 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdda80396b0 0x7fdda803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fddac006fd0 tx=0x7fddac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:02.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.585+0000 7fddbe000700 1 -- 172.21.15.62:0/2298129375 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fddb4013070 con 0x7fddc0074910 2024-04-02T14:34:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.826+0000 7fddc6e00700 1 -- 172.21.15.62:0/2298129375 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fddc019af70 con 0x7fdda80396b0 2024-04-02T14:34:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.828+0000 7fddbe000700 1 -- 172.21.15.62:0/2298129375 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fddc019af70 con 0x7fdda80396b0 2024-04-02T14:34:02.831 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:34:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.113", "hostname": "smithi113", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-04-02T14:34:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.831+0000 7fddc6e00700 1 -- 172.21.15.62:0/2298129375 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdda80396b0 msgr2=0x7fdda803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.831+0000 7fddc6e00700 1 --2- 172.21.15.62:0/2298129375 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdda80396b0 0x7fdda803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fddac006fd0 tx=0x7fddac006e40 comp rx=0 tx=0).stop 2024-04-02T14:34:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.832+0000 7fddc6e00700 1 -- 172.21.15.62:0/2298129375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddc0074910 msgr2=0x7fddc019f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.832+0000 7fddc6e00700 1 --2- 172.21.15.62:0/2298129375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddc0074910 0x7fddc019f880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fddb4006010 tx=0x7fddb4004cc0 comp rx=0 tx=0).stop 2024-04-02T14:34:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.832+0000 7fddc6e00700 1 -- 172.21.15.62:0/2298129375 shutdown_connections 2024-04-02T14:34:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.832+0000 7fddc6e00700 1 --2- 172.21.15.62:0/2298129375 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdda80396b0 0x7fdda803bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.832+0000 7fddc6e00700 1 --2- 172.21.15.62:0/2298129375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddc0074910 0x7fddc019f880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.832+0000 7fddc6e00700 1 -- 172.21.15.62:0/2298129375 >> 172.21.15.62:0/2298129375 conn(0x7fddc00fd920 msgr2=0x7fddc0106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.832+0000 7fddc6e00700 1 -- 172.21.15.62:0/2298129375 shutdown_connections 2024-04-02T14:34:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:02.832+0000 7fddc6e00700 1 -- 172.21.15.62:0/2298129375 wait complete. 2024-04-02T14:34:03.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:03 smithi062 ceph-mon[32116]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:03.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:03 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:03.553 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-02T14:34:03.554 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd crush tunables default 2024-04-02T14:34:04.026 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:34:04.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:04 smithi062 ceph-mon[32116]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:34:05.310 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:05 smithi062 ceph-mon[32116]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:06.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.160+0000 7efc02e00700 1 -- 172.21.15.62:0/799138257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbfc102070 msgr2=0x7efbfc102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:06.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.160+0000 7efc02e00700 1 --2- 172.21.15.62:0/799138257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbfc102070 0x7efbfc102450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009910 tx=0x7efbf0009c20 comp rx=0 tx=0).stop 2024-04-02T14:34:06.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.160+0000 7efc02e00700 1 -- 172.21.15.62:0/799138257 shutdown_connections 2024-04-02T14:34:06.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.160+0000 7efc02e00700 1 --2- 172.21.15.62:0/799138257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbfc102070 0x7efbfc102450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:06.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.160+0000 7efc02e00700 1 -- 172.21.15.62:0/799138257 >> 172.21.15.62:0/799138257 conn(0x7efbfc0fd920 msgr2=0x7efbfc0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:06.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.161+0000 7efc02e00700 1 -- 172.21.15.62:0/799138257 shutdown_connections 2024-04-02T14:34:06.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.161+0000 7efc02e00700 1 -- 172.21.15.62:0/799138257 wait complete. 2024-04-02T14:34:06.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.162+0000 7efc02e00700 1 Processor -- start 2024-04-02T14:34:06.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.162+0000 7efc02e00700 1 -- start start 2024-04-02T14:34:06.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.162+0000 7efc02e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbfc102070 0x7efbfc19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:06.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.162+0000 7efc02e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc10c580 con 0x7efbfc102070 2024-04-02T14:34:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.163+0000 7efc01a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbfc102070 0x7efbfc19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.163+0000 7efc01a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbfc102070 0x7efbfc19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48762/0 (socket says 172.21.15.62:48762) 2024-04-02T14:34:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.163+0000 7efc01a00700 1 -- 172.21.15.62:0/3393856878 learned_addr learned my addr 172.21.15.62:0/3393856878 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:34:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.163+0000 7efc01a00700 1 -- 172.21.15.62:0/3393856878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc19fd70 con 0x7efbfc102070 2024-04-02T14:34:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.163+0000 7efc01a00700 1 --2- 172.21.15.62:0/3393856878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbfc102070 0x7efbfc19f830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efbf0006010 tx=0x7efbf0004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.164+0000 7efbfa000700 1 -- 172.21.15.62:0/3393856878 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf0026070 con 0x7efbfc102070 2024-04-02T14:34:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.164+0000 7efbfa000700 1 -- 172.21.15.62:0/3393856878 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efbf00044a0 con 0x7efbfc102070 2024-04-02T14:34:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.164+0000 7efc02e00700 1 -- 172.21.15.62:0/3393856878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbfc1a0060 con 0x7efbfc102070 2024-04-02T14:34:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.164+0000 7efbfa000700 1 -- 172.21.15.62:0/3393856878 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf0003a70 con 0x7efbfc102070 2024-04-02T14:34:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.164+0000 7efc02e00700 1 -- 172.21.15.62:0/3393856878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbfc1999f0 con 0x7efbfc102070 2024-04-02T14:34:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.165+0000 7efbfa000700 1 -- 172.21.15.62:0/3393856878 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efbf0029070 con 0x7efbfc102070 2024-04-02T14:34:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.165+0000 7efc02e00700 1 -- 172.21.15.62:0/3393856878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbfc103550 con 0x7efbfc102070 2024-04-02T14:34:06.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.166+0000 7efbfa000700 1 --2- 172.21.15.62:0/3393856878 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7efbe4039650 0x7efbe403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:06.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.166+0000 7efbfa000700 1 -- 172.21.15.62:0/3393856878 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efbf001e070 con 0x7efbfc102070 2024-04-02T14:34:06.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.166+0000 7efc01000700 1 --2- 172.21.15.62:0/3393856878 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7efbe4039650 0x7efbe403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:06.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.167+0000 7efc01000700 1 --2- 172.21.15.62:0/3393856878 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7efbe4039650 0x7efbe403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efbe8006fd0 tx=0x7efbe8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:06.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.171+0000 7efbfa000700 1 -- 172.21.15.62:0/3393856878 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efbf0013070 con 0x7efbfc102070 2024-04-02T14:34:06.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:06.430+0000 7efc02e00700 1 -- 172.21.15.62:0/3393856878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7efbfc19afc0 con 0x7efbfc102070 2024-04-02T14:34:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:06 smithi062 ceph-mon[32116]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:06.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:34:06.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:06.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:06.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:06 smithi062 ceph-mon[32116]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-02T14:34:06.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:06 smithi062 ceph-mon[32116]: Updating smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:34:06.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:06 smithi062 ceph-mon[32116]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:34:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:07.287+0000 7efbfa000700 1 -- 172.21.15.62:0/3393856878 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7efbf00282c0 con 0x7efbfc102070 2024-04-02T14:34:07.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:07.292+0000 7efc02e00700 1 -- 172.21.15.62:0/3393856878 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7efbe4039650 msgr2=0x7efbe403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:07.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:07.292+0000 7efc02e00700 1 --2- 172.21.15.62:0/3393856878 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7efbe4039650 0x7efbe403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efbe8006fd0 tx=0x7efbe8006e40 comp rx=0 tx=0).stop 2024-04-02T14:34:07.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:07.292+0000 7efc02e00700 1 -- 172.21.15.62:0/3393856878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbfc102070 msgr2=0x7efbfc19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:07.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:07.292+0000 7efc02e00700 1 --2- 172.21.15.62:0/3393856878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbfc102070 0x7efbfc19f830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efbf0006010 tx=0x7efbf0004cc0 comp rx=0 tx=0).stop 2024-04-02T14:34:07.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:07.292+0000 7efc02e00700 1 -- 172.21.15.62:0/3393856878 shutdown_connections 2024-04-02T14:34:07.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:07.292+0000 7efc02e00700 1 --2- 172.21.15.62:0/3393856878 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7efbe4039650 0x7efbe403bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:07.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:07.292+0000 7efc02e00700 1 --2- 172.21.15.62:0/3393856878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efbfc102070 0x7efbfc19f830 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:07.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:07.292+0000 7efc02e00700 1 -- 172.21.15.62:0/3393856878 >> 172.21.15.62:0/3393856878 conn(0x7efbfc0fd920 msgr2=0x7efbfc0762e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:07.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:07.292+0000 7efc02e00700 1 -- 172.21.15.62:0/3393856878 shutdown_connections 2024-04-02T14:34:07.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:07.292+0000 7efc02e00700 1 -- 172.21.15.62:0/3393856878 wait complete. 2024-04-02T14:34:07.295 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2024-04-02T14:34:07.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:07 smithi062 ceph-mon[32116]: Updating smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:34:07.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:07 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3393856878' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-02T14:34:07.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:07 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:07.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:07 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:07.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:07 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:08.015 INFO:tasks.cephadm:Adding mon.a on smithi062 2024-04-02T14:34:08.015 INFO:tasks.cephadm:Adding mon.b on smithi113 2024-04-02T14:34:08.016 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-04-02T14:34:08.016 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch apply mon '3;smithi062:172.21.15.62=a;smithi113:172.21.15.113=b;smithi204:172.21.15.204=c' 2024-04-02T14:34:08.257 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:34:08.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:08 smithi062 ceph-mon[32116]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:08.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:08 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3393856878' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-02T14:34:08.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:08 smithi062 ceph-mon[32116]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T14:34:09.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.550+0000 7fd616484700 1 -- 172.21.15.204:0/788966840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd610102220 msgr2=0x7fd610102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:09.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.550+0000 7fd616484700 1 --2- 172.21.15.204:0/788966840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd610102220 0x7fd610102600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd600009910 tx=0x7fd600009c20 comp rx=0 tx=0).stop 2024-04-02T14:34:09.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.550+0000 7fd616484700 1 -- 172.21.15.204:0/788966840 shutdown_connections 2024-04-02T14:34:09.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.550+0000 7fd616484700 1 --2- 172.21.15.204:0/788966840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd610102220 0x7fd610102600 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:09.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.550+0000 7fd616484700 1 -- 172.21.15.204:0/788966840 >> 172.21.15.204:0/788966840 conn(0x7fd6100fdb30 msgr2=0x7fd6100fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:09.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.550+0000 7fd616484700 1 -- 172.21.15.204:0/788966840 shutdown_connections 2024-04-02T14:34:09.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.550+0000 7fd616484700 1 -- 172.21.15.204:0/788966840 wait complete. 2024-04-02T14:34:09.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.551+0000 7fd616484700 1 Processor -- start 2024-04-02T14:34:09.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.551+0000 7fd616484700 1 -- start start 2024-04-02T14:34:09.554 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.551+0000 7fd616484700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd610102220 0x7fd6100748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:09.554 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.551+0000 7fd616484700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd61010c730 con 0x7fd610102220 2024-04-02T14:34:09.554 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.552+0000 7fd60ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd610102220 0x7fd6100748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:09.554 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.552+0000 7fd60ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd610102220 0x7fd6100748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.204:44406/0 (socket says 172.21.15.204:44406) 2024-04-02T14:34:09.554 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.552+0000 7fd60ffff700 1 -- 172.21.15.204:0/3631554741 learned_addr learned my addr 172.21.15.204:0/3631554741 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T14:34:09.554 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.552+0000 7fd60ffff700 1 -- 172.21.15.204:0/3631554741 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6100767e0 con 0x7fd610102220 2024-04-02T14:34:09.555 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.553+0000 7fd60ffff700 1 --2- 172.21.15.204:0/3631554741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd610102220 0x7fd6100748f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd600006010 tx=0x7fd600004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:09.555 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.553+0000 7fd60d7fa700 1 -- 172.21.15.204:0/3631554741 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd600026070 con 0x7fd610102220 2024-04-02T14:34:09.555 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.553+0000 7fd616484700 1 -- 172.21.15.204:0/3631554741 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd610074e90 con 0x7fd610102220 2024-04-02T14:34:09.555 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.553+0000 7fd616484700 1 -- 172.21.15.204:0/3631554741 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd610075350 con 0x7fd610102220 2024-04-02T14:34:09.555 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.553+0000 7fd60d7fa700 1 -- 172.21.15.204:0/3631554741 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd60000b6f0 con 0x7fd610102220 2024-04-02T14:34:09.555 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.553+0000 7fd60d7fa700 1 -- 172.21.15.204:0/3631554741 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd600030060 con 0x7fd610102220 2024-04-02T14:34:09.556 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.554+0000 7fd616484700 1 -- 172.21.15.204:0/3631554741 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5fc005330 con 0x7fd610102220 2024-04-02T14:34:09.556 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.555+0000 7fd60d7fa700 1 -- 172.21.15.204:0/3631554741 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd600029070 con 0x7fd610102220 2024-04-02T14:34:09.556 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.555+0000 7fd60d7fa700 1 --2- 172.21.15.204:0/3631554741 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd5f8039600 0x7fd5f803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:09.556 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.555+0000 7fd60d7fa700 1 -- 172.21.15.204:0/3631554741 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd60001e070 con 0x7fd610102220 2024-04-02T14:34:09.557 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.556+0000 7fd60f7fe700 1 --2- 172.21.15.204:0/3631554741 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd5f8039600 0x7fd5f803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:09.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.556+0000 7fd60f7fe700 1 --2- 172.21.15.204:0/3631554741 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd5f8039600 0x7fd5f803bac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd604006fd0 tx=0x7fd604006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:09.572 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.571+0000 7fd60d7fa700 1 -- 172.21.15.204:0/3631554741 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd600013070 con 0x7fd610102220 2024-04-02T14:34:09.782 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.781+0000 7fd616484700 1 -- 172.21.15.204:0/3631554741 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi113:172.21.15.113=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) -- 0x7fd5fc000c80 con 0x7fd5f8039600 2024-04-02T14:34:09.798 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.796+0000 7fd60d7fa700 1 -- 172.21.15.204:0/3631554741 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd5fc000c80 con 0x7fd5f8039600 2024-04-02T14:34:09.798 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-04-02T14:34:09.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.799+0000 7fd616484700 1 -- 172.21.15.204:0/3631554741 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd5f8039600 msgr2=0x7fd5f803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:09.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.799+0000 7fd616484700 1 --2- 172.21.15.204:0/3631554741 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd5f8039600 0x7fd5f803bac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd604006fd0 tx=0x7fd604006e40 comp rx=0 tx=0).stop 2024-04-02T14:34:09.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.799+0000 7fd616484700 1 -- 172.21.15.204:0/3631554741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd610102220 msgr2=0x7fd6100748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:09.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.799+0000 7fd616484700 1 --2- 172.21.15.204:0/3631554741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd610102220 0x7fd6100748f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd600006010 tx=0x7fd600004cc0 comp rx=0 tx=0).stop 2024-04-02T14:34:09.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.799+0000 7fd616484700 1 -- 172.21.15.204:0/3631554741 shutdown_connections 2024-04-02T14:34:09.801 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.799+0000 7fd616484700 1 --2- 172.21.15.204:0/3631554741 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd5f8039600 0x7fd5f803bac0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:09.801 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.799+0000 7fd616484700 1 --2- 172.21.15.204:0/3631554741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd610102220 0x7fd6100748f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:09.801 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.799+0000 7fd616484700 1 -- 172.21.15.204:0/3631554741 >> 172.21.15.204:0/3631554741 conn(0x7fd6100fdb30 msgr2=0x7fd610103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:09.801 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.799+0000 7fd616484700 1 -- 172.21.15.204:0/3631554741 shutdown_connections 2024-04-02T14:34:09.801 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:09.799+0000 7fd616484700 1 -- 172.21.15.204:0/3631554741 wait complete. 2024-04-02T14:34:10.429 DEBUG:teuthology.orchestra.run.smithi113:mon.b> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.b.service 2024-04-02T14:34:10.431 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.c.service 2024-04-02T14:34:10.434 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T14:34:10.434 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T14:34:11.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:10 smithi062 ceph-mon[32116]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:11.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:10 smithi062 ceph-mon[32116]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi113:172.21.15.113=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:34:11.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:10 smithi062 ceph-mon[32116]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi113:172.21.15.113=b;smithi204:172.21.15.204=c;count:3 2024-04-02T14:34:11.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:10 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:11.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:10 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:11.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:10 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:11.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:10 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:11.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:10 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:11.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:10 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:11.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:10 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:11.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:10 smithi062 ceph-mon[32116]: Deploying daemon mon.c on smithi204 2024-04-02T14:34:12.376 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:34:13.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:12 smithi062 ceph-mon[32116]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:14.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.431+0000 7f262cdd5700 1 -- 172.21.15.204:0/264026901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2628101220 msgr2=0x7f2628101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:14.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.431+0000 7f262cdd5700 1 --2- 172.21.15.204:0/264026901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2628101220 0x7f2628101600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f261c009910 tx=0x7f261c009c20 comp rx=0 tx=0).stop 2024-04-02T14:34:14.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.431+0000 7f262cdd5700 1 -- 172.21.15.204:0/264026901 shutdown_connections 2024-04-02T14:34:14.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.431+0000 7f262cdd5700 1 --2- 172.21.15.204:0/264026901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2628101220 0x7f2628101600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:14.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.431+0000 7f262cdd5700 1 -- 172.21.15.204:0/264026901 >> 172.21.15.204:0/264026901 conn(0x7f26280fcab0 msgr2=0x7f26280feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:14.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.431+0000 7f262cdd5700 1 -- 172.21.15.204:0/264026901 shutdown_connections 2024-04-02T14:34:14.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.431+0000 7f262cdd5700 1 -- 172.21.15.204:0/264026901 wait complete. 2024-04-02T14:34:14.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.432+0000 7f262cdd5700 1 Processor -- start 2024-04-02T14:34:14.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.432+0000 7f262cdd5700 1 -- start start 2024-04-02T14:34:14.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.432+0000 7f262cdd5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2628101220 0x7f2628198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:14.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.432+0000 7f262cdd5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f262810b730 con 0x7f2628101220 2024-04-02T14:34:14.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.433+0000 7f262659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2628101220 0x7f2628198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:14.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.433+0000 7f262659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2628101220 0x7f2628198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.204:52132/0 (socket says 172.21.15.204:52132) 2024-04-02T14:34:14.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.433+0000 7f262659c700 1 -- 172.21.15.204:0/969938639 learned_addr learned my addr 172.21.15.204:0/969938639 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T14:34:14.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.433+0000 7f262659c700 1 -- 172.21.15.204:0/969938639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2628199020 con 0x7f2628101220 2024-04-02T14:34:14.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.434+0000 7f262659c700 1 --2- 172.21.15.204:0/969938639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2628101220 0x7f2628198ae0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f261c0052a0 tx=0x7f261c004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:14.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.434+0000 7f26177fe700 1 -- 172.21.15.204:0/969938639 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f261c026070 con 0x7f2628101220 2024-04-02T14:34:14.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.434+0000 7f262cdd5700 1 -- 172.21.15.204:0/969938639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2628199310 con 0x7f2628101220 2024-04-02T14:34:14.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.434+0000 7f262cdd5700 1 -- 172.21.15.204:0/969938639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26281997d0 con 0x7f2628101220 2024-04-02T14:34:14.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.434+0000 7f26177fe700 1 -- 172.21.15.204:0/969938639 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f261c004830 con 0x7f2628101220 2024-04-02T14:34:14.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.434+0000 7f26177fe700 1 -- 172.21.15.204:0/969938639 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f261c030060 con 0x7f2628101220 2024-04-02T14:34:14.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.435+0000 7f262cdd5700 1 -- 172.21.15.204:0/969938639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2608005330 con 0x7f2628101220 2024-04-02T14:34:14.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.436+0000 7f26177fe700 1 -- 172.21.15.204:0/969938639 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f261c029070 con 0x7f2628101220 2024-04-02T14:34:14.439 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.436+0000 7f26177fe700 1 --2- 172.21.15.204:0/969938639 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2610039650 0x7f261003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:14.439 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.436+0000 7f26177fe700 1 -- 172.21.15.204:0/969938639 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f261c01e070 con 0x7f2628101220 2024-04-02T14:34:14.439 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.436+0000 7f2625d9b700 1 --2- 172.21.15.204:0/969938639 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2610039650 0x7f261003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:14.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.437+0000 7f2625d9b700 1 --2- 172.21.15.204:0/969938639 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2610039650 0x7f261003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2618006fd0 tx=0x7f2618006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:14.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.451+0000 7f26177fe700 1 -- 172.21.15.204:0/969938639 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f261c013070 con 0x7f2628101220 2024-04-02T14:34:14.636 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.635+0000 7f26177fe700 1 -- 172.21.15.204:0/969938639 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f261c00b6f0 con 0x7f2628101220 2024-04-02T14:34:14.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:14.740+0000 7f262cdd5700 1 -- 172.21.15.204:0/969938639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f26080058f0 con 0x7f2628101220 2024-04-02T14:34:14.742 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:14 smithi204 systemd[1]: Started Ceph mon.c for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:34:19.672 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:19.671+0000 7f26177fe700 1 -- 172.21.15.204:0/969938639 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f261c017440 con 0x7f2628101220 2024-04-02T14:34:19.673 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:34:19.673 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","modified":"2024-04-02T14:34:14.633956Z","created":"2024-04-02T14:31:53.643129Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T14:34:19.675 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:19.674+0000 7f262cdd5700 1 -- 172.21.15.204:0/969938639 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2610039650 msgr2=0x7f261003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:19.675 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:19.674+0000 7f262cdd5700 1 --2- 172.21.15.204:0/969938639 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2610039650 0x7f261003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2618006fd0 tx=0x7f2618006e40 comp rx=0 tx=0).stop 2024-04-02T14:34:19.675 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:19.674+0000 7f262cdd5700 1 -- 172.21.15.204:0/969938639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2628101220 msgr2=0x7f2628198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:19.675 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:19.674+0000 7f262cdd5700 1 --2- 172.21.15.204:0/969938639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2628101220 0x7f2628198ae0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f261c0052a0 tx=0x7f261c004cc0 comp rx=0 tx=0).stop 2024-04-02T14:34:19.676 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:19.674+0000 7f262cdd5700 1 -- 172.21.15.204:0/969938639 shutdown_connections 2024-04-02T14:34:19.676 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:19.674+0000 7f262cdd5700 1 --2- 172.21.15.204:0/969938639 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2610039650 0x7f261003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:19.676 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:19.674+0000 7f262cdd5700 1 --2- 172.21.15.204:0/969938639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2628101220 0x7f2628198ae0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:19.676 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:19.674+0000 7f262cdd5700 1 -- 172.21.15.204:0/969938639 >> 172.21.15.204:0/969938639 conn(0x7f26280fcab0 msgr2=0x7f2628102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:19.676 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:19.674+0000 7f262cdd5700 1 -- 172.21.15.204:0/969938639 shutdown_connections 2024-04-02T14:34:19.676 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:19.674+0000 7f262cdd5700 1 -- 172.21.15.204:0/969938639 wait complete. 2024-04-02T14:34:19.676 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-04-02T14:34:19.974 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: mon.a calling monitor election 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='client.? 172.21.15.204:0/969938639' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: mon.c calling monitor election 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: monmap epoch 2 2024-04-02T14:34:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: last_changed 2024-04-02T14:34:14.633956+0000 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: created 2024-04-02T14:31:53.643129+0000 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: min_mon_release 19 (squid) 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: election_strategy: 1 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: fsmap 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: mgrmap e13: x(active, since 60s) 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: overall HEALTH_OK 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:19.976 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:19.977 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:20.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: mon.a calling monitor election 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='client.? 172.21.15.204:0/969938639' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: mon.c calling monitor election 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: monmap epoch 2 2024-04-02T14:34:20.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: last_changed 2024-04-02T14:34:14.633956+0000 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: created 2024-04-02T14:31:53.643129+0000 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: min_mon_release 19 (squid) 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: election_strategy: 1 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: fsmap 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: mgrmap e13: x(active, since 60s) 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: overall HEALTH_OK 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:20.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:20.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:20.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:20 smithi204 ceph-mon[35948]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:20.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:20 smithi204 ceph-mon[35948]: Deploying daemon mon.b on smithi113 2024-04-02T14:34:20.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:20 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:21.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:20 smithi062 ceph-mon[32116]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:21.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:20 smithi062 ceph-mon[32116]: Deploying daemon mon.b on smithi113 2024-04-02T14:34:21.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:20 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:21.260 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T14:34:21.261 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T14:34:21.498 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:34:22.094 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:34:21 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:34:21.635+0000 7fc59d200700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-02T14:34:22.944 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.941+0000 7f5f02793700 1 -- 172.21.15.204:0/179358160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc102330 msgr2=0x7f5efc102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.941+0000 7f5f02793700 1 --2- 172.21.15.204:0/179358160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc102330 0x7f5efc102710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009910 tx=0x7f5eec009c20 comp rx=0 tx=0).stop 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.941+0000 7f5f02793700 1 -- 172.21.15.204:0/179358160 shutdown_connections 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.941+0000 7f5f02793700 1 --2- 172.21.15.204:0/179358160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc102330 0x7f5efc102710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.941+0000 7f5f02793700 1 -- 172.21.15.204:0/179358160 >> 172.21.15.204:0/179358160 conn(0x7f5efc0fdbc0 msgr2=0x7f5efc0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.942+0000 7f5f02793700 1 -- 172.21.15.204:0/179358160 shutdown_connections 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.942+0000 7f5f02793700 1 -- 172.21.15.204:0/179358160 wait complete. 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.942+0000 7f5f02793700 1 Processor -- start 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.942+0000 7f5f02793700 1 -- start start 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.942+0000 7f5f02793700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5efc102330 0x7f5efc0748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.942+0000 7f5f02793700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc074e30 0x7f5efc0752b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.943+0000 7f5f02793700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc10c840 con 0x7f5efc074e30 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.943+0000 7f5f02793700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc10c6c0 con 0x7f5efc102330 2024-04-02T14:34:22.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.943+0000 7f5efbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5efc102330 0x7f5efc0748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:22.946 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.943+0000 7f5efbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5efc102330 0x7f5efc0748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38550/0 (socket says 172.21.15.204:38550) 2024-04-02T14:34:22.946 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.943+0000 7f5efbfff700 1 -- 172.21.15.204:0/1496009718 learned_addr learned my addr 172.21.15.204:0/1496009718 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T14:34:22.946 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.943+0000 7f5efbfff700 1 -- 172.21.15.204:0/1496009718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5efc102330 msgr2=0x7f5efc0748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-02T14:34:22.946 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.943+0000 7f5efbfff700 1 -- 172.21.15.204:0/1496009718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5efc102330 msgr2=0x7f5efc0748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T14:34:22.946 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.943+0000 7f5efbfff700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5efc102330 0x7f5efc0748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T14:34:22.946 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.943+0000 7f5efbfff700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5efc102330 0x7f5efc0748f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T14:34:22.946 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.943+0000 7f5efb7fe700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc074e30 0x7f5efc0752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:22.951 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.944+0000 7f5efb7fe700 1 -- 172.21.15.204:0/1496009718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5efc102330 msgr2=0x7f5efc0748f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.944+0000 7f5efb7fe700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5efc102330 0x7f5efc0748f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.944+0000 7f5efb7fe700 1 -- 172.21.15.204:0/1496009718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc076810 con 0x7f5efc074e30 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.944+0000 7f5efb7fe700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc074e30 0x7f5efc0752b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000b4d0 tx=0x7f5ef000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.944+0000 7f5ef97fa700 1 -- 172.21.15.204:0/1496009718 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5ef0011630 con 0x7f5efc074e30 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.944+0000 7f5f02793700 1 -- 172.21.15.204:0/1496009718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5efc1a4150 con 0x7f5efc074e30 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.945+0000 7f5f02793700 1 -- 172.21.15.204:0/1496009718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5efc1a4690 con 0x7f5efc074e30 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.945+0000 7f5ef97fa700 1 -- 172.21.15.204:0/1496009718 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5ef0011cd0 con 0x7f5efc074e30 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.945+0000 7f5ef97fa700 1 -- 172.21.15.204:0/1496009718 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5ef00193b0 con 0x7f5efc074e30 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.945+0000 7f5f02793700 1 -- 172.21.15.204:0/1496009718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee8005330 con 0x7f5efc074e30 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.946+0000 7f5ef97fa700 1 -- 172.21.15.204:0/1496009718 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5ef0018020 con 0x7f5efc074e30 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.946+0000 7f5ef97fa700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5ee40393f0 0x7f5ee403b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.946+0000 7f5ef97fa700 1 -- 172.21.15.204:0/1496009718 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5ef004c580 con 0x7f5efc074e30 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.947+0000 7f5efbfff700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5ee40393f0 0x7f5ee403b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:22.952 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.948+0000 7f5efbfff700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5ee40393f0 0x7f5ee403b8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00b5c0 tx=0x7f5eec009e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:22.965 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:22.963+0000 7f5ef97fa700 1 -- 172.21.15.204:0/1496009718 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5ef001f020 con 0x7f5efc074e30 2024-04-02T14:34:23.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:22 smithi204 ceph-mon[35948]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:23.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:22 smithi062 ceph-mon[32116]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:23.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.242+0000 7f5f02793700 1 -- 172.21.15.204:0/1496009718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5ee80058f0 con 0x7f5efc074e30 2024-04-02T14:34:23.244 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.243+0000 7f5ef97fa700 1 -- 172.21.15.204:0/1496009718 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f5ef0010380 con 0x7f5efc074e30 2024-04-02T14:34:23.245 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:34:23.245 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","modified":"2024-04-02T14:34:14.633956Z","created":"2024-04-02T14:31:53.643129Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T14:34:23.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.245+0000 7f5f02793700 1 -- 172.21.15.204:0/1496009718 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5ee40393f0 msgr2=0x7f5ee403b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:23.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.245+0000 7f5f02793700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5ee40393f0 0x7f5ee403b8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00b5c0 tx=0x7f5eec009e60 comp rx=0 tx=0).stop 2024-04-02T14:34:23.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.245+0000 7f5f02793700 1 -- 172.21.15.204:0/1496009718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc074e30 msgr2=0x7f5efc0752b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:23.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.245+0000 7f5f02793700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc074e30 0x7f5efc0752b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000b4d0 tx=0x7f5ef000b890 comp rx=0 tx=0).stop 2024-04-02T14:34:23.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.245+0000 7f5f02793700 1 -- 172.21.15.204:0/1496009718 shutdown_connections 2024-04-02T14:34:23.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.245+0000 7f5f02793700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5ee40393f0 0x7f5ee403b8b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:23.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.245+0000 7f5f02793700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5efc102330 0x7f5efc0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:23.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.245+0000 7f5f02793700 1 --2- 172.21.15.204:0/1496009718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc074e30 0x7f5efc0752b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:23.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.245+0000 7f5f02793700 1 -- 172.21.15.204:0/1496009718 >> 172.21.15.204:0/1496009718 conn(0x7f5efc0fdbc0 msgr2=0x7f5efc103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:23.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.245+0000 7f5f02793700 1 -- 172.21.15.204:0/1496009718 shutdown_connections 2024-04-02T14:34:23.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:23.245+0000 7f5f02793700 1 -- 172.21.15.204:0/1496009718 wait complete. 2024-04-02T14:34:23.248 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-04-02T14:34:24.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:23 smithi113 systemd[1]: Started Ceph mon.b for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:34:24.797 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T14:34:24.798 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T14:34:25.038 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:34:29.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:29.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: mon.c calling monitor election 2024-04-02T14:34:29.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:34:29.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: mon.a calling monitor election 2024-04-02T14:34:29.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:29.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:29.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:29.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T14:34:29.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: monmap epoch 3 2024-04-02T14:34:29.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:34:29.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: last_changed 2024-04-02T14:34:23.710264+0000 2024-04-02T14:34:29.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: created 2024-04-02T14:31:53.643129+0000 2024-04-02T14:34:29.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: min_mon_release 19 (squid) 2024-04-02T14:34:29.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: election_strategy: 1 2024-04-02T14:34:29.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-02T14:34:29.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-02T14:34:29.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: 2: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2024-04-02T14:34:29.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: fsmap 2024-04-02T14:34:29.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T14:34:29.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: mgrmap e13: x(active, since 69s) 2024-04-02T14:34:29.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: overall HEALTH_OK 2024-04-02T14:34:29.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:29.099 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:28 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:29.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:29.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: mon.c calling monitor election 2024-04-02T14:34:29.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:34:29.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: mon.a calling monitor election 2024-04-02T14:34:29.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:29.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:29.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: monmap epoch 3 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: last_changed 2024-04-02T14:34:23.710264+0000 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: created 2024-04-02T14:31:53.643129+0000 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: min_mon_release 19 (squid) 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: election_strategy: 1 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: 2: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2024-04-02T14:34:29.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: fsmap 2024-04-02T14:34:29.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T14:34:29.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: mgrmap e13: x(active, since 69s) 2024-04-02T14:34:29.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: overall HEALTH_OK 2024-04-02T14:34:29.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:29.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:28 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:29.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.454+0000 7f7d668bb700 1 -- 172.21.15.204:0/2440734569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d60103280 msgr2=0x7f7d4c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:29.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.454+0000 7f7d668bb700 1 --2- 172.21.15.204:0/2440734569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d60103280 0x7f7d4c005610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d50005fa0 tx=0x7f7d5000ffc0 comp rx=0 tx=0).stop 2024-04-02T14:34:29.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.454+0000 7f7d668bb700 1 -- 172.21.15.204:0/2440734569 shutdown_connections 2024-04-02T14:34:29.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.454+0000 7f7d668bb700 1 --2- 172.21.15.204:0/2440734569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d60103280 0x7f7d4c005610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:29.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.454+0000 7f7d668bb700 1 -- 172.21.15.204:0/2440734569 >> 172.21.15.204:0/2440734569 conn(0x7f7d600fca90 msgr2=0x7f7d600feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:29.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.454+0000 7f7d668bb700 1 -- 172.21.15.204:0/2440734569 shutdown_connections 2024-04-02T14:34:29.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.454+0000 7f7d668bb700 1 -- 172.21.15.204:0/2440734569 wait complete. 2024-04-02T14:34:29.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d668bb700 1 Processor -- start 2024-04-02T14:34:29.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d668bb700 1 -- start start 2024-04-02T14:34:29.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d668bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d6019a220 0x7f7d6019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d668bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6019ab40 0x7f7d6019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d668bb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d6019f030 0x7f7d6019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d668bb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6010a260 con 0x7f7d6019ab40 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d668bb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6010a0e0 con 0x7f7d6019f030 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d668bb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6010a3e0 con 0x7f7d6019a220 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d5ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d6019a220 0x7f7d6019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d5ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d6019a220 0x7f7d6019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36676/0 (socket says 172.21.15.204:36676) 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d5ffff700 1 -- 172.21.15.204:0/2292912800 learned_addr learned my addr 172.21.15.204:0/2292912800 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d5ffff700 1 -- 172.21.15.204:0/2292912800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d6019a220 msgr2=0x7f7d6019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d5ffff700 1 -- 172.21.15.204:0/2292912800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d6019a220 msgr2=0x7f7d6019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d5ffff700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d6019a220 0x7f7d6019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T14:34:29.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d5ffff700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d6019a220 0x7f7d6019a600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d64e58700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d6019f030 0x7f7d6019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d64e58700 1 -- 172.21.15.204:0/2292912800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d6019f030 msgr2=0x7f7d6019f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d64e58700 1 -- 172.21.15.204:0/2292912800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d6019f030 msgr2=0x7f7d6019f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d64e58700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d6019f030 0x7f7d6019f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d5f7fe700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6019ab40 0x7f7d6019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.455+0000 7f7d64e58700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d6019f030 0x7f7d6019f4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.456+0000 7f7d5f7fe700 1 -- 172.21.15.204:0/2292912800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d6019a220 msgr2=0x7f7d6019a600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.456+0000 7f7d5f7fe700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d6019a220 0x7f7d6019a600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.456+0000 7f7d5f7fe700 1 -- 172.21.15.204:0/2292912800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d6019f030 msgr2=0x7f7d6019f4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.456+0000 7f7d5f7fe700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d6019f030 0x7f7d6019f4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.456+0000 7f7d5f7fe700 1 -- 172.21.15.204:0/2292912800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d601a7070 con 0x7f7d6019ab40 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.457+0000 7f7d5f7fe700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6019ab40 0x7f7d6019eaf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800b550 tx=0x7f7d4800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.457+0000 7f7d5d7fa700 1 -- 172.21.15.204:0/2292912800 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48004230 con 0x7f7d6019ab40 2024-04-02T14:34:29.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.457+0000 7f7d668bb700 1 -- 172.21.15.204:0/2292912800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d601a7360 con 0x7f7d6019ab40 2024-04-02T14:34:29.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.457+0000 7f7d668bb700 1 -- 172.21.15.204:0/2292912800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d601a78f0 con 0x7f7d6019ab40 2024-04-02T14:34:29.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.457+0000 7f7d5d7fa700 1 -- 172.21.15.204:0/2292912800 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7d480043d0 con 0x7f7d6019ab40 2024-04-02T14:34:29.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.457+0000 7f7d5d7fa700 1 -- 172.21.15.204:0/2292912800 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48011bf0 con 0x7f7d6019ab40 2024-04-02T14:34:29.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.458+0000 7f7d668bb700 1 -- 172.21.15.204:0/2292912800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d60063c20 con 0x7f7d6019ab40 2024-04-02T14:34:29.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.458+0000 7f7d5d7fa700 1 -- 172.21.15.204:0/2292912800 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7d480036a0 con 0x7f7d6019ab40 2024-04-02T14:34:29.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.459+0000 7f7d5d7fa700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7d40039ae0 0x7f7d4003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:29.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.459+0000 7f7d5d7fa700 1 -- 172.21.15.204:0/2292912800 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7d4804c190 con 0x7f7d6019ab40 2024-04-02T14:34:29.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.459+0000 7f7d5ffff700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7d40039ae0 0x7f7d4003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:29.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.460+0000 7f7d5ffff700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7d40039ae0 0x7f7d4003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7d50005ed0 tx=0x7f7d50005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:29.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.475+0000 7f7d5d7fa700 1 -- 172.21.15.204:0/2292912800 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7d4801f070 con 0x7f7d6019ab40 2024-04-02T14:34:29.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.779+0000 7f7d668bb700 1 -- 172.21.15.204:0/2292912800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7d6002dd30 con 0x7f7d6019ab40 2024-04-02T14:34:29.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.780+0000 7f7d5d7fa700 1 -- 172.21.15.204:0/2292912800 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f7d480202a0 con 0x7f7d6019ab40 2024-04-02T14:34:29.781 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T14:34:29.782 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","modified":"2024-04-02T14:34:23.710264Z","created":"2024-04-02T14:31:53.643129Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T14:34:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 -- 172.21.15.204:0/2292912800 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7d40039ae0 msgr2=0x7f7d4003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7d40039ae0 0x7f7d4003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7d50005ed0 tx=0x7f7d50005da0 comp rx=0 tx=0).stop 2024-04-02T14:34:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 -- 172.21.15.204:0/2292912800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6019ab40 msgr2=0x7f7d6019eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6019ab40 0x7f7d6019eaf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800b550 tx=0x7f7d4800b910 comp rx=0 tx=0).stop 2024-04-02T14:34:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 -- 172.21.15.204:0/2292912800 shutdown_connections 2024-04-02T14:34:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7d40039ae0 0x7f7d4003bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d6019f030 0x7f7d6019f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d6019a220 0x7f7d6019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 --2- 172.21.15.204:0/2292912800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6019ab40 0x7f7d6019eaf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 -- 172.21.15.204:0/2292912800 >> 172.21.15.204:0/2292912800 conn(0x7f7d600fca90 msgr2=0x7f7d60101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 -- 172.21.15.204:0/2292912800 shutdown_connections 2024-04-02T14:34:29.785 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:29.782+0000 7f7d668bb700 1 -- 172.21.15.204:0/2292912800 wait complete. 2024-04-02T14:34:29.785 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-04-02T14:34:30.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:29 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:30.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:29 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:30.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:29 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:30.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:29 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:30.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:29 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:30.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:29 smithi062 ceph-mon[32116]: from='client.? 172.21.15.204:0/2292912800' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T14:34:30.235 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:29 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:30.235 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:29 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:30.235 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:29 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:30.235 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:29 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:30.235 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:29 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:30.235 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:29 smithi204 ceph-mon[35948]: from='client.? 172.21.15.204:0/2292912800' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T14:34:30.529 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-02T14:34:30.529 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-04-02T14:34:30.990 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:34:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: mon.c calling monitor election 2024-04-02T14:34:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:34:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: mon.a calling monitor election 2024-04-02T14:34:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:34:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:31.126 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:31.126 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:31.126 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:31.126 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:31.126 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:31.126 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T14:34:31.126 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: monmap epoch 3 2024-04-02T14:34:31.126 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:34:31.126 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: last_changed 2024-04-02T14:34:23.710264+0000 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: created 2024-04-02T14:31:53.643129+0000 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: min_mon_release 19 (squid) 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: election_strategy: 1 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: 2: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: fsmap 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: mgrmap e13: x(active, since 69s) 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: overall HEALTH_OK 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:31.127 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:31.128 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:31.128 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:31.128 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:30 smithi113 ceph-mon[36135]: from='client.? 172.21.15.204:0/2292912800' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T14:34:31.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: mon.b calling monitor election 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: mon.b calling monitor election 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: mon.c calling monitor election 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: mon.a calling monitor election 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: monmap epoch 3 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: last_changed 2024-04-02T14:34:23.710264+0000 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: created 2024-04-02T14:31:53.643129+0000 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: min_mon_release 19 (squid) 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: election_strategy: 1 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-02T14:34:31.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: 2: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2024-04-02T14:34:31.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: fsmap 2024-04-02T14:34:31.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T14:34:31.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: mgrmap e13: x(active, since 71s) 2024-04-02T14:34:31.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: overall HEALTH_OK 2024-04-02T14:34:31.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:31.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:31.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:31.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:31.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-02T14:34:31.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: Updating smithi113:/etc/ceph/ceph.conf 2024-04-02T14:34:31.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:31 smithi062 ceph-mon[32116]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-02T14:34:31.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: mon.b calling monitor election 2024-04-02T14:34:31.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: mon.b calling monitor election 2024-04-02T14:34:31.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: mon.c calling monitor election 2024-04-02T14:34:31.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: mon.a calling monitor election 2024-04-02T14:34:31.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T14:34:31.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: monmap epoch 3 2024-04-02T14:34:31.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:34:31.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: last_changed 2024-04-02T14:34:23.710264+0000 2024-04-02T14:34:31.624 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: created 2024-04-02T14:31:53.643129+0000 2024-04-02T14:34:31.624 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: min_mon_release 19 (squid) 2024-04-02T14:34:31.624 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: election_strategy: 1 2024-04-02T14:34:31.624 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-02T14:34:31.624 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-02T14:34:31.624 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: 2: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2024-04-02T14:34:31.625 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: fsmap 2024-04-02T14:34:31.625 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T14:34:31.625 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: mgrmap e13: x(active, since 71s) 2024-04-02T14:34:31.625 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: overall HEALTH_OK 2024-04-02T14:34:31.625 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:31.626 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:31.626 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:31.626 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:31.626 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-02T14:34:31.626 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: Updating smithi113:/etc/ceph/ceph.conf 2024-04-02T14:34:31.626 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:31 smithi113 ceph-mon[36135]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-02T14:34:31.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: mon.b calling monitor election 2024-04-02T14:34:31.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: mon.b calling monitor election 2024-04-02T14:34:31.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: mon.c calling monitor election 2024-04-02T14:34:31.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: mon.a calling monitor election 2024-04-02T14:34:31.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T14:34:31.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: monmap epoch 3 2024-04-02T14:34:31.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:34:31.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: last_changed 2024-04-02T14:34:23.710264+0000 2024-04-02T14:34:31.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: created 2024-04-02T14:31:53.643129+0000 2024-04-02T14:34:31.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: min_mon_release 19 (squid) 2024-04-02T14:34:31.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: election_strategy: 1 2024-04-02T14:34:31.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-02T14:34:31.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-02T14:34:31.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: 2: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2024-04-02T14:34:31.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: fsmap 2024-04-02T14:34:31.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T14:34:31.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: mgrmap e13: x(active, since 71s) 2024-04-02T14:34:31.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: overall HEALTH_OK 2024-04-02T14:34:31.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:31.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:31.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:31.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:31.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-02T14:34:31.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: Updating smithi113:/etc/ceph/ceph.conf 2024-04-02T14:34:31.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:31 smithi204 ceph-mon[35948]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-02T14:34:32.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: Updating smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:34:32.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: Updating smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:34:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: Updating smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:34:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.849 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: Updating smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:34:32.849 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: Updating smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:34:32.849 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:32.849 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.849 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.849 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.849 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.849 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: Updating smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:34:32.849 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:32.849 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.850 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.850 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: Updating smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: Updating smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: Updating smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:32.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:33.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.361+0000 7f107d800700 1 -- 172.21.15.62:0/1647176493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1078100f70 msgr2=0x7f1078101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:33.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.361+0000 7f107d800700 1 --2- 172.21.15.62:0/1647176493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1078100f70 0x7f1078101350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1068009910 tx=0x7f1068009c20 comp rx=0 tx=0).stop 2024-04-02T14:34:33.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.361+0000 7f107d800700 1 -- 172.21.15.62:0/1647176493 shutdown_connections 2024-04-02T14:34:33.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.361+0000 7f107d800700 1 --2- 172.21.15.62:0/1647176493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1078100f70 0x7f1078101350 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:33.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.361+0000 7f107d800700 1 -- 172.21.15.62:0/1647176493 >> 172.21.15.62:0/1647176493 conn(0x7f1078077b30 msgr2=0x7f10780fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:33.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.361+0000 7f107d800700 1 -- 172.21.15.62:0/1647176493 shutdown_connections 2024-04-02T14:34:33.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.361+0000 7f107d800700 1 -- 172.21.15.62:0/1647176493 wait complete. 2024-04-02T14:34:33.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.362+0000 7f107d800700 1 Processor -- start 2024-04-02T14:34:33.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.362+0000 7f107d800700 1 -- start start 2024-04-02T14:34:33.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.363+0000 7f107d800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1078100f70 0x7f107819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:33.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.363+0000 7f107d800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f107819e9a0 0x7f10781a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:33.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.363+0000 7f107d800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10781a3270 0x7f10781a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:33.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.363+0000 7f107d800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1078112db0 con 0x7f1078100f70 2024-04-02T14:34:33.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.363+0000 7f107d800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1078112c30 con 0x7f107819e9a0 2024-04-02T14:34:33.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.363+0000 7f107d800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1078112f30 con 0x7f10781a3270 2024-04-02T14:34:33.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.363+0000 7f1076a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1078100f70 0x7f107819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:33.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.363+0000 7f1076a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1078100f70 0x7f107819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56400/0 (socket says 172.21.15.62:56400) 2024-04-02T14:34:33.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.363+0000 7f1076a00700 1 -- 172.21.15.62:0/797296388 learned_addr learned my addr 172.21.15.62:0/797296388 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:34:33.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.363+0000 7f1077400700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10781a3270 0x7f10781a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:33.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.363+0000 7f1076000700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f107819e9a0 0x7f10781a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:33.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.364+0000 7f1076a00700 1 -- 172.21.15.62:0/797296388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10781a3270 msgr2=0x7f10781a3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:33.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.364+0000 7f1076a00700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10781a3270 0x7f10781a3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:33.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.364+0000 7f1076a00700 1 -- 172.21.15.62:0/797296388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f107819e9a0 msgr2=0x7f10781a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:33.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.364+0000 7f1076a00700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f107819e9a0 0x7f10781a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:33.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.364+0000 7f1076a00700 1 -- 172.21.15.62:0/797296388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10781aafe0 con 0x7f1078100f70 2024-04-02T14:34:33.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.364+0000 7f1076a00700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1078100f70 0x7f107819e460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1068005320 tx=0x7f10680048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:33.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.365+0000 7f106f400700 1 -- 172.21.15.62:0/797296388 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068027070 con 0x7f1078100f70 2024-04-02T14:34:33.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.365+0000 7f106f400700 1 -- 172.21.15.62:0/797296388 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f106800bc50 con 0x7f1078100f70 2024-04-02T14:34:33.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.365+0000 7f106f400700 1 -- 172.21.15.62:0/797296388 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068022de0 con 0x7f1078100f70 2024-04-02T14:34:33.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.365+0000 7f107d800700 1 -- 172.21.15.62:0/797296388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10781ab1b0 con 0x7f1078100f70 2024-04-02T14:34:33.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.365+0000 7f107d800700 1 -- 172.21.15.62:0/797296388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10781ab640 con 0x7f1078100f70 2024-04-02T14:34:33.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.366+0000 7f106f400700 1 -- 172.21.15.62:0/797296388 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f106802a070 con 0x7f1078100f70 2024-04-02T14:34:33.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.366+0000 7f107d800700 1 -- 172.21.15.62:0/797296388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107806a2c0 con 0x7f1078100f70 2024-04-02T14:34:33.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.367+0000 7f106f400700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f105c039ae0 0x7f105c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:33.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.367+0000 7f106f400700 1 -- 172.21.15.62:0/797296388 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f106804d8e0 con 0x7f1078100f70 2024-04-02T14:34:33.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.367+0000 7f1076000700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f105c039ae0 0x7f105c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:33.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.368+0000 7f1076000700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f105c039ae0 0x7f105c03bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1064009ad0 tx=0x7f1064009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.373+0000 7f106f400700 1 -- 172.21.15.62:0/797296388 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1068013070 con 0x7f1078100f70 2024-04-02T14:34:33.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.616+0000 7f107d800700 1 -- 172.21.15.62:0/797296388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f10781ab8d0 con 0x7f1078100f70 2024-04-02T14:34:33.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.619+0000 7f106f400700 1 -- 172.21.15.62:0/797296388 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f1068031020 con 0x7f1078100f70 2024-04-02T14:34:33.622 INFO:teuthology.orchestra.run.smithi062.stdout:# minimal ceph.conf for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:34:33.622 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-04-02T14:34:33.622 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:34:33.622 INFO:teuthology.orchestra.run.smithi062.stdout: mon_host = [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.622+0000 7f107d800700 1 -- 172.21.15.62:0/797296388 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f105c039ae0 msgr2=0x7f105c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.622+0000 7f107d800700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f105c039ae0 0x7f105c03bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1064009ad0 tx=0x7f1064009210 comp rx=0 tx=0).stop 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.622+0000 7f107d800700 1 -- 172.21.15.62:0/797296388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1078100f70 msgr2=0x7f107819e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.622+0000 7f107d800700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1078100f70 0x7f107819e460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1068005320 tx=0x7f10680048c0 comp rx=0 tx=0).stop 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.622+0000 7f107d800700 1 -- 172.21.15.62:0/797296388 shutdown_connections 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.622+0000 7f107d800700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f105c039ae0 0x7f105c03bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.622+0000 7f107d800700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10781a3270 0x7f10781a3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.622+0000 7f107d800700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1078100f70 0x7f107819e460 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.622+0000 7f107d800700 1 --2- 172.21.15.62:0/797296388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f107819e9a0 0x7f10781a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.622+0000 7f107d800700 1 -- 172.21.15.62:0/797296388 >> 172.21.15.62:0/797296388 conn(0x7f1078077b30 msgr2=0x7f10780fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.623+0000 7f107d800700 1 -- 172.21.15.62:0/797296388 shutdown_connections 2024-04-02T14:34:33.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:33.623+0000 7f107d800700 1 -- 172.21.15.62:0/797296388 wait complete. 2024-04-02T14:34:33.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:33.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:33.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:33.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:33.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:33 smithi062 ceph-mon[32116]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T14:34:33.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:33.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T14:34:33.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:33.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:33 smithi062 ceph-mon[32116]: Reconfiguring daemon mon.a on smithi062 2024-04-02T14:34:33.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:33.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:33.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:33.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:33.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:33 smithi113 ceph-mon[36135]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T14:34:33.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:33.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T14:34:33.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:33.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:33 smithi113 ceph-mon[36135]: Reconfiguring daemon mon.a on smithi062 2024-04-02T14:34:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:33 smithi204 ceph-mon[35948]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T14:34:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T14:34:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:33 smithi204 ceph-mon[35948]: Reconfiguring daemon mon.a on smithi062 2024-04-02T14:34:34.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:34 smithi113 ceph-mon[36135]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:34.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:34 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/797296388' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:34.930 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:34 smithi062 ceph-mon[32116]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:34.930 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:34 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/797296388' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:35.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:34 smithi204 ceph-mon[35948]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:35.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:34 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/797296388' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:35.351 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-02T14:34:35.351 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:34:35.351 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T14:34:35.433 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:34:35.434 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:34:35.571 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-02T14:34:35.572 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T14:34:35.602 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-02T14:34:35.602 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:34:35.669 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T14:34:35.669 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T14:34:35.705 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T14:34:35.705 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:34:35.776 INFO:tasks.cephadm:Adding mgr.x on smithi062 2024-04-02T14:34:35.776 INFO:tasks.cephadm:Adding mgr.y on smithi113 2024-04-02T14:34:35.777 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-04-02T14:34:35.777 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch apply mgr '3;smithi062=x;smithi113=y;smithi204=z' 2024-04-02T14:34:36.052 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:34:36.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:36 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:36.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:36 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:36.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:36 smithi062 ceph-mon[32116]: Reconfiguring mon.b (monmap changed)... 2024-04-02T14:34:36.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:36 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:36.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:36 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T14:34:36.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:36 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:36.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:36 smithi062 ceph-mon[32116]: Reconfiguring daemon mon.b on smithi113 2024-04-02T14:34:36.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:36 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:36.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:36 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:36.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:36 smithi113 ceph-mon[36135]: Reconfiguring mon.b (monmap changed)... 2024-04-02T14:34:36.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:36 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:36.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:36 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T14:34:36.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:36 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:36.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:36 smithi113 ceph-mon[36135]: Reconfiguring daemon mon.b on smithi113 2024-04-02T14:34:36.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:36 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:36.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:36 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:36.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:36 smithi204 ceph-mon[35948]: Reconfiguring mon.b (monmap changed)... 2024-04-02T14:34:36.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:36 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:36.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:36 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T14:34:36.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:36 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:36.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:36 smithi204 ceph-mon[35948]: Reconfiguring daemon mon.b on smithi113 2024-04-02T14:34:37.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:37 smithi204 ceph-mon[35948]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:37.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:37.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:37.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:37.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T14:34:37.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:37.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:37 smithi062 ceph-mon[32116]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:37.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:37.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:37.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:37.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T14:34:37.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:37.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:37 smithi113 ceph-mon[36135]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:37.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:37.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:37.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T14:34:37.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T14:34:37.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:37.985 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.970+0000 7f87f18ff700 1 -- 172.21.15.204:0/3122223870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87ec101050 msgr2=0x7f87ec101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:37.985 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.970+0000 7f87f18ff700 1 --2- 172.21.15.204:0/3122223870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87ec101050 0x7f87ec101470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009910 tx=0x7f87e0009c20 comp rx=0 tx=0).stop 2024-04-02T14:34:37.985 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.970+0000 7f87f18ff700 1 -- 172.21.15.204:0/3122223870 shutdown_connections 2024-04-02T14:34:37.985 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.970+0000 7f87f18ff700 1 --2- 172.21.15.204:0/3122223870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87ec101050 0x7f87ec101470 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:37.985 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.970+0000 7f87f18ff700 1 -- 172.21.15.204:0/3122223870 >> 172.21.15.204:0/3122223870 conn(0x7f87ec0fc5f0 msgr2=0x7f87ec0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:37.985 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.970+0000 7f87f18ff700 1 -- 172.21.15.204:0/3122223870 shutdown_connections 2024-04-02T14:34:37.985 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.970+0000 7f87f18ff700 1 -- 172.21.15.204:0/3122223870 wait complete. 2024-04-02T14:34:37.985 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.970+0000 7f87f18ff700 1 Processor -- start 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87f18ff700 1 -- start start 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87f18ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87ec101050 0x7f87ec197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87f18ff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87ec197f40 0x7f87ec19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87f18ff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87ec1983c0 0x7f87ec19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87f18ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec107220 con 0x7f87ec101050 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87f18ff700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1070a0 con 0x7f87ec197f40 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87f18ff700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1073a0 con 0x7f87ec1983c0 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87eb7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87ec1983c0 0x7f87ec19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87eb7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87ec1983c0 0x7f87ec19f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43688/0 (socket says 172.21.15.204:43688) 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87eb7fe700 1 -- 172.21.15.204:0/1600225133 learned_addr learned my addr 172.21.15.204:0/1600225133 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87ea7fc700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87ec197f40 0x7f87ec19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87eb7fe700 1 -- 172.21.15.204:0/1600225133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87ec197f40 msgr2=0x7f87ec19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:37.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87eb7fe700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87ec197f40 0x7f87ec19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87eb7fe700 1 -- 172.21.15.204:0/1600225133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87ec101050 msgr2=0x7f87ec197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87eb7fe700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87ec101050 0x7f87ec197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87eb7fe700 1 -- 172.21.15.204:0/1600225133 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ec19fe10 con 0x7f87ec1983c0 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87ea7fc700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87ec197f40 0x7f87ec19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.971+0000 7f87eb7fe700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87ec1983c0 0x7f87ec19f540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f87e400ba50 tx=0x7f87e400be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.972+0000 7f87f08fd700 1 -- 172.21.15.204:0/1600225133 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4019070 con 0x7f87ec1983c0 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.972+0000 7f87f18ff700 1 -- 172.21.15.204:0/1600225133 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ec1a0100 con 0x7f87ec1983c0 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.972+0000 7f87f08fd700 1 -- 172.21.15.204:0/1600225133 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f87e4014440 con 0x7f87ec1983c0 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.972+0000 7f87f08fd700 1 -- 172.21.15.204:0/1600225133 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4007590 con 0x7f87ec1983c0 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.972+0000 7f87f18ff700 1 -- 172.21.15.204:0/1600225133 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ec1a0670 con 0x7f87ec1983c0 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.972+0000 7f87f08fd700 1 -- 172.21.15.204:0/1600225133 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f87e401c070 con 0x7f87ec1983c0 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.973+0000 7f87f18ff700 1 -- 172.21.15.204:0/1600225133 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87ec063c20 con 0x7f87ec1983c0 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.973+0000 7f87f08fd700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87d4039ae0 0x7f87d403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:37.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.973+0000 7f87f08fd700 1 -- 172.21.15.204:0/1600225133 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f87e404c080 con 0x7f87ec1983c0 2024-04-02T14:34:37.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.973+0000 7f87eaffd700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87d4039ae0 0x7f87d403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:37.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.974+0000 7f87eaffd700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87d4039ae0 0x7f87d403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87e0005e00 tx=0x7f87e0005380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:37.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:37.978+0000 7f87f08fd700 1 -- 172.21.15.204:0/1600225133 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f87e4014a10 con 0x7f87ec1983c0 2024-04-02T14:34:38.191 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:38 smithi204 ceph-mon[35948]: Reconfiguring mon.c (monmap changed)... 2024-04-02T14:34:38.192 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:38 smithi204 ceph-mon[35948]: Reconfiguring daemon mon.c on smithi204 2024-04-02T14:34:38.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.192+0000 7f87f18ff700 1 -- 172.21.15.204:0/1600225133 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi113=y;smithi204=z", "target": ["mon-mgr", ""]}) -- 0x7f87ec062ac0 con 0x7f87d4039ae0 2024-04-02T14:34:38.207 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.206+0000 7f87f08fd700 1 -- 172.21.15.204:0/1600225133 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f87ec062ac0 con 0x7f87d4039ae0 2024-04-02T14:34:38.207 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-04-02T14:34:38.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 -- 172.21.15.204:0/1600225133 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87d4039ae0 msgr2=0x7f87d403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:38.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87d4039ae0 0x7f87d403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87e0005e00 tx=0x7f87e0005380 comp rx=0 tx=0).stop 2024-04-02T14:34:38.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 -- 172.21.15.204:0/1600225133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87ec1983c0 msgr2=0x7f87ec19f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:38.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87ec1983c0 0x7f87ec19f540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f87e400ba50 tx=0x7f87e400be10 comp rx=0 tx=0).stop 2024-04-02T14:34:38.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 -- 172.21.15.204:0/1600225133 shutdown_connections 2024-04-02T14:34:38.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87d4039ae0 0x7f87d403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:38.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87ec1983c0 0x7f87ec19f540 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:38.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87ec101050 0x7f87ec197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:38.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 --2- 172.21.15.204:0/1600225133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87ec197f40 0x7f87ec19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:38.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 -- 172.21.15.204:0/1600225133 >> 172.21.15.204:0/1600225133 conn(0x7f87ec0fc5f0 msgr2=0x7f87ec06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:38.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 -- 172.21.15.204:0/1600225133 shutdown_connections 2024-04-02T14:34:38.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:34:38.208+0000 7f87f18ff700 1 -- 172.21.15.204:0/1600225133 wait complete. 2024-04-02T14:34:38.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:38 smithi113 ceph-mon[36135]: Reconfiguring mon.c (monmap changed)... 2024-04-02T14:34:38.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:38 smithi113 ceph-mon[36135]: Reconfiguring daemon mon.c on smithi204 2024-04-02T14:34:38.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:38 smithi062 ceph-mon[32116]: Reconfiguring mon.c (monmap changed)... 2024-04-02T14:34:38.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:38 smithi062 ceph-mon[32116]: Reconfiguring daemon mon.c on smithi204 2024-04-02T14:34:39.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:39.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:39.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:39.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:39.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T14:34:39.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T14:34:39.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T14:34:39.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:39.125 DEBUG:teuthology.orchestra.run.smithi113:mgr.y> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.y.service 2024-04-02T14:34:39.127 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.z.service 2024-04-02T14:34:39.129 INFO:tasks.cephadm:Deploying OSDs... 2024-04-02T14:34:39.129 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:34:39.130 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T14:34:39.188 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T14:34:39.188 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-04-02T14:34:39.247 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T14:34:39.247 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:39.247 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-04-02T14:34:39.247 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:39.248 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:39.248 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-02 14:33:32.497920785 +0000 2024-04-02T14:34:39.248 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-02 14:27:55.782564679 +0000 2024-04-02T14:34:39.248 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-02 14:27:55.782564679 +0000 2024-04-02T14:34:39.248 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-02 14:27:55.782564679 +0000 2024-04-02T14:34:39.250 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T14:34:39.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-02T14:34:39.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-02T14:34:39.348 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.00039161 s, 1.3 MB/s 2024-04-02T14:34:39.352 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T14:34:39.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T14:34:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T14:34:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T14:34:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:39.413 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-04-02T14:34:39.464 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T14:34:39.465 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:39.465 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-04-02T14:34:39.465 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:39.465 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:39.465 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-02 14:33:32.618918871 +0000 2024-04-02T14:34:39.465 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-02 14:27:56.372555421 +0000 2024-04-02T14:34:39.465 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-02 14:27:56.372555421 +0000 2024-04-02T14:34:39.465 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-02 14:27:56.372555421 +0000 2024-04-02T14:34:39.466 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T14:34:39.550 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:39.550 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.550 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.550 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.550 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:39.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:39.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:39.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:39.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T14:34:39.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T14:34:39.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T14:34:39.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:39.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-02T14:34:39.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-02T14:34:39.561 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000364809 s, 1.4 MB/s 2024-04-02T14:34:39.565 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T14:34:39.624 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-04-02T14:34:39.676 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T14:34:39.677 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-04-02T14:34:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-02 14:33:32.711917400 +0000 2024-04-02T14:34:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-02 14:27:57.079544328 +0000 2024-04-02T14:34:39.677 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-02 14:27:57.079544328 +0000 2024-04-02T14:34:39.677 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-02 14:27:57.079544328 +0000 2024-04-02T14:34:39.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T14:34:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-02T14:34:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-02T14:34:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000331416 s, 1.5 MB/s 2024-04-02T14:34:39.767 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T14:34:39.825 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-04-02T14:34:39.879 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T14:34:39.880 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:39.880 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-04-02T14:34:39.880 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:39.880 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:39.880 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-02 14:33:32.803915944 +0000 2024-04-02T14:34:39.880 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-02 14:27:57.671535039 +0000 2024-04-02T14:34:39.880 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-02 14:27:57.671535039 +0000 2024-04-02T14:34:39.880 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-02 14:27:57.671535039 +0000 2024-04-02T14:34:39.880 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T14:34:39.972 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-02T14:34:39.972 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-02T14:34:39.972 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000360903 s, 1.4 MB/s 2024-04-02T14:34:39.975 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T14:34:40.033 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-02T14:34:40.033 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T14:34:40.053 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T14:34:40.148 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-04-02T14:34:40.167 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T14:34:40.167 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-04-02T14:34:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-02 14:34:23.143558919 +0000 2024-04-02T14:34:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-02 14:26:29.819810799 +0000 2024-04-02T14:34:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-02 14:26:29.819810799 +0000 2024-04-02T14:34:40.167 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-02 14:26:29.819810799 +0000 2024-04-02T14:34:40.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T14:34:40.239 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-02T14:34:40.239 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-02T14:34:40.239 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000212083 s, 2.4 MB/s 2024-04-02T14:34:40.240 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T14:34:40.302 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-04-02T14:34:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:40 smithi062 ceph-mon[32116]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi113=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:34:40.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:40 smithi062 ceph-mon[32116]: Saving service mgr spec with placement smithi062=x;smithi113=y;smithi204=z;count:3 2024-04-02T14:34:40.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:40 smithi062 ceph-mon[32116]: Deploying daemon mgr.z on smithi204 2024-04-02T14:34:40.359 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T14:34:40.359 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-04-02T14:34:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-02 14:34:23.143558919 +0000 2024-04-02T14:34:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-02 14:26:30.172802903 +0000 2024-04-02T14:34:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-02 14:26:30.172802903 +0000 2024-04-02T14:34:40.360 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-02 14:26:30.172802903 +0000 2024-04-02T14:34:40.360 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T14:34:40.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:40 smithi113 ceph-mon[36135]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi113=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:34:40.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:40 smithi113 ceph-mon[36135]: Saving service mgr spec with placement smithi062=x;smithi113=y;smithi204=z;count:3 2024-04-02T14:34:40.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:40 smithi113 ceph-mon[36135]: Deploying daemon mgr.z on smithi204 2024-04-02T14:34:40.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-02T14:34:40.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-02T14:34:40.403 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.00025344 s, 2.0 MB/s 2024-04-02T14:34:40.403 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T14:34:40.462 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-04-02T14:34:40.496 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:40 smithi204 ceph-mon[35948]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi113=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:34:40.497 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:40 smithi204 ceph-mon[35948]: Saving service mgr spec with placement smithi062=x;smithi113=y;smithi204=z;count:3 2024-04-02T14:34:40.497 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:40 smithi204 ceph-mon[35948]: Deploying daemon mgr.z on smithi204 2024-04-02T14:34:40.522 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T14:34:40.522 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-04-02T14:34:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-02 14:34:23.143558919 +0000 2024-04-02T14:34:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-02 14:26:30.520795119 +0000 2024-04-02T14:34:40.523 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-02 14:26:30.520795119 +0000 2024-04-02T14:34:40.523 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-02 14:26:30.520795119 +0000 2024-04-02T14:34:40.523 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T14:34:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-02T14:34:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-02T14:34:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000254732 s, 2.0 MB/s 2024-04-02T14:34:40.590 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T14:34:40.646 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-04-02T14:34:40.701 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T14:34:40.701 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-02T14:34:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:40.702 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-02 14:34:23.144558897 +0000 2024-04-02T14:34:40.702 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-02 14:26:30.856787604 +0000 2024-04-02T14:34:40.702 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-02 14:26:30.856787604 +0000 2024-04-02T14:34:40.702 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-02 14:26:30.856787604 +0000 2024-04-02T14:34:40.702 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T14:34:40.760 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:40 smithi204 systemd[1]: Starting Ceph mgr.z for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:34:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-02T14:34:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-02T14:34:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000201761 s, 2.5 MB/s 2024-04-02T14:34:40.769 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T14:34:40.825 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T14:34:40.825 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T14:34:40.841 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T14:34:40.841 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-04-02T14:34:40.895 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T14:34:40.896 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:40.896 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-02T14:34:40.896 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:40.896 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:40.896 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-02 14:34:40.603371046 +0000 2024-04-02T14:34:40.896 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-02 14:26:58.630483664 +0000 2024-04-02T14:34:40.896 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-02 14:26:58.630483664 +0000 2024-04-02T14:34:40.896 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-02 14:26:58.630483664 +0000 2024-04-02T14:34:40.896 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T14:34:40.964 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-02T14:34:40.964 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-02T14:34:40.964 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000247078 s, 2.1 MB/s 2024-04-02T14:34:40.965 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T14:34:41.022 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-04-02T14:34:41.086 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T14:34:41.086 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:41.086 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 679 Links: 1 2024-04-02T14:34:41.086 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:41.087 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:41.087 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-02 14:34:40.603371046 +0000 2024-04-02T14:34:41.087 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-02 14:26:58.975478775 +0000 2024-04-02T14:34:41.087 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-02 14:26:58.975478775 +0000 2024-04-02T14:34:41.087 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-02 14:26:58.975478775 +0000 2024-04-02T14:34:41.087 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T14:34:41.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:41 smithi204 ceph-mon[35948]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:41.129 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:40 smithi204 podman[37584]: 2024-04-02 14:34:40.780256921 +0000 UTC m=+0.018362266 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:34:41.129 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:40 smithi204 podman[37584]: 2024-04-02 14:34:40.883107608 +0000 UTC m=+0.121212931 container create a846a5f3b4b88121e598a1ec70578d1af8ad4d12a60691545ce39c0bcc374ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-02T14:34:41.130 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:41 smithi204 podman[37584]: 2024-04-02 14:34:41.102384375 +0000 UTC m=+0.340489710 container init a846a5f3b4b88121e598a1ec70578d1af8ad4d12a60691545ce39c0bcc374ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-04-02T14:34:41.130 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:41 smithi204 podman[37584]: 2024-04-02 14:34:41.104820635 +0000 UTC m=+0.342925969 container start a846a5f3b4b88121e598a1ec70578d1af8ad4d12a60691545ce39c0bcc374ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-02T14:34:41.130 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:41 smithi204 ceph-mgr[37688]: -- 172.21.15.204:0/3841604035 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5652cbb985a0 con 0x5652cbb6c800 2024-04-02T14:34:41.153 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-02T14:34:41.153 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-02T14:34:41.153 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000222844 s, 2.3 MB/s 2024-04-02T14:34:41.155 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T14:34:41.210 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-04-02T14:34:41.267 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T14:34:41.267 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:41.267 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 698 Links: 1 2024-04-02T14:34:41.267 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:41.267 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:41.268 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-02 14:34:40.603371046 +0000 2024-04-02T14:34:41.268 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-02 14:26:59.506471251 +0000 2024-04-02T14:34:41.268 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-02 14:26:59.506471251 +0000 2024-04-02T14:34:41.268 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-02 14:26:59.506471251 +0000 2024-04-02T14:34:41.268 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T14:34:41.334 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-02T14:34:41.334 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-02T14:34:41.334 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000227972 s, 2.2 MB/s 2024-04-02T14:34:41.335 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T14:34:41.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:41 smithi113 ceph-mon[36135]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:41.393 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-04-02T14:34:41.449 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T14:34:41.449 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T14:34:41.449 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-04-02T14:34:41.449 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:34:41.449 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T14:34:41.449 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-02 14:34:40.604371032 +0000 2024-04-02T14:34:41.449 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-02 14:26:59.835466590 +0000 2024-04-02T14:34:41.449 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-02 14:26:59.835466590 +0000 2024-04-02T14:34:41.450 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-02 14:26:59.835466590 +0000 2024-04-02T14:34:41.450 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T14:34:41.510 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:41 smithi204 bash[37584]: a846a5f3b4b88121e598a1ec70578d1af8ad4d12a60691545ce39c0bcc374ea2 2024-04-02T14:34:41.510 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:41 smithi204 systemd[1]: Started Ceph mgr.z for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:34:41.515 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-02T14:34:41.515 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-02T14:34:41.515 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000211632 s, 2.4 MB/s 2024-04-02T14:34:41.516 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T14:34:41.571 INFO:tasks.cephadm:Deploying osd.0 on smithi062 with /dev/vg_nvme/lv_4... 2024-04-02T14:34:41.572 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T14:34:41.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:41 smithi062 ceph-mon[32116]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:41.763 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:41 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:41.669+0000 7f744f237200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T14:34:42.030 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:34:42.260 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:41 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:41.762+0000 7f744f237200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T14:34:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:42 smithi062 ceph-mon[32116]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T14:34:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T14:34:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T14:34:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:42 smithi062 ceph-mon[32116]: Deploying daemon mgr.y on smithi113 2024-04-02T14:34:42.589 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:42 smithi113 ceph-mon[36135]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:42.589 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:42.589 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:42.589 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:42.589 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T14:34:42.589 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T14:34:42.589 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T14:34:42.589 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:42.589 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:42 smithi113 ceph-mon[36135]: Deploying daemon mgr.y on smithi113 2024-04-02T14:34:42.657 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:42.412+0000 7f744f237200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T14:34:42.657 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:42.507+0000 7f744f237200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T14:34:42.657 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:42.654+0000 7f744f237200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T14:34:42.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-mon[35948]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:42.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:42.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:42.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:42.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T14:34:42.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T14:34:42.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T14:34:42.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:42.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-mon[35948]: Deploying daemon mgr.y on smithi113 2024-04-02T14:34:43.011 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:42.752+0000 7f744f237200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T14:34:43.011 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:42.838+0000 7f744f237200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T14:34:43.402 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:43.108+0000 7f744f237200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T14:34:43.402 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:43.219+0000 7f744f237200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T14:34:43.402 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:43.307+0000 7f744f237200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T14:34:43.402 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:43.400+0000 7f744f237200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T14:34:43.484 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:43 smithi113 systemd[1]: Starting Ceph mgr.y for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:34:43.760 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:43.575+0000 7f744f237200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T14:34:43.786 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:43 smithi113 podman[37508]: 2024-04-02 14:34:43.4829529 +0000 UTC m=+0.086476777 container create 995bf8c68dae4406685885865913e207112fcd27a68154aedb0d341676209391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:34:43.786 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:43 smithi113 podman[37508]: 2024-04-02 14:34:43.414049777 +0000 UTC m=+0.017573663 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:34:43.786 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:43 smithi113 podman[37508]: 2024-04-02 14:34:43.685332992 +0000 UTC m=+0.288856870 container init 995bf8c68dae4406685885865913e207112fcd27a68154aedb0d341676209391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-04-02T14:34:43.786 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:43 smithi113 podman[37508]: 2024-04-02 14:34:43.687740231 +0000 UTC m=+0.291264113 container start 995bf8c68dae4406685885865913e207112fcd27a68154aedb0d341676209391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-02T14:34:43.787 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:43 smithi113 ceph-mgr[37527]: -- 172.21.15.113:0/3151435349 <== mon.2 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b22e1c5a0 con 0x560b22df0800 2024-04-02T14:34:43.787 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:43 smithi113 bash[37508]: 995bf8c68dae4406685885865913e207112fcd27a68154aedb0d341676209391 2024-04-02T14:34:44.121 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:43 smithi113 systemd[1]: Started Ceph mgr.y for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:34:44.621 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:44.242+0000 7f28aad6e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T14:34:44.622 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:44.339+0000 7f28aad6e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T14:34:44.867 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:44 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:44.565+0000 7f744f237200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T14:34:44.867 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:44 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:44.699+0000 7f744f237200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T14:34:44.944 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T14:34:44.944 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T14:34:44.944 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-02T14:34:44.944 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-02T14:34:44.944 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0356611 s, 294 MB/s 2024-04-02T14:34:44.944 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-02T14:34:45.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:44 smithi062 ceph-mon[32116]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:45.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:44 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:44 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:44 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:44 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:44 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:45.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:44 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:44 smithi113 ceph-mon[36135]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:45.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:44 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:44 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:44 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:44 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:44 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:45.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:44 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.122 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:45 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:45.021+0000 7f28aad6e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T14:34:45.122 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:45 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:45.115+0000 7f28aad6e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T14:34:45.149 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:45.054+0000 7f744f237200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T14:34:45.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:44 smithi204 ceph-mon[35948]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:45.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:44 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:44 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:44 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:44 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:44 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:45.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:44 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:45.438 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:45 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:45.261+0000 7f28aad6e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T14:34:45.438 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:45 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:45.354+0000 7f28aad6e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T14:34:45.438 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:45 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:45.436+0000 7f28aad6e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T14:34:45.510 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:45.147+0000 7f744f237200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T14:34:45.511 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:45.348+0000 7f744f237200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:34:45.754 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2024-04-02T14:34:45.827 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:45.538+0000 7f744f237200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T14:34:46.004 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:45 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:45.714+0000 7f28aad6e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T14:34:46.004 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:45 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:45.824+0000 7f28aad6e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T14:34:46.005 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:45 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:45.910+0000 7f28aad6e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T14:34:46.005 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:46.003+0000 7f28aad6e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T14:34:46.090 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:45.825+0000 7f744f237200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:34:46.267 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:34:46.371 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:46.175+0000 7f28aad6e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T14:34:46.511 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:46.088+0000 7f744f237200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T14:34:46.511 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:46.324+0000 7f744f237200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T14:34:46.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:46.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:46.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:46.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:46.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:46.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:46.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:46.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:46.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T14:34:46.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T14:34:46.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T14:34:46.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:46.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:46 smithi062 ceph-mon[32116]: Reconfiguring daemon mgr.x on smithi062 2024-04-02T14:34:46.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:46.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:46.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:46.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:46.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:46.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:46.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:46.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:46.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T14:34:46.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T14:34:46.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T14:34:46.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:46.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:46 smithi113 ceph-mon[36135]: Reconfiguring daemon mgr.x on smithi062 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T14:34:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:47.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:46 smithi204 ceph-mon[35948]: Reconfiguring daemon mgr.x on smithi062 2024-04-02T14:34:47.510 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:47.126+0000 7f744f237200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T14:34:47.603 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:47 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:47.138+0000 7f28aad6e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T14:34:47.603 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:47 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:47.269+0000 7f28aad6e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T14:34:47.603 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:47 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:47.602+0000 7f28aad6e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T14:34:47.871 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:47 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:47.691+0000 7f28aad6e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T14:34:48.009 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:47.697+0000 7f744f237200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T14:34:48.009 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:47.792+0000 7f744f237200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T14:34:48.009 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:34:48 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:34:48.007+0000 7f744f237200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T14:34:48.350 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:47 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:47.886+0000 7f28aad6e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:34:48.350 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:48 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:48.068+0000 7f28aad6e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T14:34:48.604 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:48 smithi113 ceph-mon[36135]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:48.604 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:48 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T14:34:48.605 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:48 smithi113 ceph-mon[36135]: Standby manager daemon z started 2024-04-02T14:34:48.605 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:48 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:34:48.605 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:48 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T14:34:48.605 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:48 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:34:48.605 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:48 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:48.349+0000 7f28aad6e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:34:48.605 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:48 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:48.603+0000 7f28aad6e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T14:34:48.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:48 smithi062 ceph-mon[32116]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:48.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:48 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T14:34:48.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:48 smithi062 ceph-mon[32116]: Standby manager daemon z started 2024-04-02T14:34:48.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:48 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:34:48.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:48 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T14:34:48.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:48 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:34:48.871 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:48 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:48.839+0000 7f28aad6e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T14:34:49.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:48 smithi204 ceph-mon[35948]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:49.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:48 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T14:34:49.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:48 smithi204 ceph-mon[35948]: Standby manager daemon z started 2024-04-02T14:34:49.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:48 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:34:49.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:48 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T14:34:49.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:48 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.204:0/1217778128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:34:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.344+0000 7f8b21000700 1 -- 172.21.15.62:0/3586949049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c1041f0 msgr2=0x7f8b1c10aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.344+0000 7f8b21000700 1 --2- 172.21.15.62:0/3586949049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c1041f0 0x7f8b1c10aaa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8b140099c0 tx=0x7f8b14009cd0 comp rx=0 tx=0).stop 2024-04-02T14:34:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.344+0000 7f8b21000700 1 -- 172.21.15.62:0/3586949049 shutdown_connections 2024-04-02T14:34:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.344+0000 7f8b21000700 1 --2- 172.21.15.62:0/3586949049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c1041f0 0x7f8b1c10aaa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.344+0000 7f8b21000700 1 --2- 172.21.15.62:0/3586949049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b1c102650 0x7f8b1c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.344+0000 7f8b21000700 1 --2- 172.21.15.62:0/3586949049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1c103850 0x7f8b1c103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.344+0000 7f8b21000700 1 -- 172.21.15.62:0/3586949049 >> 172.21.15.62:0/3586949049 conn(0x7f8b1c0fdbf0 msgr2=0x7f8b1c100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:34:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.344+0000 7f8b21000700 1 -- 172.21.15.62:0/3586949049 shutdown_connections 2024-04-02T14:34:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.344+0000 7f8b21000700 1 -- 172.21.15.62:0/3586949049 wait complete. 2024-04-02T14:34:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.346+0000 7f8b21000700 1 Processor -- start 2024-04-02T14:34:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.346+0000 7f8b21000700 1 -- start start 2024-04-02T14:34:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.347+0000 7f8b21000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c102650 0x7f8b1c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.347+0000 7f8b21000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b1c103850 0x7f8b1c198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.347+0000 7f8b21000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1c1041f0 0x7f8b1c19fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.347+0000 7f8b21000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c10e360 con 0x7f8b1c1041f0 2024-04-02T14:34:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.347+0000 7f8b21000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c10e1e0 con 0x7f8b1c102650 2024-04-02T14:34:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.347+0000 7f8b21000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c10e4e0 con 0x7f8b1c103850 2024-04-02T14:34:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.347+0000 7f8b19e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c102650 0x7f8b1c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.347+0000 7f8b19e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c102650 0x7f8b1c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:60928/0 (socket says 172.21.15.62:60928) 2024-04-02T14:34:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.347+0000 7f8b19e00700 1 -- 172.21.15.62:0/3291519484 learned_addr learned my addr 172.21.15.62:0/3291519484 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:34:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.347+0000 7f8b13e00700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b1c103850 0x7f8b1c198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.348+0000 7f8b1a800700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1c1041f0 0x7f8b1c19fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.348+0000 7f8b19e00700 1 -- 172.21.15.62:0/3291519484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b1c103850 msgr2=0x7f8b1c198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.348+0000 7f8b19e00700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b1c103850 0x7f8b1c198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.348+0000 7f8b19e00700 1 -- 172.21.15.62:0/3291519484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1c1041f0 msgr2=0x7f8b1c19fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:34:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.348+0000 7f8b19e00700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1c1041f0 0x7f8b1c19fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:34:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.348+0000 7f8b19e00700 1 -- 172.21.15.62:0/3291519484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c1a0510 con 0x7f8b1c102650 2024-04-02T14:34:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.348+0000 7f8b19e00700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c102650 0x7f8b1c1988b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00efc0 tx=0x7f8b0c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.348+0000 7f8b12a00700 1 -- 172.21.15.62:0/3291519484 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0092e0 con 0x7f8b1c102650 2024-04-02T14:34:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.348+0000 7f8b12a00700 1 -- 172.21.15.62:0/3291519484 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b0c003a00 con 0x7f8b1c102650 2024-04-02T14:34:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.348+0000 7f8b21000700 1 -- 172.21.15.62:0/3291519484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1c1a07a0 con 0x7f8b1c102650 2024-04-02T14:34:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.348+0000 7f8b12a00700 1 -- 172.21.15.62:0/3291519484 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c004770 con 0x7f8b1c102650 2024-04-02T14:34:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.349+0000 7f8b21000700 1 -- 172.21.15.62:0/3291519484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1c1a0d30 con 0x7f8b1c102650 2024-04-02T14:34:49.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.350+0000 7f8b21000700 1 -- 172.21.15.62:0/3291519484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b1c0692b0 con 0x7f8b1c102650 2024-04-02T14:34:49.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.350+0000 7f8b12a00700 1 -- 172.21.15.62:0/3291519484 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f8b0c01b030 con 0x7f8b1c102650 2024-04-02T14:34:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.351+0000 7f8b12a00700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8b0006ea20 0x7f8b00070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:34:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.352+0000 7f8b12a00700 1 -- 172.21.15.62:0/3291519484 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8b0c08d830 con 0x7f8b1c102650 2024-04-02T14:34:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.352+0000 7f8b13e00700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8b0006ea20 0x7f8b00070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:34:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.352+0000 7f8b13e00700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8b0006ea20 0x7f8b00070ee0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009c60 tx=0x7f8b04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:34:49.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.356+0000 7f8b12a00700 1 -- 172.21.15.62:0/3291519484 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8b0c022020 con 0x7f8b1c102650 2024-04-02T14:34:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:49.602+0000 7f8b21000700 1 -- 172.21.15.62:0/3291519484 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8b1c062a30 con 0x7f8b0006ea20 2024-04-02T14:34:49.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:49 smithi062 ceph-mon[32116]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-02T14:34:49.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:49 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T14:34:49.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:49 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:49.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:49 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:49.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:49 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:49.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:49 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:49.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:49 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:49.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:49 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:49.871 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:49 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:49.634+0000 7f28aad6e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T14:34:49.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:49 smithi113 ceph-mon[36135]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-02T14:34:49.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:49 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T14:34:49.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:49 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:49.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:49 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:49.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:49 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:49.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:49 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:49.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:49 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:49.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:49 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:50.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:49 smithi204 ceph-mon[35948]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-02T14:34:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:49 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T14:34:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:49 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:49 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:49 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:34:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:49 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:49 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:34:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:49 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:34:50.503 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:50.198+0000 7f28aad6e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T14:34:50.503 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:50.292+0000 7f28aad6e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T14:34:50.503 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:34:50.501+0000 7f28aad6e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T14:34:50.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:50 smithi062 ceph-mon[32116]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:50 smithi062 ceph-mon[32116]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:34:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:50 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:34:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:50 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:34:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:50 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:50 smithi062 ceph-mon[32116]: Standby manager daemon y started 2024-04-02T14:34:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:50 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T14:34:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:50 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:34:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:50 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T14:34:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:50 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:34:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-mon[36135]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-mon[36135]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:34:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:34:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:34:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-mon[36135]: Standby manager daemon y started 2024-04-02T14:34:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T14:34:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:34:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T14:34:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:50 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:34:50.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:34:50.954+0000 7f8b12a00700 1 -- 172.21.15.62:0/3291519484 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f8b0c012050 con 0x7f8b1c102650 2024-04-02T14:34:51.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:50 smithi204 ceph-mon[35948]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:51.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:50 smithi204 ceph-mon[35948]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:34:51.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:50 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:34:51.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:50 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:34:51.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:50 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:34:51.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:50 smithi204 ceph-mon[35948]: Standby manager daemon y started 2024-04-02T14:34:51.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:50 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T14:34:51.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:50 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:34:51.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:50 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T14:34:51.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:50 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.113:0/368258353' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:34:52.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:51 smithi062 ceph-mon[32116]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-02T14:34:52.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:51 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T14:34:52.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:51 smithi204 ceph-mon[35948]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-02T14:34:52.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:51 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T14:34:52.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:51 smithi113 ceph-mon[36135]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-02T14:34:52.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:51 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T14:34:53.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:52 smithi204 ceph-mon[35948]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:53.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:52 smithi062 ceph-mon[32116]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:53.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:52 smithi113 ceph-mon[36135]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:54.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:53 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2227126083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fd83a35-8d86-4932-adfb-5185155bec3b"}]: dispatch 2024-04-02T14:34:54.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:53 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fd83a35-8d86-4932-adfb-5185155bec3b"}]: dispatch 2024-04-02T14:34:54.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:53 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fd83a35-8d86-4932-adfb-5185155bec3b"}]': finished 2024-04-02T14:34:54.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:53 smithi204 ceph-mon[35948]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T14:34:54.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:53 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:34:54.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:53 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2227126083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fd83a35-8d86-4932-adfb-5185155bec3b"}]: dispatch 2024-04-02T14:34:54.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:53 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fd83a35-8d86-4932-adfb-5185155bec3b"}]: dispatch 2024-04-02T14:34:54.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:53 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fd83a35-8d86-4932-adfb-5185155bec3b"}]': finished 2024-04-02T14:34:54.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:53 smithi062 ceph-mon[32116]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T14:34:54.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:53 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:34:54.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:53 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2227126083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fd83a35-8d86-4932-adfb-5185155bec3b"}]: dispatch 2024-04-02T14:34:54.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:53 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fd83a35-8d86-4932-adfb-5185155bec3b"}]: dispatch 2024-04-02T14:34:54.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:53 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fd83a35-8d86-4932-adfb-5185155bec3b"}]': finished 2024-04-02T14:34:54.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:53 smithi113 ceph-mon[36135]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T14:34:54.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:53 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:34:55.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:55 smithi062 ceph-mon[32116]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:55.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:55 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1759689239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:34:55.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:55 smithi113 ceph-mon[36135]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:55.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:55 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1759689239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:34:55.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:55 smithi204 ceph-mon[35948]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:55.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:55 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1759689239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:34:57.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:57 smithi062 ceph-mon[32116]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:57.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:57 smithi113 ceph-mon[36135]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:57.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:57 smithi204 ceph-mon[35948]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:59.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:34:59 smithi062 ceph-mon[32116]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:59.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:34:59 smithi113 ceph-mon[36135]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:34:59.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:34:59 smithi204 ceph-mon[35948]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:01.313 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:01 smithi204 ceph-mon[35948]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:01.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:01 smithi062 ceph-mon[32116]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:01.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:01 smithi113 ceph-mon[36135]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:03.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:03 smithi062 ceph-mon[32116]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:03.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:03 smithi113 ceph-mon[36135]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:03.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:03 smithi204 ceph-mon[35948]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:04.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T14:35:04.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:04.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T14:35:04.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:04.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T14:35:04.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:05.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:05 smithi062 ceph-mon[32116]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:05.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:05 smithi062 ceph-mon[32116]: Deploying daemon osd.0 on smithi062 2024-04-02T14:35:05.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:05 smithi113 ceph-mon[36135]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:05.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:05 smithi113 ceph-mon[36135]: Deploying daemon osd.0 on smithi062 2024-04-02T14:35:05.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:05 smithi204 ceph-mon[35948]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:05.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:05 smithi204 ceph-mon[35948]: Deploying daemon osd.0 on smithi062 2024-04-02T14:35:07.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:07 smithi062 ceph-mon[32116]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:07.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:07 smithi113 ceph-mon[36135]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:07.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:07 smithi204 ceph-mon[35948]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:09.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:09 smithi062 ceph-mon[32116]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:09.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:09 smithi113 ceph-mon[36135]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:09.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:09 smithi204 ceph-mon[35948]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:11.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:11 smithi062 ceph-mon[32116]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:11.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:11 smithi113 ceph-mon[36135]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:11.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:11 smithi204 ceph-mon[35948]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:12.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:12 smithi062 ceph-mon[32116]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:12.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:12.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:12.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:13.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:12 smithi204 ceph-mon[35948]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:13.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:13.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:13.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:13.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:12 smithi113 ceph-mon[36135]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:13.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:13.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:13.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:14.828 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:14 smithi062 ceph-mon[32116]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:15.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:14 smithi204 ceph-mon[35948]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:15.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:14 smithi113 ceph-mon[36135]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:15.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:15 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:15.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:15 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:15.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:15 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:15.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:15 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:15.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:15 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:15.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:15 smithi062 ceph-mon[32116]: from='osd.0 [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T14:35:15.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:15 smithi062 ceph-mon[32116]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T14:35:16.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:15 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:16.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:15 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:16.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:15 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:16.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:15 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:16.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:15 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:16.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:15 smithi113 ceph-mon[36135]: from='osd.0 [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T14:35:16.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:15 smithi113 ceph-mon[36135]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T14:35:16.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:15 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:16.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:15 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:16.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:15 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:16.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:15 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:16.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:15 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:16.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:15 smithi204 ceph-mon[35948]: from='osd.0 [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T14:35:16.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:15 smithi204 ceph-mon[35948]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T14:35:16.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.762+0000 7f8b12a00700 1 -- 172.21.15.62:0/3291519484 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8b1c062a30 con 0x7f8b0006ea20 2024-04-02T14:35:16.765 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 0 on host 'smithi062' 2024-04-02T14:35:16.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.766+0000 7f8b21000700 1 -- 172.21.15.62:0/3291519484 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8b0006ea20 msgr2=0x7f8b00070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:35:16.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.766+0000 7f8b21000700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8b0006ea20 0x7f8b00070ee0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009c60 tx=0x7f8b04009260 comp rx=0 tx=0).stop 2024-04-02T14:35:16.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.766+0000 7f8b21000700 1 -- 172.21.15.62:0/3291519484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c102650 msgr2=0x7f8b1c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:35:16.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.766+0000 7f8b21000700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c102650 0x7f8b1c1988b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00efc0 tx=0x7f8b0c00c3a0 comp rx=0 tx=0).stop 2024-04-02T14:35:16.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.766+0000 7f8b21000700 1 -- 172.21.15.62:0/3291519484 shutdown_connections 2024-04-02T14:35:16.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.766+0000 7f8b21000700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8b0006ea20 0x7f8b00070ee0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:16.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.766+0000 7f8b21000700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c102650 0x7f8b1c1988b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:16.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.766+0000 7f8b21000700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1c1041f0 0x7f8b1c19fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:16.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.766+0000 7f8b21000700 1 --2- 172.21.15.62:0/3291519484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b1c103850 0x7f8b1c198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:16.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.766+0000 7f8b21000700 1 -- 172.21.15.62:0/3291519484 >> 172.21.15.62:0/3291519484 conn(0x7f8b1c0fdbf0 msgr2=0x7f8b1c106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:35:16.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.767+0000 7f8b21000700 1 -- 172.21.15.62:0/3291519484 shutdown_connections 2024-04-02T14:35:16.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:35:16.767+0000 7f8b21000700 1 -- 172.21.15.62:0/3291519484 wait complete. 2024-04-02T14:35:17.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:16 smithi062 ceph-mon[32116]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:17.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:16 smithi062 ceph-mon[32116]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T14:35:17.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:16 smithi062 ceph-mon[32116]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T14:35:17.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:16 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:17.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:16 smithi062 ceph-mon[32116]: from='osd.0 [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:35:17.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:16 smithi062 ceph-mon[32116]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:35:17.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:16 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:17.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:16 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:17.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:16 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:17.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:16 smithi113 ceph-mon[36135]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:17.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:16 smithi113 ceph-mon[36135]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T14:35:17.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:16 smithi113 ceph-mon[36135]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T14:35:17.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:16 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:17.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:16 smithi113 ceph-mon[36135]: from='osd.0 [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:35:17.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:16 smithi113 ceph-mon[36135]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:35:17.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:16 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:17.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:16 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:17.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:16 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:17.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:16 smithi204 ceph-mon[35948]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:17.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:16 smithi204 ceph-mon[35948]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T14:35:17.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:16 smithi204 ceph-mon[35948]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T14:35:17.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:16 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:17.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:16 smithi204 ceph-mon[35948]: from='osd.0 [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:35:17.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:16 smithi204 ceph-mon[35948]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:35:17.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:16 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:17.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:16 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:17.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:16 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:17.587 DEBUG:teuthology.orchestra.run.smithi062:osd.0> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.0.service 2024-04-02T14:35:17.594 INFO:tasks.cephadm:Deploying osd.1 on smithi113 with /dev/vg_nvme/lv_4... 2024-04-02T14:35:17.595 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T14:35:17.838 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:35:17.959 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:17 smithi113 ceph-mon[36135]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:35:17.959 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:17 smithi113 ceph-mon[36135]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T14:35:17.960 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:17 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:17.960 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:17 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:18.073 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:17 smithi062 ceph-mon[32116]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:35:18.073 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:17 smithi062 ceph-mon[32116]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T14:35:18.073 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:17 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:18.073 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:17 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:18.073 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:35:17 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0[44630]: 2024-04-02T14:35:17.884+0000 7f11aaa00700 -1 osd.0 0 waiting for initial osdmap 2024-04-02T14:35:18.073 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:35:17 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0[44630]: 2024-04-02T14:35:17.904+0000 7f11a0800700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:35:18.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:17 smithi204 ceph-mon[35948]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:35:18.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:17 smithi204 ceph-mon[35948]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T14:35:18.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:17 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:18.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:17 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:18.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:18 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:35:18.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:18 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:35:18.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:18 smithi113 ceph-mon[36135]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:18.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:18 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:18.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:18 smithi113 ceph-mon[36135]: from='osd.0 ' entity='osd.0' 2024-04-02T14:35:18.938 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:18 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:35:18.938 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:18 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:35:18.938 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:18 smithi062 ceph-mon[32116]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:18.938 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:18 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:18.938 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:18 smithi062 ceph-mon[32116]: from='osd.0 ' entity='osd.0' 2024-04-02T14:35:19.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:18 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:35:19.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:18 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:35:19.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:18 smithi204 ceph-mon[35948]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:19.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:18 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:19.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:18 smithi204 ceph-mon[35948]: from='osd.0 ' entity='osd.0' 2024-04-02T14:35:19.663 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T14:35:19.663 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T14:35:19.663 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-04-02T14:35:19.664 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-04-02T14:35:19.664 INFO:teuthology.orchestra.run.smithi113.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00930283 s, 1.1 GB/s 2024-04-02T14:35:19.664 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-04-02T14:35:19.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:19.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:19 smithi062 ceph-mon[32116]: osd.0 [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191] boot 2024-04-02T14:35:19.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:19 smithi062 ceph-mon[32116]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T14:35:19.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:20.021 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:19 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:20.021 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:19 smithi113 ceph-mon[36135]: osd.0 [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191] boot 2024-04-02T14:35:20.021 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:19 smithi113 ceph-mon[36135]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T14:35:20.021 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:19 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:20.258 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi113:vg_nvme/lv_4 2024-04-02T14:35:20.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:20.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:19 smithi204 ceph-mon[35948]: osd.0 [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191] boot 2024-04-02T14:35:20.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:19 smithi204 ceph-mon[35948]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T14:35:20.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:35:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:35:21.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:20 smithi204 ceph-mon[35948]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:21.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:20 smithi204 ceph-mon[35948]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T14:35:21.271 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:20 smithi113 ceph-mon[36135]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:21.271 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:20 smithi113 ceph-mon[36135]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T14:35:21.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:20 smithi062 ceph-mon[32116]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T14:35:21.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:20 smithi062 ceph-mon[32116]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T14:35:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.085+0000 7f87a1db7700 1 -- 172.21.15.113:0/6965022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c075050 msgr2=0x7f879c0754d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:35:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.085+0000 7f87a1db7700 1 --2- 172.21.15.113:0/6965022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c075050 0x7f879c0754d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8790009990 tx=0x7f8790009ca0 comp rx=0 tx=0).stop 2024-04-02T14:35:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.085+0000 7f87a1db7700 1 -- 172.21.15.113:0/6965022 shutdown_connections 2024-04-02T14:35:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.085+0000 7f87a1db7700 1 --2- 172.21.15.113:0/6965022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f879c10a510 0x7f879c10c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:22.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.085+0000 7f87a1db7700 1 --2- 172.21.15.113:0/6965022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f879c076620 0x7f879c074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:22.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.085+0000 7f87a1db7700 1 --2- 172.21.15.113:0/6965022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c075050 0x7f879c0754d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:22.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.085+0000 7f87a1db7700 1 -- 172.21.15.113:0/6965022 >> 172.21.15.113:0/6965022 conn(0x7f879c0fdc00 msgr2=0x7f879c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:35:22.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.085+0000 7f87a1db7700 1 -- 172.21.15.113:0/6965022 shutdown_connections 2024-04-02T14:35:22.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.085+0000 7f87a1db7700 1 -- 172.21.15.113:0/6965022 wait complete. 2024-04-02T14:35:22.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f87a1db7700 1 Processor -- start 2024-04-02T14:35:22.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f87a1db7700 1 -- start start 2024-04-02T14:35:22.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f87a1db7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f879c075050 0x7f879c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:35:22.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f87a1db7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c076620 0x7f879c19b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:35:22.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f87a1db7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f879c10a510 0x7f879c1a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:35:22.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f87a1db7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f879c110660 con 0x7f879c10a510 2024-04-02T14:35:22.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f87a1db7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f879c1104e0 con 0x7f879c076620 2024-04-02T14:35:22.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f87a1db7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f879c1107e0 con 0x7f879c075050 2024-04-02T14:35:22.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f879affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c076620 0x7f879c19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:35:22.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f879affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c076620 0x7f879c19b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56748/0 (socket says 172.21.15.113:56748) 2024-04-02T14:35:22.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f879affd700 1 -- 172.21.15.113:0/244664935 learned_addr learned my addr 172.21.15.113:0/244664935 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-02T14:35:22.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.086+0000 7f879b7fe700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f879c075050 0x7f879c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.087+0000 7f879affd700 1 -- 172.21.15.113:0/244664935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f879c075050 msgr2=0x7f879c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.087+0000 7f879affd700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f879c075050 0x7f879c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.087+0000 7f879affd700 1 -- 172.21.15.113:0/244664935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f879c10a510 msgr2=0x7f879c1a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.087+0000 7f879affd700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f879c10a510 0x7f879c1a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.087+0000 7f879affd700 1 -- 172.21.15.113:0/244664935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c1a2880 con 0x7f879c076620 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.087+0000 7f879affd700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c076620 0x7f879c19b170 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8790000c00 tx=0x7f879000f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.087+0000 7f8798ff9700 1 -- 172.21.15.113:0/244664935 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790027070 con 0x7f879c076620 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.087+0000 7f8798ff9700 1 -- 172.21.15.113:0/244664935 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f879000fbf0 con 0x7f879c076620 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.087+0000 7f87a1db7700 1 -- 172.21.15.113:0/244664935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879c1a2b10 con 0x7f879c076620 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.087+0000 7f8798ff9700 1 -- 172.21.15.113:0/244664935 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87900176e0 con 0x7f879c076620 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.087+0000 7f87a1db7700 1 -- 172.21.15.113:0/244664935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879c1a3040 con 0x7f879c076620 2024-04-02T14:35:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.088+0000 7f87a1db7700 1 -- 172.21.15.113:0/244664935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879c0692b0 con 0x7f879c076620 2024-04-02T14:35:22.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.089+0000 7f8798ff9700 1 -- 172.21.15.113:0/244664935 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f879002a070 con 0x7f879c076620 2024-04-02T14:35:22.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.090+0000 7f8798ff9700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87840a3c30 0x7f87840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:35:22.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.090+0000 7f8798ff9700 1 -- 172.21.15.113:0/244664935 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f87900cff60 con 0x7f879c076620 2024-04-02T14:35:22.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.090+0000 7f879b7fe700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87840a3c30 0x7f87840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:35:22.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.091+0000 7f879b7fe700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87840a3c30 0x7f87840a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f878c006fd0 tx=0x7f878c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:35:22.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.094+0000 7f8798ff9700 1 -- 172.21.15.113:0/244664935 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8790013070 con 0x7f879c076620 2024-04-02T14:35:22.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:22.301+0000 7f87a1db7700 1 -- 172.21.15.113:0/244664935 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f879c062a30 con 0x7f87840a3c30 2024-04-02T14:35:23.221 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:22 smithi113 ceph-mon[36135]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:23.221 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:35:23.221 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:35:23.221 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:23.221 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:23.221 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:23.221 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:35:23.221 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:23.221 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:23.221 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:23.221 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:23.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:22 smithi204 ceph-mon[35948]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:23.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:35:23.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:35:23.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:23.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:23.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:23.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:35:23.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:23.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:23.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:23.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:23.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:22 smithi062 ceph-mon[32116]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:23.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:35:23.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:35:23.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:23.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:23.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:23.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:35:23.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:23.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:23.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:23.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:24.183 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:23 smithi113 ceph-mon[36135]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:35:24.183 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:23 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi062 2024-04-02T14:35:24.184 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:23 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi062 to 17091M 2024-04-02T14:35:24.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:23 smithi204 ceph-mon[35948]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:35:24.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:23 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi062 2024-04-02T14:35:24.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:23 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi062 to 17091M 2024-04-02T14:35:24.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:23 smithi062 ceph-mon[32116]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:35:24.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:23 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi062 2024-04-02T14:35:24.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:23 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi062 to 17091M 2024-04-02T14:35:25.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:24 smithi204 ceph-mon[35948]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:25.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:24 smithi062 ceph-mon[32116]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:25.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:24 smithi113 ceph-mon[36135]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:26.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:25 smithi204 ceph-mon[35948]: from='client.? 172.21.15.113:0/50714595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5"}]: dispatch 2024-04-02T14:35:26.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:25 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5"}]: dispatch 2024-04-02T14:35:26.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:25 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5"}]': finished 2024-04-02T14:35:26.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:25 smithi204 ceph-mon[35948]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T14:35:26.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:25 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:26.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:25 smithi204 ceph-mon[35948]: from='client.? 172.21.15.113:0/2544007577' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:35:26.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:25 smithi062 ceph-mon[32116]: from='client.? 172.21.15.113:0/50714595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5"}]: dispatch 2024-04-02T14:35:26.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:25 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5"}]: dispatch 2024-04-02T14:35:26.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:25 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5"}]': finished 2024-04-02T14:35:26.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:25 smithi062 ceph-mon[32116]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T14:35:26.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:25 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:26.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:25 smithi062 ceph-mon[32116]: from='client.? 172.21.15.113:0/2544007577' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:35:26.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:25 smithi113 ceph-mon[36135]: from='client.? 172.21.15.113:0/50714595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5"}]: dispatch 2024-04-02T14:35:26.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:25 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5"}]: dispatch 2024-04-02T14:35:26.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:25 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5"}]': finished 2024-04-02T14:35:26.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:25 smithi113 ceph-mon[36135]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T14:35:26.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:25 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:26.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:25 smithi113 ceph-mon[36135]: from='client.? 172.21.15.113:0/2544007577' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:35:27.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:27 smithi062 ceph-mon[32116]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:27.376 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:27 smithi113 ceph-mon[36135]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:27.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:27 smithi204 ceph-mon[35948]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:29.291 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:29 smithi113 ceph-mon[36135]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:29.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:29 smithi062 ceph-mon[32116]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:29.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:29 smithi204 ceph-mon[35948]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:31.312 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:31 smithi204 ceph-mon[35948]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:31.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:31 smithi113 ceph-mon[36135]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:31.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:31 smithi062 ceph-mon[32116]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:33.257 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:33 smithi113 ceph-mon[36135]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:33.257 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T14:35:33.257 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:33.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:33 smithi062 ceph-mon[32116]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:33.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T14:35:33.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:33.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:33 smithi204 ceph-mon[35948]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:33.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T14:35:33.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:34.198 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:34 smithi113 ceph-mon[36135]: Deploying daemon osd.1 on smithi113 2024-04-02T14:35:34.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:34 smithi062 ceph-mon[32116]: Deploying daemon osd.1 on smithi113 2024-04-02T14:35:34.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:34 smithi204 ceph-mon[35948]: Deploying daemon osd.1 on smithi113 2024-04-02T14:35:35.101 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:35 smithi113 ceph-mon[36135]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:35.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:35 smithi062 ceph-mon[32116]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:35.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:35 smithi204 ceph-mon[35948]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:37.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:37 smithi113 ceph-mon[36135]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:37.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:37 smithi062 ceph-mon[32116]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:37.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:37 smithi204 ceph-mon[35948]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:39.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:38 smithi204 ceph-mon[35948]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:39.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:38 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:39.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:38 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:39.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:38 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:39.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:38 smithi062 ceph-mon[32116]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:39.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:38 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:39.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:38 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:39.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:38 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:39.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:38 smithi113 ceph-mon[36135]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:39.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:38 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:39.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:38 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:39.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:38 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:40.987 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:40 smithi113 ceph-mon[36135]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:40.988 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:40.988 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:40.988 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:40.988 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:40.988 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:41.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:40 smithi204 ceph-mon[35948]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:41.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:41.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:41.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:41.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:41.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:41.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:40 smithi062 ceph-mon[32116]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:41.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.347+0000 7f8798ff9700 1 -- 172.21.15.113:0/244664935 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f879c062a30 con 0x7f87840a3c30 2024-04-02T14:35:41.349 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 1 on host 'smithi113' 2024-04-02T14:35:41.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 -- 172.21.15.113:0/244664935 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87840a3c30 msgr2=0x7f87840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:35:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87840a3c30 0x7f87840a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f878c006fd0 tx=0x7f878c008040 comp rx=0 tx=0).stop 2024-04-02T14:35:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 -- 172.21.15.113:0/244664935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c076620 msgr2=0x7f879c19b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:35:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c076620 0x7f879c19b170 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8790000c00 tx=0x7f879000f740 comp rx=0 tx=0).stop 2024-04-02T14:35:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 -- 172.21.15.113:0/244664935 shutdown_connections 2024-04-02T14:35:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f87840a3c30 0x7f87840a60f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f879c10a510 0x7f879c1a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f879c075050 0x7f879c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 --2- 172.21.15.113:0/244664935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c076620 0x7f879c19b170 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 -- 172.21.15.113:0/244664935 >> 172.21.15.113:0/244664935 conn(0x7f879c0fdc00 msgr2=0x7f879c106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:35:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 -- 172.21.15.113:0/244664935 shutdown_connections 2024-04-02T14:35:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:35:41.350+0000 7f87a1db7700 1 -- 172.21.15.113:0/244664935 wait complete. 2024-04-02T14:35:42.059 DEBUG:teuthology.orchestra.run.smithi113:osd.1> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.1.service 2024-04-02T14:35:42.062 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-04-02T14:35:42.062 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T14:35:42.307 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:35:42.430 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:42 smithi204 ceph-mon[35948]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:42.430 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:42.430 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:42.431 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:42.431 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:42 smithi204 ceph-mon[35948]: from='osd.1 [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T14:35:42.431 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:42 smithi204 ceph-mon[35948]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T14:35:42.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:42 smithi062 ceph-mon[32116]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:42.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:42.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:42.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:42.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:42 smithi062 ceph-mon[32116]: from='osd.1 [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T14:35:42.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:42 smithi062 ceph-mon[32116]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T14:35:42.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:42 smithi113 ceph-mon[36135]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:42.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:42.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:35:42.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:42.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:42 smithi113 ceph-mon[36135]: from='osd.1 [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T14:35:42.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:42 smithi113 ceph-mon[36135]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T14:35:43.537 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:43 smithi113 ceph-mon[36135]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T14:35:43.538 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:43 smithi113 ceph-mon[36135]: from='osd.1 [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:35:43.538 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:43 smithi113 ceph-mon[36135]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T14:35:43.538 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:43 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:43.538 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:43 smithi113 ceph-mon[36135]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:35:43.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:43 smithi062 ceph-mon[32116]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T14:35:43.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:43 smithi062 ceph-mon[32116]: from='osd.1 [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:35:43.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:43 smithi062 ceph-mon[32116]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T14:35:43.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:43 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:43.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:43 smithi062 ceph-mon[32116]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:35:43.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:43 smithi204 ceph-mon[35948]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T14:35:43.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:43 smithi204 ceph-mon[35948]: from='osd.1 [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:35:43.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:43 smithi204 ceph-mon[35948]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T14:35:43.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:43 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:43.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:43 smithi204 ceph-mon[35948]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:35:44.121 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:35:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1[40719]: 2024-04-02T14:35:44.096+0000 7fb47da65700 -1 osd.1 0 waiting for initial osdmap 2024-04-02T14:35:44.230 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T14:35:44.230 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T14:35:44.230 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-04-02T14:35:44.230 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-04-02T14:35:44.230 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130915 s, 801 MB/s 2024-04-02T14:35:44.230 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-04-02T14:35:44.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:44 smithi204 ceph-mon[35948]: pgmap v73: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:44.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:44 smithi204 ceph-mon[35948]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:35:44.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:44 smithi204 ceph-mon[35948]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T14:35:44.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:44 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:44.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:44 smithi204 ceph-mon[35948]: from='osd.1 ' entity='osd.1' 2024-04-02T14:35:44.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:44 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:44.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:44 smithi113 ceph-mon[36135]: pgmap v73: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:44.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:44 smithi113 ceph-mon[36135]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:35:44.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:44 smithi113 ceph-mon[36135]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T14:35:44.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:44 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:44.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:44 smithi113 ceph-mon[36135]: from='osd.1 ' entity='osd.1' 2024-04-02T14:35:44.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:44 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:44.622 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:35:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1[40719]: 2024-04-02T14:35:44.176+0000 7fb475850700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:35:44.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:44 smithi062 ceph-mon[32116]: pgmap v73: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:44.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:44 smithi062 ceph-mon[32116]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:35:44.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:44 smithi062 ceph-mon[32116]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T14:35:44.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:44 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:44.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:44 smithi062 ceph-mon[32116]: from='osd.1 ' entity='osd.1' 2024-04-02T14:35:44.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:44 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:44.885 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-04-02T14:35:45.126 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:35:45.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:35:45.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:35:45.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi113 2024-04-02T14:35:45.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: osd.1 [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295] boot 2024-04-02T14:35:45.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T14:35:45.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:45.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:45.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:45.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:35:45.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi113 to 17080M 2024-04-02T14:35:45.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:45.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:45.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:45.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:45 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:45.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:35:45.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:35:45.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi113 2024-04-02T14:35:45.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: osd.1 [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295] boot 2024-04-02T14:35:45.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T14:35:45.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:45.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:45.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:45.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:35:45.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi113 to 17080M 2024-04-02T14:35:45.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:45.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:45.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:45.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:45 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi113 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: osd.1 [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295] boot 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi113 to 17080M 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:45.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:45.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:35:45.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:45 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:35:46.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:46 smithi204 ceph-mon[35948]: pgmap v76: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:46.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:46 smithi113 ceph-mon[36135]: pgmap v76: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:46.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.677+0000 7eff7547b700 1 -- 172.21.15.204:0/3930048007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff70101520 msgr2=0x7eff70101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:35:46.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.677+0000 7eff7547b700 1 --2- 172.21.15.204:0/3930048007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff70101520 0x7eff70101940 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff60009990 tx=0x7eff60009ca0 comp rx=0 tx=0).stop 2024-04-02T14:35:46.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.677+0000 7eff7547b700 1 -- 172.21.15.204:0/3930048007 shutdown_connections 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.677+0000 7eff7547b700 1 --2- 172.21.15.204:0/3930048007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff701030c0 0x7eff70109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.677+0000 7eff7547b700 1 --2- 172.21.15.204:0/3930048007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff70101520 0x7eff70101940 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.677+0000 7eff7547b700 1 --2- 172.21.15.204:0/3930048007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff70102720 0x7eff70102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.677+0000 7eff7547b700 1 -- 172.21.15.204:0/3930048007 >> 172.21.15.204:0/3930048007 conn(0x7eff700fcac0 msgr2=0x7eff700fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.677+0000 7eff7547b700 1 -- 172.21.15.204:0/3930048007 shutdown_connections 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.677+0000 7eff7547b700 1 -- 172.21.15.204:0/3930048007 wait complete. 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.678+0000 7eff7547b700 1 Processor -- start 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.678+0000 7eff7547b700 1 -- start start 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.678+0000 7eff7547b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70101520 0x7eff70197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.678+0000 7eff7547b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff70102720 0x7eff70197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.678+0000 7eff7547b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff701030c0 0x7eff7019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:35:46.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.678+0000 7eff7547b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7eff7010d320 con 0x7eff701030c0 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.678+0000 7eff7547b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7eff7010d1a0 con 0x7eff70101520 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.678+0000 7eff7547b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff7010d4a0 con 0x7eff70102720 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.678+0000 7eff6e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff70102720 0x7eff70197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.678+0000 7eff6e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff70102720 0x7eff70197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:45376/0 (socket says 172.21.15.204:45376) 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.678+0000 7eff6e7fc700 1 -- 172.21.15.204:0/557324956 learned_addr learned my addr 172.21.15.204:0/557324956 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff6effd700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70101520 0x7eff70197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff6e7fc700 1 -- 172.21.15.204:0/557324956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70101520 msgr2=0x7eff70197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff6e7fc700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70101520 0x7eff70197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff6e7fc700 1 -- 172.21.15.204:0/557324956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff701030c0 msgr2=0x7eff7019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff6e7fc700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff701030c0 0x7eff7019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff6e7fc700 1 -- 172.21.15.204:0/557324956 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7019f5a0 con 0x7eff70102720 2024-04-02T14:35:46.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff6e7fc700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff70102720 0x7eff70197e80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7eff6400da10 tx=0x7eff6400ddd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:35:46.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff57fff700 1 -- 172.21.15.204:0/557324956 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6401b070 con 0x7eff70102720 2024-04-02T14:35:46.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff57fff700 1 -- 172.21.15.204:0/557324956 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff6400ecd0 con 0x7eff70102720 2024-04-02T14:35:46.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff57fff700 1 -- 172.21.15.204:0/557324956 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff640147d0 con 0x7eff70102720 2024-04-02T14:35:46.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff7547b700 1 -- 172.21.15.204:0/557324956 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7006a300 con 0x7eff70102720 2024-04-02T14:35:46.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.679+0000 7eff7547b700 1 -- 172.21.15.204:0/557324956 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff7006a870 con 0x7eff70102720 2024-04-02T14:35:46.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.680+0000 7eff7547b700 1 -- 172.21.15.204:0/557324956 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff700692b0 con 0x7eff70102720 2024-04-02T14:35:46.685 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.681+0000 7eff57fff700 1 -- 172.21.15.204:0/557324956 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7eff6401e070 con 0x7eff70102720 2024-04-02T14:35:46.685 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.682+0000 7eff57fff700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff580a3bb0 0x7eff580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:35:46.685 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.682+0000 7eff57fff700 1 -- 172.21.15.204:0/557324956 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7eff640d0650 con 0x7eff70102720 2024-04-02T14:35:46.685 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.682+0000 7eff6effd700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff580a3bb0 0x7eff580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:35:46.685 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.683+0000 7eff6effd700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff580a3bb0 0x7eff580a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7eff6000b5c0 tx=0x7eff60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:35:46.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.686+0000 7eff57fff700 1 -- 172.21.15.204:0/557324956 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7eff6409e880 con 0x7eff70102720 2024-04-02T14:35:46.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:46 smithi062 ceph-mon[32116]: pgmap v76: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T14:35:46.893 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:35:46.891+0000 7eff7547b700 1 -- 172.21.15.204:0/557324956 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7eff7006ab30 con 0x7eff580a3bb0 2024-04-02T14:35:47.624 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:47 smithi113 ceph-mon[36135]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T14:35:47.624 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:47 smithi113 ceph-mon[36135]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:35:47.625 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:35:47.625 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:35:47.625 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:47.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:47 smithi204 ceph-mon[35948]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T14:35:47.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:47 smithi204 ceph-mon[35948]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:35:47.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:35:47.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:35:47.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:47.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:47 smithi062 ceph-mon[32116]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T14:35:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:47 smithi062 ceph-mon[32116]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:35:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:35:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:35:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:48.625 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:48 smithi204 ceph-mon[35948]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:48.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:48 smithi062 ceph-mon[32116]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:48.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:48 smithi113 ceph-mon[36135]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:50.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:50 smithi204 ceph-mon[35948]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:50.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:50 smithi204 ceph-mon[35948]: from='client.? 172.21.15.204:0/2849951136' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26dd6e9a-9371-40b8-bdb7-909b89f97877"}]: dispatch 2024-04-02T14:35:50.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:50 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26dd6e9a-9371-40b8-bdb7-909b89f97877"}]: dispatch 2024-04-02T14:35:50.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:50 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26dd6e9a-9371-40b8-bdb7-909b89f97877"}]': finished 2024-04-02T14:35:50.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:50 smithi204 ceph-mon[35948]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T14:35:50.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:50 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:35:50.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:50 smithi204 ceph-mon[35948]: from='client.? 172.21.15.204:0/686673175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:35:50.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:50 smithi062 ceph-mon[32116]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:50 smithi062 ceph-mon[32116]: from='client.? 172.21.15.204:0/2849951136' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26dd6e9a-9371-40b8-bdb7-909b89f97877"}]: dispatch 2024-04-02T14:35:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:50 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26dd6e9a-9371-40b8-bdb7-909b89f97877"}]: dispatch 2024-04-02T14:35:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:50 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26dd6e9a-9371-40b8-bdb7-909b89f97877"}]': finished 2024-04-02T14:35:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:50 smithi062 ceph-mon[32116]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T14:35:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:50 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:35:50.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:50 smithi062 ceph-mon[32116]: from='client.? 172.21.15.204:0/686673175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:35:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:50 smithi113 ceph-mon[36135]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:50 smithi113 ceph-mon[36135]: from='client.? 172.21.15.204:0/2849951136' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26dd6e9a-9371-40b8-bdb7-909b89f97877"}]: dispatch 2024-04-02T14:35:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:50 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26dd6e9a-9371-40b8-bdb7-909b89f97877"}]: dispatch 2024-04-02T14:35:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:50 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26dd6e9a-9371-40b8-bdb7-909b89f97877"}]': finished 2024-04-02T14:35:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:50 smithi113 ceph-mon[36135]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T14:35:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:50 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:35:50.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:50 smithi113 ceph-mon[36135]: from='client.? 172.21.15.204:0/686673175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:35:52.647 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:52 smithi062 ceph-mon[32116]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:52.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:52 smithi204 ceph-mon[35948]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:52.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:52 smithi113 ceph-mon[36135]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:54.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:54 smithi113 ceph-mon[36135]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:54.685 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:54 smithi204 ceph-mon[35948]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:54.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:54 smithi062 ceph-mon[32116]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:56.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:56 smithi204 ceph-mon[35948]: pgmap v83: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:56.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:56 smithi062 ceph-mon[32116]: pgmap v83: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:56.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:56 smithi113 ceph-mon[36135]: pgmap v83: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:57.682 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:57 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T14:35:57.682 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:57 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:57.682 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:57 smithi204 ceph-mon[35948]: Deploying daemon osd.2 on smithi204 2024-04-02T14:35:57.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:57 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T14:35:57.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:57 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:57.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:57 smithi062 ceph-mon[32116]: Deploying daemon osd.2 on smithi204 2024-04-02T14:35:57.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:57 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T14:35:57.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:57 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:35:57.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:57 smithi113 ceph-mon[36135]: Deploying daemon osd.2 on smithi204 2024-04-02T14:35:58.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:35:58 smithi204 ceph-mon[35948]: pgmap v84: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:58.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:35:58 smithi062 ceph-mon[32116]: pgmap v84: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:35:58.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:35:58 smithi113 ceph-mon[36135]: pgmap v84: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:00.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:00 smithi204 ceph-mon[35948]: pgmap v85: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:00.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:00 smithi062 ceph-mon[32116]: pgmap v85: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:00.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:00 smithi113 ceph-mon[36135]: pgmap v85: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:03.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:03 smithi204 ceph-mon[35948]: pgmap v86: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:03.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:03 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:03.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:03 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:03.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:03 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:03.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:03 smithi062 ceph-mon[32116]: pgmap v86: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:03.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:03 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:03.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:03 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:03.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:03 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:03.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:03 smithi113 ceph-mon[36135]: pgmap v86: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:03.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:03 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:03.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:03 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:03.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:03 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:05.430 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:05 smithi204 ceph-mon[35948]: pgmap v87: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:05.526 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.524+0000 7eff57fff700 1 -- 172.21.15.204:0/557324956 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff7006ab30 con 0x7eff580a3bb0 2024-04-02T14:36:05.526 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-04-02T14:36:05.528 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 -- 172.21.15.204:0/557324956 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff580a3bb0 msgr2=0x7eff580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:36:05.528 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff580a3bb0 0x7eff580a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7eff6000b5c0 tx=0x7eff60005fb0 comp rx=0 tx=0).stop 2024-04-02T14:36:05.528 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 -- 172.21.15.204:0/557324956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff70102720 msgr2=0x7eff70197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:36:05.528 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff70102720 0x7eff70197e80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7eff6400da10 tx=0x7eff6400ddd0 comp rx=0 tx=0).stop 2024-04-02T14:36:05.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 -- 172.21.15.204:0/557324956 shutdown_connections 2024-04-02T14:36:05.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff580a3bb0 0x7eff580a6070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:05.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70101520 0x7eff70197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:05.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff701030c0 0x7eff7019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:05.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 --2- 172.21.15.204:0/557324956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff70102720 0x7eff70197e80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:05.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 -- 172.21.15.204:0/557324956 >> 172.21.15.204:0/557324956 conn(0x7eff700fcac0 msgr2=0x7eff70105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:36:05.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 -- 172.21.15.204:0/557324956 shutdown_connections 2024-04-02T14:36:05.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:36:05.527+0000 7eff7547b700 1 -- 172.21.15.204:0/557324956 wait complete. 2024-04-02T14:36:05.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:05 smithi062 ceph-mon[32116]: pgmap v87: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:05.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:05 smithi113 ceph-mon[36135]: pgmap v87: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:06.179 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.2.service 2024-04-02T14:36:06.182 INFO:tasks.cephadm:Deploying osd.3 on smithi062 with /dev/vg_nvme/lv_3... 2024-04-02T14:36:06.182 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T14:36:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:06 smithi062 ceph-mon[32116]: pgmap v88: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:06 smithi062 ceph-mon[32116]: from='osd.2 [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T14:36:06.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:06 smithi062 ceph-mon[32116]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T14:36:06.655 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:36:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:06 smithi204 ceph-mon[35948]: pgmap v88: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:06 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:06 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:06 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:06 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:06 smithi204 ceph-mon[35948]: from='osd.2 [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T14:36:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:06 smithi204 ceph-mon[35948]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T14:36:06.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:06 smithi113 ceph-mon[36135]: pgmap v88: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:06.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:06 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:06 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:06 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:06 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:06.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:06 smithi113 ceph-mon[36135]: from='osd.2 [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T14:36:06.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:06 smithi113 ceph-mon[36135]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T14:36:07.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:07 smithi062 ceph-mon[32116]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T14:36:07.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:07 smithi062 ceph-mon[32116]: from='osd.2 [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:36:07.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:07 smithi062 ceph-mon[32116]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T14:36:07.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:07 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:07.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:07 smithi062 ceph-mon[32116]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:36:07.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:07 smithi113 ceph-mon[36135]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T14:36:07.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:07 smithi113 ceph-mon[36135]: from='osd.2 [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:36:07.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:07 smithi113 ceph-mon[36135]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T14:36:07.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:07 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:07.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:07 smithi113 ceph-mon[36135]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:36:08.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:07 smithi204 ceph-mon[35948]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T14:36:08.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:07 smithi204 ceph-mon[35948]: from='osd.2 [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:36:08.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:07 smithi204 ceph-mon[35948]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T14:36:08.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:07 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:08.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:07 smithi204 ceph-mon[35948]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:36:08.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:08 smithi062 ceph-mon[32116]: pgmap v90: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:08.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:08 smithi062 ceph-mon[32116]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:36:08.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:08 smithi062 ceph-mon[32116]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T14:36:08.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:08 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:08.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:08 smithi062 ceph-mon[32116]: from='osd.2 ' entity='osd.2' 2024-04-02T14:36:08.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:08 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:08.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:08 smithi204 ceph-mon[35948]: pgmap v90: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:08.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:08 smithi204 ceph-mon[35948]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:36:08.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:08 smithi204 ceph-mon[35948]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T14:36:08.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:08 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:08.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:08 smithi204 ceph-mon[35948]: from='osd.2 ' entity='osd.2' 2024-04-02T14:36:08.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:08 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:08.865 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:36:08 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2[41105]: 2024-04-02T14:36:08.513+0000 7fbe5f193700 -1 osd.2 0 waiting for initial osdmap 2024-04-02T14:36:08.865 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:36:08 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2[41105]: 2024-04-02T14:36:08.615+0000 7fbe56f7e700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:36:08.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:08 smithi113 ceph-mon[36135]: pgmap v90: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:08.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:08 smithi113 ceph-mon[36135]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:36:08.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:08 smithi113 ceph-mon[36135]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T14:36:08.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:08 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:08.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:08 smithi113 ceph-mon[36135]: from='osd.2 ' entity='osd.2' 2024-04-02T14:36:08.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:08 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T14:36:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T14:36:09.463 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-02T14:36:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-02T14:36:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0398102 s, 263 MB/s 2024-04-02T14:36:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-02T14:36:09.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:09.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:09.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:09.917 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:36:09.917 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:36:09.917 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:09.917 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:09.917 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:36:09.917 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:09.917 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:09.917 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:09.917 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:09.917 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:09.917 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:09.918 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:09.918 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:09.918 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:10.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:10.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:10.323 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2024-04-02T14:36:10.790 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:36:10.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:10 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi204 2024-04-02T14:36:10.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:10 smithi113 ceph-mon[36135]: pgmap v92: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:10.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:10 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi204 to 17080M 2024-04-02T14:36:10.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:10 smithi113 ceph-mon[36135]: osd.2 [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144] boot 2024-04-02T14:36:10.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:10 smithi113 ceph-mon[36135]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T14:36:10.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:10 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:10.940 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:10 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi204 2024-04-02T14:36:10.941 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:10 smithi062 ceph-mon[32116]: pgmap v92: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:10.941 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:10 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi204 to 17080M 2024-04-02T14:36:10.941 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:10 smithi062 ceph-mon[32116]: osd.2 [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144] boot 2024-04-02T14:36:10.941 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:10 smithi062 ceph-mon[32116]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T14:36:10.941 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:10 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:11.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:10 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi204 2024-04-02T14:36:11.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:10 smithi204 ceph-mon[35948]: pgmap v92: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-02T14:36:11.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:10 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi204 to 17080M 2024-04-02T14:36:11.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:10 smithi204 ceph-mon[35948]: osd.2 [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144] boot 2024-04-02T14:36:11.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:10 smithi204 ceph-mon[35948]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T14:36:11.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:10 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:36:11.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:11 smithi113 ceph-mon[36135]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T14:36:11.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:11 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:36:11.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:11 smithi062 ceph-mon[32116]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T14:36:11.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:11 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:36:12.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:11 smithi204 ceph-mon[35948]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T14:36:12.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:11 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:36:12.796 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:12 smithi062 ceph-mon[32116]: pgmap v95: 0 pgs: ; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:12.796 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T14:36:12.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:12 smithi062 ceph-mon[32116]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T14:36:12.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:36:12.978 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:12 smithi113 ceph-mon[36135]: pgmap v95: 0 pgs: ; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:12.978 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T14:36:12.978 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:12 smithi113 ceph-mon[36135]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T14:36:12.979 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:36:12.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.981+0000 7f0470400700 1 -- 172.21.15.62:0/3770437482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04681008e0 msgr2=0x7f0468100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:36:12.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.981+0000 7f0470400700 1 --2- 172.21.15.62:0/3770437482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04681008e0 0x7f0468100d00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f045c009990 tx=0x7f045c009ca0 comp rx=0 tx=0).stop 2024-04-02T14:36:12.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.981+0000 7f0470400700 1 -- 172.21.15.62:0/3770437482 shutdown_connections 2024-04-02T14:36:12.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.981+0000 7f0470400700 1 --2- 172.21.15.62:0/3770437482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0468104260 0x7f046810ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:12.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.981+0000 7f0470400700 1 --2- 172.21.15.62:0/3770437482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04681008e0 0x7f0468100d00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:12.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.981+0000 7f0470400700 1 --2- 172.21.15.62:0/3770437482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04681038c0 0x7f0468103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:12.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.981+0000 7f0470400700 1 -- 172.21.15.62:0/3770437482 >> 172.21.15.62:0/3770437482 conn(0x7f04680fc260 msgr2=0x7f04680fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:36:12.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.981+0000 7f0470400700 1 -- 172.21.15.62:0/3770437482 shutdown_connections 2024-04-02T14:36:12.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.982+0000 7f0470400700 1 -- 172.21.15.62:0/3770437482 wait complete. 2024-04-02T14:36:12.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.982+0000 7f0470400700 1 Processor -- start 2024-04-02T14:36:12.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f0470400700 1 -- start start 2024-04-02T14:36:12.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f0470400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04681008e0 0x7f0468103220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:36:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f0470400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04681038c0 0x7f0468101870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:36:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f0470400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0468104260 0x7f0468101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:36:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f0470400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f046810e310 con 0x7f0468104260 2024-04-02T14:36:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f0470400700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f046810e190 con 0x7f04681038c0 2024-04-02T14:36:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f0470400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f046810e490 con 0x7f04681008e0 2024-04-02T14:36:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f046d000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04681038c0 0x7f0468101870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:36:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f046d000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04681038c0 0x7f0468101870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:51124/0 (socket says 172.21.15.62:51124) 2024-04-02T14:36:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f046e400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0468104260 0x7f0468101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:36:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f046d000700 1 -- 172.21.15.62:0/2964426168 learned_addr learned my addr 172.21.15.62:0/2964426168 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:36:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.983+0000 7f046da00700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04681008e0 0x7f0468103220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:36:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.984+0000 7f046d000700 1 -- 172.21.15.62:0/2964426168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04681008e0 msgr2=0x7f0468103220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:36:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.984+0000 7f046d000700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04681008e0 0x7f0468103220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.984+0000 7f046d000700 1 -- 172.21.15.62:0/2964426168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0468104260 msgr2=0x7f0468101db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:36:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.984+0000 7f046d000700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0468104260 0x7f0468101db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.984+0000 7f046d000700 1 -- 172.21.15.62:0/2964426168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0468102660 con 0x7f04681038c0 2024-04-02T14:36:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.984+0000 7f046da00700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04681008e0 0x7f0468103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:36:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.984+0000 7f046d000700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04681038c0 0x7f0468101870 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f045400e9a0 tx=0x7f045400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:36:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.985+0000 7f0466000700 1 -- 172.21.15.62:0/2964426168 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045400ca10 con 0x7f04681038c0 2024-04-02T14:36:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.985+0000 7f0466000700 1 -- 172.21.15.62:0/2964426168 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f045400cbb0 con 0x7f04681038c0 2024-04-02T14:36:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.985+0000 7f0470400700 1 -- 172.21.15.62:0/2964426168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04681a2a30 con 0x7f04681038c0 2024-04-02T14:36:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.985+0000 7f0466000700 1 -- 172.21.15.62:0/2964426168 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454018710 con 0x7f04681038c0 2024-04-02T14:36:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.985+0000 7f0470400700 1 -- 172.21.15.62:0/2964426168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04681a2f70 con 0x7f04681038c0 2024-04-02T14:36:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.986+0000 7f0470400700 1 -- 172.21.15.62:0/2964426168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04680692b0 con 0x7f04681038c0 2024-04-02T14:36:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.988+0000 7f0466000700 1 -- 172.21.15.62:0/2964426168 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f0454010430 con 0x7f04681038c0 2024-04-02T14:36:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.989+0000 7f0466000700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04500a3cb0 0x7f04500a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:36:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.989+0000 7f0466000700 1 -- 172.21.15.62:0/2964426168 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f04540cf0e0 con 0x7f04681038c0 2024-04-02T14:36:12.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.990+0000 7f046da00700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04500a3cb0 0x7f04500a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:36:12.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.990+0000 7f046da00700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04500a3cb0 0x7f04500a6170 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f045c00b820 tx=0x7f045c00b8f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:36:12.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:12.994+0000 7f0466000700 1 -- 172.21.15.62:0/2964426168 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f045409ced0 con 0x7f04681038c0 2024-04-02T14:36:13.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:12 smithi204 ceph-mon[35948]: pgmap v95: 0 pgs: ; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:13.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T14:36:13.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:12 smithi204 ceph-mon[35948]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T14:36:13.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:36:13.095 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:36:12 smithi062 sudo[49299]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T14:36:13.095 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:36:12 smithi062 sudo[49299]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.095 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:36:12 smithi062 sudo[49299]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.095 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:36:12 smithi062 sudo[49299]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.095 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:36:12 smithi062 sudo[49302]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T14:36:13.095 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:36:12 smithi062 sudo[49302]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.095 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:36:12 smithi062 sudo[49302]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.095 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:36:12 smithi062 sudo[49302]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:13.255+0000 7f0470400700 1 -- 172.21.15.62:0/2964426168 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0468062af0 con 0x7f04500a3cb0 2024-04-02T14:36:13.325 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:36:12 smithi113 sudo[44898]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T14:36:13.325 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:36:12 smithi113 sudo[44898]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.325 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:36:12 smithi113 sudo[44898]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.326 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:36:12 smithi113 sudo[44898]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.326 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44902]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T14:36:13.326 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44902]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.326 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44902]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.326 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44902]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.411 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45065]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T14:36:13.411 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45065]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.411 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45065]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.411 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45065]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.411 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45068]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T14:36:13.411 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45068]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.411 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45068]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.411 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45068]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45071]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T14:36:13.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45071]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45071]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45071]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45074]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T14:36:13.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45074]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45074]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 sudo[45074]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 sudo[49325]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T14:36:13.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 sudo[49325]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 sudo[49325]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 sudo[49325]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 sudo[49328]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T14:36:13.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 sudo[49328]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 sudo[49328]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 sudo[49328]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44905]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T14:36:13.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44905]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44905]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44905]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44908]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T14:36:13.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44908]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T14:36:13.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44908]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T14:36:13.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 sudo[44908]: pam_unix(sudo:session): session closed for user root 2024-04-02T14:36:13.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T14:36:13.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T14:36:13.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:13.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:13.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:13.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T14:36:13.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T14:36:13.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:36:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:36:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:36:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T14:36:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:36:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:36:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:36:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:36:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T14:36:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T14:36:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:36:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:36:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:36:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:36:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:13 smithi204 ceph-mon[35948]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T14:36:14.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:36:14.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:36:14.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:36:14.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:36:14.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T14:36:14.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T14:36:14.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:36:14.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:36:14.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:36:14.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:36:14.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:14.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:13 smithi062 ceph-mon[32116]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T14:36:14.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T14:36:14.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T14:36:14.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:14.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:14.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:14.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T14:36:14.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T14:36:14.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:36:14.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:36:14.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:36:14.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T14:36:14.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:36:14.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:36:14.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:36:14.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:36:14.124 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T14:36:14.124 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T14:36:14.124 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:36:14.124 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:36:14.124 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:36:14.124 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:36:14.124 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:14.124 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:13 smithi113 ceph-mon[36135]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T14:36:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:14.131+0000 7f0466000700 1 -- 172.21.15.62:0/2964426168 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0454098e40 con 0x7f04681038c0 2024-04-02T14:36:14.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:14 smithi062 ceph-mon[32116]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:36:14.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:14 smithi062 ceph-mon[32116]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:14.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:14 smithi062 ceph-mon[32116]: osdmap e22: 3 total, 3 up, 3 in 2024-04-02T14:36:14.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:14 smithi062 ceph-mon[32116]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T14:36:15.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:14 smithi204 ceph-mon[35948]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:36:15.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:14 smithi204 ceph-mon[35948]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:15.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:14 smithi204 ceph-mon[35948]: osdmap e22: 3 total, 3 up, 3 in 2024-04-02T14:36:15.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:14 smithi204 ceph-mon[35948]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T14:36:15.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:14 smithi113 ceph-mon[36135]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:36:15.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:14 smithi113 ceph-mon[36135]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:15.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:14 smithi113 ceph-mon[36135]: osdmap e22: 3 total, 3 up, 3 in 2024-04-02T14:36:15.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:14 smithi113 ceph-mon[36135]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T14:36:17.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:16 smithi204 ceph-mon[35948]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 105 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:17.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:16 smithi062 ceph-mon[32116]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 105 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:17.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:16 smithi113 ceph-mon[36135]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 105 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:17.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:17 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1921067733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8017fd90-25ff-460d-8273-c49a205ae472"}]: dispatch 2024-04-02T14:36:17.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:17 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8017fd90-25ff-460d-8273-c49a205ae472"}]: dispatch 2024-04-02T14:36:17.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:17 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8017fd90-25ff-460d-8273-c49a205ae472"}]': finished 2024-04-02T14:36:17.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:17 smithi062 ceph-mon[32116]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T14:36:17.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:17 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:18.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:17 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1921067733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8017fd90-25ff-460d-8273-c49a205ae472"}]: dispatch 2024-04-02T14:36:18.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:17 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8017fd90-25ff-460d-8273-c49a205ae472"}]: dispatch 2024-04-02T14:36:18.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:17 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8017fd90-25ff-460d-8273-c49a205ae472"}]': finished 2024-04-02T14:36:18.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:17 smithi204 ceph-mon[35948]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T14:36:18.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:17 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:18.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:17 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1921067733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8017fd90-25ff-460d-8273-c49a205ae472"}]: dispatch 2024-04-02T14:36:18.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:17 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8017fd90-25ff-460d-8273-c49a205ae472"}]: dispatch 2024-04-02T14:36:18.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:17 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8017fd90-25ff-460d-8273-c49a205ae472"}]': finished 2024-04-02T14:36:18.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:17 smithi113 ceph-mon[36135]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T14:36:18.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:17 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:19.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:18 smithi204 ceph-mon[35948]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:19.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:18 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2351387623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:36:19.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:18 smithi062 ceph-mon[32116]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:19.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:18 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2351387623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:36:19.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:18 smithi113 ceph-mon[36135]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:19.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:18 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2351387623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:36:21.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:20 smithi204 ceph-mon[35948]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:21.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:20 smithi062 ceph-mon[32116]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:21.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:20 smithi113 ceph-mon[36135]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:23.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:22 smithi204 ceph-mon[35948]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:23.062 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:22 smithi113 ceph-mon[36135]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:23.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:22 smithi062 ceph-mon[32116]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:24.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:24 smithi062 ceph-mon[32116]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:25.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:24 smithi204 ceph-mon[35948]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:25.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:24 smithi113 ceph-mon[36135]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:27.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:26 smithi204 ceph-mon[35948]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:27.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T14:36:27.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:27.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:26 smithi062 ceph-mon[32116]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:27.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T14:36:27.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:27.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:26 smithi113 ceph-mon[36135]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:27.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T14:36:27.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:28.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:27 smithi204 ceph-mon[35948]: Deploying daemon osd.3 on smithi062 2024-04-02T14:36:28.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:27 smithi062 ceph-mon[32116]: Deploying daemon osd.3 on smithi062 2024-04-02T14:36:28.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:27 smithi113 ceph-mon[36135]: Deploying daemon osd.3 on smithi062 2024-04-02T14:36:28.783 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:28 smithi062 ceph-mon[32116]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:29.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:28 smithi204 ceph-mon[35948]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:29.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:28 smithi113 ceph-mon[36135]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:31.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:30 smithi062 ceph-mon[32116]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:31.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:30 smithi113 ceph-mon[36135]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:31.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:30 smithi204 ceph-mon[35948]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:33.062 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:32 smithi204 ceph-mon[35948]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:33.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:32 smithi062 ceph-mon[32116]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:33.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:32 smithi113 ceph-mon[36135]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:34.952 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:34 smithi062 ceph-mon[32116]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:35.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:34 smithi113 ceph-mon[36135]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:35.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:34 smithi204 ceph-mon[35948]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:36.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:35 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:36.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:35 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:36.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:35 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:36.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:35 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:36.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:35 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:36.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:35 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:36.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:35 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:36.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:35 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:36.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:35 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:37.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:36 smithi062 ceph-mon[32116]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:37.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:36 smithi113 ceph-mon[36135]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:37.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:36 smithi204 ceph-mon[35948]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:39.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:38 smithi062 ceph-mon[32116]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:39.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:38 smithi113 ceph-mon[36135]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:39.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:38 smithi204 ceph-mon[35948]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:40.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:40.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:40.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:40 smithi062 ceph-mon[32116]: from='osd.3 [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T14:36:40.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:40 smithi062 ceph-mon[32116]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T14:36:40.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:40.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:40.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:40 smithi113 ceph-mon[36135]: from='osd.3 [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T14:36:40.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:40 smithi113 ceph-mon[36135]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T14:36:40.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.461+0000 7f0466000700 1 -- 172.21.15.62:0/2964426168 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0468062af0 con 0x7f04500a3cb0 2024-04-02T14:36:40.464 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 3 on host 'smithi062' 2024-04-02T14:36:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.464+0000 7f0470400700 1 -- 172.21.15.62:0/2964426168 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04500a3cb0 msgr2=0x7f04500a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:36:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.464+0000 7f0470400700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04500a3cb0 0x7f04500a6170 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f045c00b820 tx=0x7f045c00b8f0 comp rx=0 tx=0).stop 2024-04-02T14:36:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.464+0000 7f0470400700 1 -- 172.21.15.62:0/2964426168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04681038c0 msgr2=0x7f0468101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:36:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.464+0000 7f0470400700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04681038c0 0x7f0468101870 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f045400e9a0 tx=0x7f045400ecb0 comp rx=0 tx=0).stop 2024-04-02T14:36:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.465+0000 7f0470400700 1 -- 172.21.15.62:0/2964426168 shutdown_connections 2024-04-02T14:36:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.465+0000 7f0470400700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04500a3cb0 0x7f04500a6170 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.465+0000 7f0470400700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0468104260 0x7f0468101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.465+0000 7f0470400700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04681008e0 0x7f0468103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.465+0000 7f0470400700 1 --2- 172.21.15.62:0/2964426168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04681038c0 0x7f0468101870 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:40.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.465+0000 7f0470400700 1 -- 172.21.15.62:0/2964426168 >> 172.21.15.62:0/2964426168 conn(0x7f04680fc260 msgr2=0x7f0468106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:36:40.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.465+0000 7f0470400700 1 -- 172.21.15.62:0/2964426168 shutdown_connections 2024-04-02T14:36:40.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:36:40.465+0000 7f0470400700 1 -- 172.21.15.62:0/2964426168 wait complete. 2024-04-02T14:36:40.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:40.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:40.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:40 smithi204 ceph-mon[35948]: from='osd.3 [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T14:36:40.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:40 smithi204 ceph-mon[35948]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T14:36:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T14:36:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: from='osd.3 [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:36:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T14:36:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:36:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:41.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:41.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:41.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:41.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:41 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:41.263 DEBUG:teuthology.orchestra.run.smithi062:osd.3> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.3.service 2024-04-02T14:36:41.271 INFO:tasks.cephadm:Deploying osd.4 on smithi113 with /dev/vg_nvme/lv_3... 2024-04-02T14:36:41.271 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T14:36:41.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:41.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T14:36:41.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: from='osd.3 [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:36:41.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T14:36:41.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:41.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:36:41.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:41.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:41.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:41.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:41.374 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:41.374 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:41 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: from='osd.3 [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:41.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:41 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:36:41.529 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:36:42.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:42 smithi113 ceph-mon[36135]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:36:42.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:42 smithi113 ceph-mon[36135]: osdmap e25: 4 total, 3 up, 4 in 2024-04-02T14:36:42.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:42.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:42.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:42 smithi113 ceph-mon[36135]: osdmap e26: 4 total, 3 up, 4 in 2024-04-02T14:36:42.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:42 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:42.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:42 smithi062 ceph-mon[32116]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:36:42.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:42 smithi062 ceph-mon[32116]: osdmap e25: 4 total, 3 up, 4 in 2024-04-02T14:36:42.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:42.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:42.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:42 smithi062 ceph-mon[32116]: osdmap e26: 4 total, 3 up, 4 in 2024-04-02T14:36:42.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:42 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:42.345 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:36:42 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3[51591]: 2024-04-02T14:36:42.128+0000 7fc2aba00700 -1 osd.3 0 waiting for initial osdmap 2024-04-02T14:36:42.345 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:36:42 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3[51591]: 2024-04-02T14:36:42.155+0000 7fc2a1800700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:36:42.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:42 smithi204 ceph-mon[35948]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:36:42.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:42 smithi204 ceph-mon[35948]: osdmap e25: 4 total, 3 up, 4 in 2024-04-02T14:36:42.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:42.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:42.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:42 smithi204 ceph-mon[35948]: osdmap e26: 4 total, 3 up, 4 in 2024-04-02T14:36:42.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:42 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:43.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:43 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:36:43.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:43 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:36:43.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:43 smithi062 ceph-mon[32116]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:43.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:43 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:43.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:43 smithi062 ceph-mon[32116]: from='osd.3 ' entity='osd.3' 2024-04-02T14:36:43.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:43 smithi062 ceph-mon[32116]: osd.3 [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868] boot 2024-04-02T14:36:43.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:43 smithi062 ceph-mon[32116]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T14:36:43.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:43 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:43.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:43 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:36:43.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:43 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:36:43.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:43 smithi113 ceph-mon[36135]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:43.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:43 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:43.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:43 smithi113 ceph-mon[36135]: from='osd.3 ' entity='osd.3' 2024-04-02T14:36:43.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:43 smithi113 ceph-mon[36135]: osd.3 [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868] boot 2024-04-02T14:36:43.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:43 smithi113 ceph-mon[36135]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T14:36:43.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:43 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:43.483 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T14:36:43.483 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T14:36:43.483 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-04-02T14:36:43.483 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-04-02T14:36:43.483 INFO:teuthology.orchestra.run.smithi113.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00799152 s, 1.3 GB/s 2024-04-02T14:36:43.483 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-04-02T14:36:43.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:43 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:36:43.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:43 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:36:43.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:43 smithi204 ceph-mon[35948]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-04-02T14:36:43.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:43 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:43.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:43 smithi204 ceph-mon[35948]: from='osd.3 ' entity='osd.3' 2024-04-02T14:36:43.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:43 smithi204 ceph-mon[35948]: osd.3 [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868] boot 2024-04-02T14:36:43.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:43 smithi204 ceph-mon[35948]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T14:36:43.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:43 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:36:44.068 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi113:vg_nvme/lv_3 2024-04-02T14:36:44.306 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:36:44.742 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:44 smithi113 ceph-mon[36135]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:44.742 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:44 smithi113 ceph-mon[36135]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T14:36:44.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:44 smithi204 ceph-mon[35948]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:44.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:44 smithi204 ceph-mon[35948]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T14:36:44.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:44 smithi062 ceph-mon[32116]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:44.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:44 smithi062 ceph-mon[32116]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T14:36:45.610 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:45 smithi113 ceph-mon[36135]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T14:36:45.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:45 smithi204 ceph-mon[35948]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T14:36:45.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.814+0000 7ffb08fa5700 1 -- 172.21.15.113:0/1025227397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb04101520 msgr2=0x7ffb04101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:36:45.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.814+0000 7ffb08fa5700 1 --2- 172.21.15.113:0/1025227397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb04101520 0x7ffb04101940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40099c0 tx=0x7ffaf4009cd0 comp rx=0 tx=0).stop 2024-04-02T14:36:45.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.815+0000 7ffb08fa5700 1 -- 172.21.15.113:0/1025227397 shutdown_connections 2024-04-02T14:36:45.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.815+0000 7ffb08fa5700 1 --2- 172.21.15.113:0/1025227397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb04101520 0x7ffb04101940 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:45.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.815+0000 7ffb08fa5700 1 --2- 172.21.15.113:0/1025227397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb041030c0 0x7ffb04109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:45.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.815+0000 7ffb08fa5700 1 --2- 172.21.15.113:0/1025227397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04102720 0x7ffb04102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:45.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.815+0000 7ffb08fa5700 1 -- 172.21.15.113:0/1025227397 >> 172.21.15.113:0/1025227397 conn(0x7ffb040fcae0 msgr2=0x7ffb040fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:36:45.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.815+0000 7ffb08fa5700 1 -- 172.21.15.113:0/1025227397 shutdown_connections 2024-04-02T14:36:45.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.815+0000 7ffb08fa5700 1 -- 172.21.15.113:0/1025227397 wait complete. 2024-04-02T14:36:45.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.815+0000 7ffb08fa5700 1 Processor -- start 2024-04-02T14:36:45.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.815+0000 7ffb08fa5700 1 -- start start 2024-04-02T14:36:45.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb08fa5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04101520 0x7ffb041054b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:36:45.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb08fa5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb04102720 0x7ffb04103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:36:45.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb08fa5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb041030c0 0x7ffb04104040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:36:45.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb08fa5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0410d2a0 con 0x7ffb041030c0 2024-04-02T14:36:45.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb08fa5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0410d120 con 0x7ffb04102720 2024-04-02T14:36:45.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb08fa5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0410d420 con 0x7ffb04101520 2024-04-02T14:36:45.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb01d9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb04102720 0x7ffb04103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:36:45.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb01d9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb04102720 0x7ffb04103b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33296/0 (socket says 172.21.15.113:33296) 2024-04-02T14:36:45.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb01d9b700 1 -- 172.21.15.113:0/2313499220 learned_addr learned my addr 172.21.15.113:0/2313499220 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-02T14:36:45.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb0259c700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04101520 0x7ffb041054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:36:45.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb01d9b700 1 -- 172.21.15.113:0/2313499220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04101520 msgr2=0x7ffb041054b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:36:45.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb01d9b700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04101520 0x7ffb041054b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:45.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb01d9b700 1 -- 172.21.15.113:0/2313499220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb041030c0 msgr2=0x7ffb04104040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:36:45.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb01d9b700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb041030c0 0x7ffb04104040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:36:45.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb01d9b700 1 -- 172.21.15.113:0/2313499220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb041048f0 con 0x7ffb04102720 2024-04-02T14:36:45.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb01d9b700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb04102720 0x7ffb04103b00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800c920 tx=0x7ffaf800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:36:45.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffaf37fe700 1 -- 172.21.15.113:0/2313499220 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8007b00 con 0x7ffb04102720 2024-04-02T14:36:45.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb08fa5700 1 -- 172.21.15.113:0/2313499220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb041a2b00 con 0x7ffb04102720 2024-04-02T14:36:45.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffaf37fe700 1 -- 172.21.15.113:0/2313499220 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffaf8004d30 con 0x7ffb04102720 2024-04-02T14:36:45.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffaf37fe700 1 -- 172.21.15.113:0/2313499220 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8005680 con 0x7ffb04102720 2024-04-02T14:36:45.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.816+0000 7ffb08fa5700 1 -- 172.21.15.113:0/2313499220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb041a3010 con 0x7ffb04102720 2024-04-02T14:36:45.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.817+0000 7ffb08fa5700 1 -- 172.21.15.113:0/2313499220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb040692e0 con 0x7ffb04102720 2024-04-02T14:36:45.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.818+0000 7ffaf37fe700 1 -- 172.21.15.113:0/2313499220 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffaf8004750 con 0x7ffb04102720 2024-04-02T14:36:45.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.819+0000 7ffaf37fe700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ffaec0a3c40 0x7ffaec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:36:45.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.819+0000 7ffaf37fe700 1 -- 172.21.15.113:0/2313499220 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7ffaf80ced60 con 0x7ffb04102720 2024-04-02T14:36:45.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.820+0000 7ffb0259c700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ffaec0a3c40 0x7ffaec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:36:45.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.820+0000 7ffb0259c700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ffaec0a3c40 0x7ffaec0a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4003990 tx=0x7ffaf4005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:36:45.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:45.824+0000 7ffaf37fe700 1 -- 172.21.15.113:0/2313499220 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ffb040692e0 con 0x7ffb04102720 2024-04-02T14:36:45.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:45 smithi062 ceph-mon[32116]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T14:36:46.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:36:46.029+0000 7ffb08fa5700 1 -- 172.21.15.113:0/2313499220 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ffb040fec30 con 0x7ffaec0a3c40 2024-04-02T14:36:46.690 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:46 smithi113 ceph-mon[36135]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:46.690 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:46 smithi113 ceph-mon[36135]: osdmap e30: 4 total, 4 up, 4 in 2024-04-02T14:36:46.690 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:46 smithi113 ceph-mon[36135]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:36:46.690 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:36:46.691 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:36:46.691 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:46.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:46 smithi204 ceph-mon[35948]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:46.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:46 smithi204 ceph-mon[35948]: osdmap e30: 4 total, 4 up, 4 in 2024-04-02T14:36:46.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:46 smithi204 ceph-mon[35948]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:36:46.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:36:46.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:36:46.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:46.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:46 smithi062 ceph-mon[32116]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:46.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:46 smithi062 ceph-mon[32116]: osdmap e30: 4 total, 4 up, 4 in 2024-04-02T14:36:46.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:46 smithi062 ceph-mon[32116]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:36:46.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:36:46.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:36:46.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:47.654 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:47 smithi113 ceph-mon[36135]: osdmap e31: 4 total, 4 up, 4 in 2024-04-02T14:36:47.654 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:47 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi062 2024-04-02T14:36:47.654 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:47.654 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:47.654 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:36:47.654 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:36:47.655 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:47 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi062 to 8545M 2024-04-02T14:36:47.655 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:47.655 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:47.655 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:47.655 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:47.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:47 smithi204 ceph-mon[35948]: osdmap e31: 4 total, 4 up, 4 in 2024-04-02T14:36:47.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:47 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi062 2024-04-02T14:36:47.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:47.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:47.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:36:47.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:36:47.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:47 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi062 to 8545M 2024-04-02T14:36:47.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:47.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:47.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:47.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:47 smithi062 ceph-mon[32116]: osdmap e31: 4 total, 4 up, 4 in 2024-04-02T14:36:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:47 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi062 2024-04-02T14:36:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:36:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:36:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:47 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi062 to 8545M 2024-04-02T14:36:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:36:47.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:36:49.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:48 smithi204 ceph-mon[35948]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T14:36:49.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:48 smithi062 ceph-mon[32116]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T14:36:49.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:48 smithi113 ceph-mon[36135]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T14:36:50.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:49 smithi204 ceph-mon[35948]: from='client.? 172.21.15.113:0/3052654247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b899f2e-b8bc-4dce-991e-e222816cd0d6"}]: dispatch 2024-04-02T14:36:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:49 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b899f2e-b8bc-4dce-991e-e222816cd0d6"}]: dispatch 2024-04-02T14:36:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:49 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b899f2e-b8bc-4dce-991e-e222816cd0d6"}]': finished 2024-04-02T14:36:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:49 smithi204 ceph-mon[35948]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T14:36:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:49 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:36:50.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:49 smithi204 ceph-mon[35948]: from='client.? 172.21.15.113:0/4042432124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:36:50.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:49 smithi062 ceph-mon[32116]: from='client.? 172.21.15.113:0/3052654247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b899f2e-b8bc-4dce-991e-e222816cd0d6"}]: dispatch 2024-04-02T14:36:50.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:49 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b899f2e-b8bc-4dce-991e-e222816cd0d6"}]: dispatch 2024-04-02T14:36:50.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:49 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b899f2e-b8bc-4dce-991e-e222816cd0d6"}]': finished 2024-04-02T14:36:50.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:49 smithi062 ceph-mon[32116]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T14:36:50.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:49 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:36:50.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:49 smithi062 ceph-mon[32116]: from='client.? 172.21.15.113:0/4042432124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:36:50.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:49 smithi113 ceph-mon[36135]: from='client.? 172.21.15.113:0/3052654247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b899f2e-b8bc-4dce-991e-e222816cd0d6"}]: dispatch 2024-04-02T14:36:50.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:49 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b899f2e-b8bc-4dce-991e-e222816cd0d6"}]: dispatch 2024-04-02T14:36:50.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:49 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b899f2e-b8bc-4dce-991e-e222816cd0d6"}]': finished 2024-04-02T14:36:50.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:49 smithi113 ceph-mon[36135]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T14:36:50.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:49 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:36:50.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:49 smithi113 ceph-mon[36135]: from='client.? 172.21.15.113:0/4042432124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:36:51.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:50 smithi204 ceph-mon[35948]: pgmap v127: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-02T14:36:51.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:50 smithi062 ceph-mon[32116]: pgmap v127: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-02T14:36:51.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:50 smithi113 ceph-mon[36135]: pgmap v127: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-02T14:36:53.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:52 smithi204 ceph-mon[35948]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T14:36:53.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:52 smithi062 ceph-mon[32116]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T14:36:53.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:52 smithi113 ceph-mon[36135]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T14:36:54.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:54 smithi062 ceph-mon[32116]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:55.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:54 smithi204 ceph-mon[35948]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:55.051 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:54 smithi113 ceph-mon[36135]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:57.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:56 smithi204 ceph-mon[35948]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:57.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:56 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T14:36:57.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:56 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:57.042 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:56 smithi113 ceph-mon[36135]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:57.043 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:56 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T14:36:57.043 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:56 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:57.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:56 smithi062 ceph-mon[32116]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:57.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:56 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T14:36:57.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:56 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:36:57.931 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:57 smithi113 ceph-mon[36135]: Deploying daemon osd.4 on smithi113 2024-04-02T14:36:58.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:57 smithi062 ceph-mon[32116]: Deploying daemon osd.4 on smithi113 2024-04-02T14:36:58.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:57 smithi204 ceph-mon[35948]: Deploying daemon osd.4 on smithi113 2024-04-02T14:36:59.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:36:58 smithi062 ceph-mon[32116]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:59.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:36:58 smithi113 ceph-mon[36135]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:36:59.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:36:58 smithi204 ceph-mon[35948]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:00.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:00 smithi113 ceph-mon[36135]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:01.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:00 smithi062 ceph-mon[32116]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:01.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:00 smithi204 ceph-mon[35948]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:03.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:02 smithi204 ceph-mon[35948]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:03.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:03.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:03.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:03.062 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:02 smithi113 ceph-mon[36135]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:03.063 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:03.063 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:03.063 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:03.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:02 smithi062 ceph-mon[32116]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:03.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:03.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:03.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:04.940 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:04 smithi113 ceph-mon[36135]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:05.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:04 smithi204 ceph-mon[35948]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:05.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.019+0000 7ffaf37fe700 1 -- 172.21.15.113:0/2313499220 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffb040fec30 con 0x7ffaec0a3c40 2024-04-02T14:37:05.020 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 4 on host 'smithi113' 2024-04-02T14:37:05.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 -- 172.21.15.113:0/2313499220 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ffaec0a3c40 msgr2=0x7ffaec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:37:05.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ffaec0a3c40 0x7ffaec0a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4003990 tx=0x7ffaf4005ec0 comp rx=0 tx=0).stop 2024-04-02T14:37:05.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 -- 172.21.15.113:0/2313499220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb04102720 msgr2=0x7ffb04103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:37:05.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb04102720 0x7ffb04103b00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800c920 tx=0x7ffaf800cc30 comp rx=0 tx=0).stop 2024-04-02T14:37:05.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 -- 172.21.15.113:0/2313499220 shutdown_connections 2024-04-02T14:37:05.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ffaec0a3c40 0x7ffaec0a6100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:05.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb041030c0 0x7ffb04104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:05.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04101520 0x7ffb041054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:05.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 --2- 172.21.15.113:0/2313499220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb04102720 0x7ffb04103b00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:05.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 -- 172.21.15.113:0/2313499220 >> 172.21.15.113:0/2313499220 conn(0x7ffb040fcae0 msgr2=0x7ffb040fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:37:05.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 -- 172.21.15.113:0/2313499220 shutdown_connections 2024-04-02T14:37:05.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:37:05.021+0000 7ffb08fa5700 1 -- 172.21.15.113:0/2313499220 wait complete. 2024-04-02T14:37:05.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:04 smithi062 ceph-mon[32116]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:05.568 DEBUG:teuthology.orchestra.run.smithi113:osd.4> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.4.service 2024-04-02T14:37:05.571 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-04-02T14:37:05.571 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T14:37:05.820 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:37:06.239 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:05 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.240 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:05 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.240 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:05 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.240 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:05 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.240 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:05 smithi204 ceph-mon[35948]: from='osd.4 [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T14:37:06.240 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:05 smithi204 ceph-mon[35948]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T14:37:06.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:05 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:05 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:05 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:05 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:05 smithi062 ceph-mon[32116]: from='osd.4 [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T14:37:06.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:05 smithi062 ceph-mon[32116]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T14:37:06.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:05 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:05 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:05 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:05 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:06.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:05 smithi113 ceph-mon[36135]: from='osd.4 [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T14:37:06.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:05 smithi113 ceph-mon[36135]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T14:37:07.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:07 smithi204 ceph-mon[35948]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:07.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:07 smithi204 ceph-mon[35948]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T14:37:07.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:07 smithi204 ceph-mon[35948]: from='osd.4 [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:37:07.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:07 smithi204 ceph-mon[35948]: osdmap e33: 5 total, 4 up, 5 in 2024-04-02T14:37:07.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:07 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:07.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:07 smithi204 ceph-mon[35948]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:37:07.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:07 smithi062 ceph-mon[32116]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:07.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:07 smithi062 ceph-mon[32116]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T14:37:07.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:07 smithi062 ceph-mon[32116]: from='osd.4 [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:37:07.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:07 smithi062 ceph-mon[32116]: osdmap e33: 5 total, 4 up, 5 in 2024-04-02T14:37:07.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:07 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:07.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:07 smithi062 ceph-mon[32116]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:37:07.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:07 smithi113 ceph-mon[36135]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:07.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:07 smithi113 ceph-mon[36135]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T14:37:07.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:07 smithi113 ceph-mon[36135]: from='osd.4 [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:37:07.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:07 smithi113 ceph-mon[36135]: osdmap e33: 5 total, 4 up, 5 in 2024-04-02T14:37:07.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:07 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:07.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:07 smithi113 ceph-mon[36135]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:37:07.819 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T14:37:07.819 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T14:37:07.819 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-04-02T14:37:07.819 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-04-02T14:37:07.819 INFO:teuthology.orchestra.run.smithi204.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112452 s, 932 MB/s 2024-04-02T14:37:07.819 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-04-02T14:37:08.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:08 smithi113 ceph-mon[36135]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:37:08.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:08 smithi113 ceph-mon[36135]: osdmap e34: 5 total, 4 up, 5 in 2024-04-02T14:37:08.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:08 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:08.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:08 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:08.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:08 smithi113 ceph-mon[36135]: from='osd.4 ' entity='osd.4' 2024-04-02T14:37:08.123 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:37:07 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4[47700]: 2024-04-02T14:37:07.836+0000 7f72c7a5a700 -1 osd.4 0 waiting for initial osdmap 2024-04-02T14:37:08.123 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:37:07 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4[47700]: 2024-04-02T14:37:07.864+0000 7f72bf845700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:37:08.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:08 smithi062 ceph-mon[32116]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:37:08.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:08 smithi062 ceph-mon[32116]: osdmap e34: 5 total, 4 up, 5 in 2024-04-02T14:37:08.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:08 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:08.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:08 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:08.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:08 smithi062 ceph-mon[32116]: from='osd.4 ' entity='osd.4' 2024-04-02T14:37:08.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:08 smithi204 ceph-mon[35948]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:37:08.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:08 smithi204 ceph-mon[35948]: osdmap e34: 5 total, 4 up, 5 in 2024-04-02T14:37:08.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:08 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:08.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:08 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:08.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:08 smithi204 ceph-mon[35948]: from='osd.4 ' entity='osd.4' 2024-04-02T14:37:08.477 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-04-02T14:37:08.713 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:37:09.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:37:09.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:37:09.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:09.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:09.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:09.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:09.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:09.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:09.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:09.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:09.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:09.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: osd.4 [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927] boot 2024-04-02T14:37:09.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: osdmap e35: 5 total, 5 up, 5 in 2024-04-02T14:37:09.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:09.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:37:09.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:37:09.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:09.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:09.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: osd.4 [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927] boot 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: osdmap e35: 5 total, 5 up, 5 in 2024-04-02T14:37:09.373 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:37:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:37:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-04-02T14:37:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: osd.4 [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927] boot 2024-04-02T14:37:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: osdmap e35: 5 total, 5 up, 5 in 2024-04-02T14:37:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:37:10.269 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.263+0000 7f5944e8d700 1 -- 172.21.15.204:0/2950240395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5940101530 msgr2=0x7f5940101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:37:10.269 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.263+0000 7f5944e8d700 1 --2- 172.21.15.204:0/2950240395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5940101530 0x7f5940101950 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5930009990 tx=0x7f5930009ca0 comp rx=0 tx=0).stop 2024-04-02T14:37:10.270 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.263+0000 7f5944e8d700 1 -- 172.21.15.204:0/2950240395 shutdown_connections 2024-04-02T14:37:10.270 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.263+0000 7f5944e8d700 1 --2- 172.21.15.204:0/2950240395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59401030d0 0x7f5940109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:10.270 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.263+0000 7f5944e8d700 1 --2- 172.21.15.204:0/2950240395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5940101530 0x7f5940101950 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:10.270 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.263+0000 7f5944e8d700 1 --2- 172.21.15.204:0/2950240395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5940102730 0x7f5940102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:10.270 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.263+0000 7f5944e8d700 1 -- 172.21.15.204:0/2950240395 >> 172.21.15.204:0/2950240395 conn(0x7f59400fcaf0 msgr2=0x7f59400fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:37:10.270 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.263+0000 7f5944e8d700 1 -- 172.21.15.204:0/2950240395 shutdown_connections 2024-04-02T14:37:10.271 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.263+0000 7f5944e8d700 1 -- 172.21.15.204:0/2950240395 wait complete. 2024-04-02T14:37:10.271 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f5944e8d700 1 Processor -- start 2024-04-02T14:37:10.271 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f5944e8d700 1 -- start start 2024-04-02T14:37:10.271 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f5944e8d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5940101530 0x7f59401989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:37:10.271 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f5944e8d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5940102730 0x7f5940198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:37:10.271 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f5944e8d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59401030d0 0x7f594019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:37:10.272 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f5944e8d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f594010d2d0 con 0x7f59401030d0 2024-04-02T14:37:10.272 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f5944e8d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f594010d150 con 0x7f5940102730 2024-04-02T14:37:10.272 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f5944e8d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f594010d450 con 0x7f5940101530 2024-04-02T14:37:10.272 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f593e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5940101530 0x7f59401989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:37:10.272 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f593e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5940101530 0x7f59401989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47204/0 (socket says 172.21.15.204:47204) 2024-04-02T14:37:10.272 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f593e59c700 1 -- 172.21.15.204:0/2516710214 learned_addr learned my addr 172.21.15.204:0/2516710214 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T14:37:10.272 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f5937fff700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5940102730 0x7f5940198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:37:10.273 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f593e59c700 1 -- 172.21.15.204:0/2516710214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5940102730 msgr2=0x7f5940198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:37:10.273 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f593e59c700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5940102730 0x7f5940198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:10.273 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f593e59c700 1 -- 172.21.15.204:0/2516710214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59401030d0 msgr2=0x7f594019ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:37:10.273 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f593e59c700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59401030d0 0x7f594019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:10.273 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.264+0000 7f593e59c700 1 -- 172.21.15.204:0/2516710214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59401a0600 con 0x7f5940101530 2024-04-02T14:37:10.273 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.265+0000 7f593e59c700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5940101530 0x7f59401989a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5930003ad0 tx=0x7f593000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:37:10.274 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.265+0000 7f59377fe700 1 -- 172.21.15.204:0/2516710214 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930003b70 con 0x7f5940101530 2024-04-02T14:37:10.274 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.265+0000 7f59377fe700 1 -- 172.21.15.204:0/2516710214 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f59300055d0 con 0x7f5940101530 2024-04-02T14:37:10.274 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.265+0000 7f5944e8d700 1 -- 172.21.15.204:0/2516710214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59401a0890 con 0x7f5940101530 2024-04-02T14:37:10.274 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.265+0000 7f59377fe700 1 -- 172.21.15.204:0/2516710214 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930017650 con 0x7f5940101530 2024-04-02T14:37:10.274 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.265+0000 7f5944e8d700 1 -- 172.21.15.204:0/2516710214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59401a0ce0 con 0x7f5940101530 2024-04-02T14:37:10.275 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.266+0000 7f5944e8d700 1 -- 172.21.15.204:0/2516710214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59400692b0 con 0x7f5940101530 2024-04-02T14:37:10.275 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.267+0000 7f59377fe700 1 -- 172.21.15.204:0/2516710214 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f59300050d0 con 0x7f5940101530 2024-04-02T14:37:10.275 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.267+0000 7f59377fe700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f592c0a3c40 0x7f592c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:37:10.275 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.267+0000 7f59377fe700 1 -- 172.21.15.204:0/2516710214 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4101+0+0 (secure 0 0 0) 0x7f59300d04a0 con 0x7f5940101530 2024-04-02T14:37:10.275 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.268+0000 7f5937fff700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f592c0a3c40 0x7f592c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:37:10.276 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.268+0000 7f5937fff700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f592c0a3c40 0x7f592c0a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5928009a10 tx=0x7f5928009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:37:10.276 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.272+0000 7f59377fe700 1 -- 172.21.15.204:0/2516710214 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5930013070 con 0x7f5940101530 2024-04-02T14:37:10.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:10 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi113 2024-04-02T14:37:10.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:10 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi113 to 8540M 2024-04-02T14:37:10.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:10.481+0000 7f5944e8d700 1 -- 172.21.15.204:0/2516710214 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f59400fec40 con 0x7f592c0a3c40 2024-04-02T14:37:10.483 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:10 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi113 2024-04-02T14:37:10.483 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:10 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi113 to 8540M 2024-04-02T14:37:10.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:10 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi113 2024-04-02T14:37:10.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:10 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi113 to 8540M 2024-04-02T14:37:11.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:11 smithi113 ceph-mon[36135]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:11.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:11 smithi113 ceph-mon[36135]: osdmap e36: 5 total, 5 up, 5 in 2024-04-02T14:37:11.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:11 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:37:11.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:11 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:37:11.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:11 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:11.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:11 smithi204 ceph-mon[35948]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:11.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:11 smithi204 ceph-mon[35948]: osdmap e36: 5 total, 5 up, 5 in 2024-04-02T14:37:11.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:11 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:37:11.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:11 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:37:11.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:11 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:11.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:11 smithi062 ceph-mon[32116]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:11.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:11 smithi062 ceph-mon[32116]: osdmap e36: 5 total, 5 up, 5 in 2024-04-02T14:37:11.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:11 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:37:11.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:11 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:37:11.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:11 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:12.376 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:12 smithi204 ceph-mon[35948]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:37:12.461 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:12 smithi113 ceph-mon[36135]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:37:12.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:12 smithi062 ceph-mon[32116]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:37:13.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:13 smithi204 ceph-mon[35948]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:13.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:13 smithi062 ceph-mon[32116]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:13.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:13 smithi113 ceph-mon[36135]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:14.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:14 smithi204 ceph-mon[35948]: from='client.? 172.21.15.204:0/434679126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e6ae99-58a3-4ca6-b090-62db34e993b6"}]: dispatch 2024-04-02T14:37:14.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:14 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e6ae99-58a3-4ca6-b090-62db34e993b6"}]: dispatch 2024-04-02T14:37:14.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:14 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65e6ae99-58a3-4ca6-b090-62db34e993b6"}]': finished 2024-04-02T14:37:14.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:14 smithi204 ceph-mon[35948]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T14:37:14.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:14 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:14.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:14 smithi204 ceph-mon[35948]: from='client.? 172.21.15.204:0/1332809496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:37:14.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:14 smithi062 ceph-mon[32116]: from='client.? 172.21.15.204:0/434679126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e6ae99-58a3-4ca6-b090-62db34e993b6"}]: dispatch 2024-04-02T14:37:14.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:14 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e6ae99-58a3-4ca6-b090-62db34e993b6"}]: dispatch 2024-04-02T14:37:14.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:14 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65e6ae99-58a3-4ca6-b090-62db34e993b6"}]': finished 2024-04-02T14:37:14.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:14 smithi062 ceph-mon[32116]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T14:37:14.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:14 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:14.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:14 smithi062 ceph-mon[32116]: from='client.? 172.21.15.204:0/1332809496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:37:14.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:14 smithi113 ceph-mon[36135]: from='client.? 172.21.15.204:0/434679126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e6ae99-58a3-4ca6-b090-62db34e993b6"}]: dispatch 2024-04-02T14:37:14.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:14 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e6ae99-58a3-4ca6-b090-62db34e993b6"}]: dispatch 2024-04-02T14:37:14.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:14 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65e6ae99-58a3-4ca6-b090-62db34e993b6"}]': finished 2024-04-02T14:37:14.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:14 smithi113 ceph-mon[36135]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T14:37:14.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:14 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:14.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:14 smithi113 ceph-mon[36135]: from='client.? 172.21.15.204:0/1332809496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:37:15.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:15 smithi204 ceph-mon[35948]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:15.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:15 smithi062 ceph-mon[32116]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:15.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:15 smithi113 ceph-mon[36135]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:17.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:17 smithi204 ceph-mon[35948]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:17.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:17 smithi062 ceph-mon[32116]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:17.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:17 smithi113 ceph-mon[36135]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:19.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:19 smithi204 ceph-mon[35948]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:19.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:19 smithi062 ceph-mon[32116]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:19.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:19 smithi113 ceph-mon[36135]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:21.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:21 smithi204 ceph-mon[35948]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:21.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T14:37:21.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:21.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:21 smithi062 ceph-mon[32116]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:21.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T14:37:21.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:21.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:21 smithi113 ceph-mon[36135]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:21.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T14:37:21.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:22.294 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:22 smithi204 ceph-mon[35948]: Deploying daemon osd.5 on smithi204 2024-04-02T14:37:22.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:22 smithi113 ceph-mon[36135]: Deploying daemon osd.5 on smithi204 2024-04-02T14:37:22.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:22 smithi062 ceph-mon[32116]: Deploying daemon osd.5 on smithi204 2024-04-02T14:37:23.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:23 smithi204 ceph-mon[35948]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:23.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:23 smithi062 ceph-mon[32116]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:23.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:23 smithi113 ceph-mon[36135]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:24.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:24 smithi204 ceph-mon[35948]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:24.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:24 smithi062 ceph-mon[32116]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:24.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:24 smithi113 ceph-mon[36135]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:27.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:27 smithi062 ceph-mon[32116]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:27.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:27 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:27.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:27 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:27.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:27 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:27.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:27 smithi113 ceph-mon[36135]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:27.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:27 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:27.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:27 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:27.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:27 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:27.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:27 smithi204 ceph-mon[35948]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:27.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:27 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:27.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:27 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:27.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:27 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:29.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:29 smithi204 ceph-mon[35948]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:29.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:29 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:29.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:29 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:29.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:29 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:29.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:29 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:29.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:29 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:29.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:29 smithi113 ceph-mon[36135]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:29.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:29 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:29.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:29 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:29.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:29 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:29.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:29 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:29.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:29 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:29.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:29 smithi062 ceph-mon[32116]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:29.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:29 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:29.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:29 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:29.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:29 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:29.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:29 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:29.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:29 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:29.717 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.715+0000 7f59377fe700 1 -- 172.21.15.204:0/2516710214 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f59400fec40 con 0x7f592c0a3c40 2024-04-02T14:37:29.717 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 -- 172.21.15.204:0/2516710214 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f592c0a3c40 msgr2=0x7f592c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f592c0a3c40 0x7f592c0a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5928009a10 tx=0x7f5928009190 comp rx=0 tx=0).stop 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 -- 172.21.15.204:0/2516710214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5940101530 msgr2=0x7f59401989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5940101530 0x7f59401989a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5930003ad0 tx=0x7f593000f690 comp rx=0 tx=0).stop 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 -- 172.21.15.204:0/2516710214 shutdown_connections 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f592c0a3c40 0x7f592c0a6100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59401030d0 0x7f594019ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5940101530 0x7f59401989a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 --2- 172.21.15.204:0/2516710214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5940102730 0x7f5940198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 -- 172.21.15.204:0/2516710214 >> 172.21.15.204:0/2516710214 conn(0x7f59400fcaf0 msgr2=0x7f59400fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 -- 172.21.15.204:0/2516710214 shutdown_connections 2024-04-02T14:37:29.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:37:29.717+0000 7f5944e8d700 1 -- 172.21.15.204:0/2516710214 wait complete. 2024-04-02T14:37:30.329 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.5.service 2024-04-02T14:37:30.332 INFO:tasks.cephadm:Deploying osd.6 on smithi062 with /dev/vg_nvme/lv_2... 2024-04-02T14:37:30.332 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T14:37:30.798 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:37:30.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:30 smithi062 ceph-mon[32116]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:30.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:30.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:30.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:30.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:30 smithi062 ceph-mon[32116]: from='osd.5 [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T14:37:30.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:30 smithi062 ceph-mon[32116]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T14:37:31.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:30 smithi204 ceph-mon[35948]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:30 smithi204 ceph-mon[35948]: from='osd.5 [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T14:37:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:30 smithi204 ceph-mon[35948]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T14:37:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:30 smithi113 ceph-mon[36135]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:37:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:30 smithi113 ceph-mon[36135]: from='osd.5 [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T14:37:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:30 smithi113 ceph-mon[36135]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T14:37:31.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:31 smithi062 ceph-mon[32116]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T14:37:31.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:31 smithi062 ceph-mon[32116]: from='osd.5 [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:37:31.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:31 smithi062 ceph-mon[32116]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T14:37:31.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:31 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:31.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:31 smithi062 ceph-mon[32116]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:37:32.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:31 smithi204 ceph-mon[35948]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T14:37:32.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:31 smithi204 ceph-mon[35948]: from='osd.5 [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:37:32.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:31 smithi204 ceph-mon[35948]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T14:37:32.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:31 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:32.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:31 smithi204 ceph-mon[35948]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:37:32.011 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:37:31 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:37:31.680+0000 7fde378f2700 -1 osd.5 0 waiting for initial osdmap 2024-04-02T14:37:32.012 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:37:31 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:37:31.688+0000 7fde336e5700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:37:32.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:31 smithi113 ceph-mon[36135]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T14:37:32.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:31 smithi113 ceph-mon[36135]: from='osd.5 [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:37:32.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:31 smithi113 ceph-mon[36135]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T14:37:32.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:31 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:32.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:31 smithi113 ceph-mon[36135]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:37:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:32 smithi062 ceph-mon[32116]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:32 smithi062 ceph-mon[32116]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:37:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:32 smithi062 ceph-mon[32116]: osdmap e39: 6 total, 5 up, 6 in 2024-04-02T14:37:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:32 smithi062 ceph-mon[32116]: osd.5 [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574] boot 2024-04-02T14:37:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:32 smithi062 ceph-mon[32116]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T14:37:32.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:32.971 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:32 smithi204 ceph-mon[35948]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:32.971 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:32 smithi204 ceph-mon[35948]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:37:32.971 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:32 smithi204 ceph-mon[35948]: osdmap e39: 6 total, 5 up, 6 in 2024-04-02T14:37:32.972 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:32.972 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:32 smithi204 ceph-mon[35948]: osd.5 [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574] boot 2024-04-02T14:37:32.972 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:32 smithi204 ceph-mon[35948]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T14:37:32.972 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:32 smithi113 ceph-mon[36135]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:32 smithi113 ceph-mon[36135]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:37:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:32 smithi113 ceph-mon[36135]: osdmap e39: 6 total, 5 up, 6 in 2024-04-02T14:37:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:32 smithi113 ceph-mon[36135]: osd.5 [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574] boot 2024-04-02T14:37:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:32 smithi113 ceph-mon[36135]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T14:37:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:37:33.794 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi204 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi204 to 8540M 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:33.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:33 smithi062 ceph-mon[32116]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T14:37:33.826 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T14:37:33.827 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T14:37:33.827 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-02T14:37:33.827 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-02T14:37:33.827 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0350305 s, 299 MB/s 2024-04-02T14:37:33.827 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-02T14:37:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:37:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:37:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi204 2024-04-02T14:37:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi204 to 8540M 2024-04-02T14:37:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:33 smithi204 ceph-mon[35948]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T14:37:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:37:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:37:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi204 2024-04-02T14:37:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:37:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi204 to 8540M 2024-04-02T14:37:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:34.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:37:34.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:37:34.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:33 smithi113 ceph-mon[36135]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T14:37:34.759 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2024-04-02T14:37:35.188 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:37:35.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:35 smithi062 ceph-mon[32116]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:35.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:35 smithi062 ceph-mon[32116]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T14:37:35.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:35 smithi113 ceph-mon[36135]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:35.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:35 smithi113 ceph-mon[36135]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T14:37:35.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:35 smithi204 ceph-mon[35948]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 447 GiB / 447 GiB avail 2024-04-02T14:37:35.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:35 smithi204 ceph-mon[35948]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T14:37:36.948 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:36 smithi062 ceph-mon[32116]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:36.948 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:36 smithi062 ceph-mon[32116]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T14:37:37.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:36 smithi204 ceph-mon[35948]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:37.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:36 smithi204 ceph-mon[35948]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T14:37:37.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:36 smithi113 ceph-mon[36135]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:37.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:36 smithi113 ceph-mon[36135]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T14:37:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.410+0000 7f6931e00700 1 -- 172.21.15.62:0/2474852581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f692c1041d0 msgr2=0x7f692c10aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:37:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.410+0000 7f6931e00700 1 --2- 172.21.15.62:0/2474852581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f692c1041d0 0x7f692c10aa80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f692400b0d0 tx=0x7f692400b3e0 comp rx=0 tx=0).stop 2024-04-02T14:37:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.410+0000 7f6931e00700 1 -- 172.21.15.62:0/2474852581 shutdown_connections 2024-04-02T14:37:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.410+0000 7f6931e00700 1 --2- 172.21.15.62:0/2474852581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f692c102630 0x7f692c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.410+0000 7f6931e00700 1 --2- 172.21.15.62:0/2474852581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f692c1041d0 0x7f692c10aa80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.410+0000 7f6931e00700 1 --2- 172.21.15.62:0/2474852581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f692c103830 0x7f692c103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.410+0000 7f6931e00700 1 -- 172.21.15.62:0/2474852581 >> 172.21.15.62:0/2474852581 conn(0x7f692c0fdb90 msgr2=0x7f692c100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:37:37.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.410+0000 7f6931e00700 1 -- 172.21.15.62:0/2474852581 shutdown_connections 2024-04-02T14:37:37.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.410+0000 7f6931e00700 1 -- 172.21.15.62:0/2474852581 wait complete. 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.411+0000 7f6931e00700 1 Processor -- start 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.412+0000 7f6931e00700 1 -- start start 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.412+0000 7f6931e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f692c102630 0x7f692c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.412+0000 7f6931e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f692c103830 0x7f692c19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.412+0000 7f6931e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f692c1041d0 0x7f692c1a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.412+0000 7f6931e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f692c075cd0 con 0x7f692c1041d0 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.412+0000 7f6931e00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f692c075b50 con 0x7f692c103830 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.412+0000 7f6931e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f692c075e50 con 0x7f692c102630 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f692c1041d0 0x7f692c1a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692a000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f692c103830 0x7f692c19afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692a000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f692c103830 0x7f692c19afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:58848/0 (socket says 172.21.15.62:58848) 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692a000700 1 -- 172.21.15.62:0/3415010531 learned_addr learned my addr 172.21.15.62:0/3415010531 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:37:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692aa00700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f692c102630 0x7f692c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692a000700 1 -- 172.21.15.62:0/3415010531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f692c102630 msgr2=0x7f692c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692a000700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f692c102630 0x7f692c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692a000700 1 -- 172.21.15.62:0/3415010531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f692c1041d0 msgr2=0x7f692c1a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692a000700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f692c1041d0 0x7f692c1a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692a000700 1 -- 172.21.15.62:0/3415010531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c1a2700 con 0x7f692c103830 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692aa00700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f692c102630 0x7f692c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692b400700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f692c1041d0 0x7f692c1a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.413+0000 7f692a000700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f692c103830 0x7f692c19afe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f691800e9c0 tx=0x7f691800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.414+0000 7f6930a00700 1 -- 172.21.15.62:0/3415010531 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691800cbd0 con 0x7f692c103830 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.414+0000 7f6930a00700 1 -- 172.21.15.62:0/3415010531 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f691800ee60 con 0x7f692c103830 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.414+0000 7f6931e00700 1 -- 172.21.15.62:0/3415010531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692c1a29f0 con 0x7f692c103830 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.414+0000 7f6930a00700 1 -- 172.21.15.62:0/3415010531 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918010430 con 0x7f692c103830 2024-04-02T14:37:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.414+0000 7f6931e00700 1 -- 172.21.15.62:0/3415010531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692c1a2ee0 con 0x7f692c103830 2024-04-02T14:37:37.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.415+0000 7f6931e00700 1 -- 172.21.15.62:0/3415010531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692c0692b0 con 0x7f692c103830 2024-04-02T14:37:37.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.417+0000 7f6930a00700 1 -- 172.21.15.62:0/3415010531 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6918004750 con 0x7f692c103830 2024-04-02T14:37:37.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.418+0000 7f6930a00700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f69100a3c40 0x7f69100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:37:37.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.419+0000 7f6930a00700 1 -- 172.21.15.62:0/3415010531 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4533+0+0 (secure 0 0 0) 0x7f69180cf810 con 0x7f692c103830 2024-04-02T14:37:37.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.419+0000 7f692aa00700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f69100a3c40 0x7f69100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:37:37.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.419+0000 7f692aa00700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f69100a3c40 0x7f69100a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f692c103690 tx=0x7f691c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:37:37.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.423+0000 7f6930a00700 1 -- 172.21.15.62:0/3415010531 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f691809d010 con 0x7f692c103830 2024-04-02T14:37:37.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:37:37.668+0000 7f6931e00700 1 -- 172.21.15.62:0/3415010531 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f692c062a30 con 0x7f69100a3c40 2024-04-02T14:37:38.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:37 smithi204 ceph-mon[35948]: osdmap e44: 6 total, 6 up, 6 in 2024-04-02T14:37:38.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:37:38.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:37:38.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:38.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:37 smithi062 ceph-mon[32116]: osdmap e44: 6 total, 6 up, 6 in 2024-04-02T14:37:38.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:37:38.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:37:38.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:38.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:37 smithi113 ceph-mon[36135]: osdmap e44: 6 total, 6 up, 6 in 2024-04-02T14:37:38.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:37:38.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:37:38.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:38.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:38 smithi062 ceph-mon[32116]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T14:37:38.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:38 smithi062 ceph-mon[32116]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:37:39.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:38 smithi204 ceph-mon[35948]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T14:37:39.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:38 smithi204 ceph-mon[35948]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:37:39.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:38 smithi113 ceph-mon[36135]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T14:37:39.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:38 smithi113 ceph-mon[36135]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:37:41.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:40 smithi204 ceph-mon[35948]: pgmap v164: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-02T14:37:41.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:40 smithi062 ceph-mon[32116]: pgmap v164: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-02T14:37:41.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:40 smithi113 ceph-mon[36135]: pgmap v164: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-02T14:37:42.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:41 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2589692213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dfddadf-9e19-4690-aa90-b9bf9044e6c9"}]: dispatch 2024-04-02T14:37:42.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:41 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dfddadf-9e19-4690-aa90-b9bf9044e6c9"}]: dispatch 2024-04-02T14:37:42.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:41 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6dfddadf-9e19-4690-aa90-b9bf9044e6c9"}]': finished 2024-04-02T14:37:42.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:41 smithi204 ceph-mon[35948]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T14:37:42.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:41 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:37:42.023 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:41 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2589692213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dfddadf-9e19-4690-aa90-b9bf9044e6c9"}]: dispatch 2024-04-02T14:37:42.023 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:41 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dfddadf-9e19-4690-aa90-b9bf9044e6c9"}]: dispatch 2024-04-02T14:37:42.023 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:41 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6dfddadf-9e19-4690-aa90-b9bf9044e6c9"}]': finished 2024-04-02T14:37:42.023 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:41 smithi113 ceph-mon[36135]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T14:37:42.023 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:41 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:37:42.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:41 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2589692213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dfddadf-9e19-4690-aa90-b9bf9044e6c9"}]: dispatch 2024-04-02T14:37:42.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:41 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dfddadf-9e19-4690-aa90-b9bf9044e6c9"}]: dispatch 2024-04-02T14:37:42.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:41 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6dfddadf-9e19-4690-aa90-b9bf9044e6c9"}]': finished 2024-04-02T14:37:42.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:41 smithi062 ceph-mon[32116]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T14:37:42.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:41 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:37:43.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:42 smithi204 ceph-mon[35948]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-02T14:37:43.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:42 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/648385142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:37:43.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:42 smithi062 ceph-mon[32116]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-02T14:37:43.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:42 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/648385142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:37:43.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:42 smithi113 ceph-mon[36135]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-02T14:37:43.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:42 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/648385142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:37:45.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:44 smithi204 ceph-mon[35948]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:45.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:44 smithi062 ceph-mon[32116]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:45.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:44 smithi113 ceph-mon[36135]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:47.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:46 smithi062 ceph-mon[32116]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:47.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:46 smithi113 ceph-mon[36135]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:47.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:46 smithi204 ceph-mon[35948]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:49.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:48 smithi062 ceph-mon[32116]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:49.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:48 smithi113 ceph-mon[36135]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:49.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:48 smithi204 ceph-mon[35948]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:51.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:50 smithi062 ceph-mon[32116]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:51.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:50 smithi113 ceph-mon[36135]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:51.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:50 smithi204 ceph-mon[35948]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:53.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:52 smithi062 ceph-mon[32116]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:53.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T14:37:53.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:53.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:52 smithi062 ceph-mon[32116]: Deploying daemon osd.6 on smithi062 2024-04-02T14:37:53.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:52 smithi113 ceph-mon[36135]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:53.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:52 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T14:37:53.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:52 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:53.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:52 smithi113 ceph-mon[36135]: Deploying daemon osd.6 on smithi062 2024-04-02T14:37:53.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:52 smithi204 ceph-mon[35948]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:53.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:52 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T14:37:53.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:52 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:37:53.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:52 smithi204 ceph-mon[35948]: Deploying daemon osd.6 on smithi062 2024-04-02T14:37:55.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:54 smithi062 ceph-mon[32116]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:55.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:54 smithi113 ceph-mon[36135]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:55.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:54 smithi204 ceph-mon[35948]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:56 smithi062 ceph-mon[32116]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:57.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:56 smithi113 ceph-mon[36135]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:57.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:56 smithi204 ceph-mon[35948]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:59.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:37:58 smithi062 ceph-mon[32116]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:59.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:37:58 smithi113 ceph-mon[36135]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:37:59.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:37:58 smithi204 ceph-mon[35948]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:01.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:00 smithi062 ceph-mon[32116]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:01.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:01.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:01.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:01.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:00 smithi113 ceph-mon[36135]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:01.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:01.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:01.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:01.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:00 smithi204 ceph-mon[35948]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:01.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:01.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:01.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:03.063 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:02 smithi204 ceph-mon[35948]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:03.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:02 smithi062 ceph-mon[32116]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:03.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:02 smithi113 ceph-mon[36135]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:04.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:04 smithi062 ceph-mon[32116]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:04.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:04.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:04.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:04 smithi062 ceph-mon[32116]: from='osd.6 [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T14:38:04.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:04 smithi062 ceph-mon[32116]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T14:38:05.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:04 smithi113 ceph-mon[36135]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:05.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:05.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:05.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:04 smithi113 ceph-mon[36135]: from='osd.6 [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T14:38:05.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:04 smithi113 ceph-mon[36135]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T14:38:05.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:04 smithi204 ceph-mon[35948]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:05.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:05.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:05.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:04 smithi204 ceph-mon[35948]: from='osd.6 [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T14:38:05.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:04 smithi204 ceph-mon[35948]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T14:38:05.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.626+0000 7f6930a00700 1 -- 172.21.15.62:0/3415010531 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f692c062a30 con 0x7f69100a3c40 2024-04-02T14:38:05.628 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 6 on host 'smithi062' 2024-04-02T14:38:05.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 -- 172.21.15.62:0/3415010531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f69100a3c40 msgr2=0x7f69100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:05.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f69100a3c40 0x7f69100a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f692c103690 tx=0x7f691c00b410 comp rx=0 tx=0).stop 2024-04-02T14:38:05.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 -- 172.21.15.62:0/3415010531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f692c103830 msgr2=0x7f692c19afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:05.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f692c103830 0x7f692c19afe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f691800e9c0 tx=0x7f691800ecd0 comp rx=0 tx=0).stop 2024-04-02T14:38:05.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 -- 172.21.15.62:0/3415010531 shutdown_connections 2024-04-02T14:38:05.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f69100a3c40 0x7f69100a6100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:05.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f692c1041d0 0x7f692c1a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:05.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f692c102630 0x7f692c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:05.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 --2- 172.21.15.62:0/3415010531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f692c103830 0x7f692c19afe0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:05.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 -- 172.21.15.62:0/3415010531 >> 172.21.15.62:0/3415010531 conn(0x7f692c0fdb90 msgr2=0x7f692c0ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:38:05.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 -- 172.21.15.62:0/3415010531 shutdown_connections 2024-04-02T14:38:05.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:38:05.629+0000 7f6931e00700 1 -- 172.21.15.62:0/3415010531 wait complete. 2024-04-02T14:38:06.491 DEBUG:teuthology.orchestra.run.smithi062:osd.6> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.6.service 2024-04-02T14:38:06.496 INFO:tasks.cephadm:Deploying osd.7 on smithi113 with /dev/vg_nvme/lv_2... 2024-04-02T14:38:06.496 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T14:38:06.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:06 smithi113 ceph-mon[36135]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:06.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:06 smithi113 ceph-mon[36135]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T14:38:06.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:06 smithi113 ceph-mon[36135]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T14:38:06.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:06 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:06.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:06 smithi113 ceph-mon[36135]: from='osd.6 [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:38:06.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:06 smithi113 ceph-mon[36135]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:38:06.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:06 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:06.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:06 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:06.696 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:06 smithi062 ceph-mon[32116]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:06.696 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:06 smithi062 ceph-mon[32116]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T14:38:06.697 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:06 smithi062 ceph-mon[32116]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T14:38:06.697 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:06.697 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:06 smithi062 ceph-mon[32116]: from='osd.6 [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:38:06.697 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:06 smithi062 ceph-mon[32116]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:38:06.697 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:06.697 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:06 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:06.743 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:38:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:06 smithi204 ceph-mon[35948]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:06 smithi204 ceph-mon[35948]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T14:38:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:06 smithi204 ceph-mon[35948]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T14:38:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:06 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:06 smithi204 ceph-mon[35948]: from='osd.6 [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:38:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:06 smithi204 ceph-mon[35948]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:38:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:06 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:06.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:06 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:07 smithi062 ceph-mon[32116]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:38:07.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:07 smithi113 ceph-mon[36135]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:38:07.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:07 smithi113 ceph-mon[36135]: osdmap e47: 7 total, 6 up, 7 in 2024-04-02T14:38:07.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:07 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:07.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:07 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:07.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:07 smithi062 ceph-mon[32116]: osdmap e47: 7 total, 6 up, 7 in 2024-04-02T14:38:07.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:07 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:07.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:07 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:07.678 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:38:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:38:07.382+0000 7fca5b000700 -1 osd.6 0 waiting for initial osdmap 2024-04-02T14:38:07.678 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:38:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:38:07.423+0000 7fca50e00700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:38:07.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:07 smithi204 ceph-mon[35948]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:38:07.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:07 smithi204 ceph-mon[35948]: osdmap e47: 7 total, 6 up, 7 in 2024-04-02T14:38:07.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:07 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:07.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:07 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:08.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:08 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:38:08.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:08 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:38:08.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:08 smithi113 ceph-mon[36135]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:08.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:08 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:08.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:08 smithi113 ceph-mon[36135]: from='osd.6 ' entity='osd.6' 2024-04-02T14:38:08.644 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T14:38:08.644 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T14:38:08.644 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-04-02T14:38:08.644 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-04-02T14:38:08.644 INFO:teuthology.orchestra.run.smithi113.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00792731 s, 1.3 GB/s 2024-04-02T14:38:08.645 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 2024-04-02T14:38:08.645 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-04-02T14:38:08.662 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:08 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:38:08.662 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:08 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:38:08.662 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:08 smithi062 ceph-mon[32116]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:08.662 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:08 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:08.662 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:08 smithi062 ceph-mon[32116]: from='osd.6 ' entity='osd.6' 2024-04-02T14:38:08.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:08 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:38:08.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:08 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:38:08.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:08 smithi204 ceph-mon[35948]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:08.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:08 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:08.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:08 smithi204 ceph-mon[35948]: from='osd.6 ' entity='osd.6' 2024-04-02T14:38:09.334 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi113:vg_nvme/lv_2 2024-04-02T14:38:09.573 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:38:09.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:09.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:09 smithi113 ceph-mon[36135]: osd.6 [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892] boot 2024-04-02T14:38:09.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:09 smithi113 ceph-mon[36135]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T14:38:09.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:09.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:09.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:09 smithi204 ceph-mon[35948]: osd.6 [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892] boot 2024-04-02T14:38:09.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:09 smithi204 ceph-mon[35948]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T14:38:09.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:09.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:09.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:09 smithi062 ceph-mon[32116]: osd.6 [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892] boot 2024-04-02T14:38:09.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:09 smithi062 ceph-mon[32116]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T14:38:09.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:38:10.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:10 smithi113 ceph-mon[36135]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:10.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:10 smithi113 ceph-mon[36135]: osdmap e49: 7 total, 7 up, 7 in 2024-04-02T14:38:10.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:10 smithi062 ceph-mon[32116]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:10.756 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:10 smithi062 ceph-mon[32116]: osdmap e49: 7 total, 7 up, 7 in 2024-04-02T14:38:10.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:10 smithi204 ceph-mon[35948]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T14:38:10.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:10 smithi204 ceph-mon[35948]: osdmap e49: 7 total, 7 up, 7 in 2024-04-02T14:38:11.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.106+0000 7f049325f700 1 -- 172.21.15.113:0/1990474438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f048c1041c0 msgr2=0x7f048c10aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:11.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.106+0000 7f049325f700 1 --2- 172.21.15.113:0/1990474438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f048c1041c0 0x7f048c10aa70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f047c0099c0 tx=0x7f047c009cd0 comp rx=0 tx=0).stop 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.106+0000 7f049325f700 1 -- 172.21.15.113:0/1990474438 shutdown_connections 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.106+0000 7f049325f700 1 --2- 172.21.15.113:0/1990474438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f048c1041c0 0x7f048c10aa70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.106+0000 7f049325f700 1 --2- 172.21.15.113:0/1990474438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f048c102620 0x7f048c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.106+0000 7f049325f700 1 --2- 172.21.15.113:0/1990474438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c103820 0x7f048c103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.106+0000 7f049325f700 1 -- 172.21.15.113:0/1990474438 >> 172.21.15.113:0/1990474438 conn(0x7f048c0fdc00 msgr2=0x7f048c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f049325f700 1 -- 172.21.15.113:0/1990474438 shutdown_connections 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f049325f700 1 -- 172.21.15.113:0/1990474438 wait complete. 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f049325f700 1 Processor -- start 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f049325f700 1 -- start start 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f049325f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f048c102620 0x7f048c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f049325f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f048c103820 0x7f048c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f049325f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c1041c0 0x7f048c19ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:38:11.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f049325f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f048c10e3f0 con 0x7f048c1041c0 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f049325f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f048c10e270 con 0x7f048c102620 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f049325f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f048c10e570 con 0x7f048c103820 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f0490ffb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f048c102620 0x7f048c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f0490ffb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f048c102620 0x7f048c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42178/0 (socket says 172.21.15.113:42178) 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.107+0000 7f0490ffb700 1 -- 172.21.15.113:0/1003933970 learned_addr learned my addr 172.21.15.113:0/1003933970 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f048bfff700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f048c103820 0x7f048c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f04917fc700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c1041c0 0x7f048c19ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f0490ffb700 1 -- 172.21.15.113:0/1003933970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f048c103820 msgr2=0x7f048c198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f0490ffb700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f048c103820 0x7f048c198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f0490ffb700 1 -- 172.21.15.113:0/1003933970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c1041c0 msgr2=0x7f048c19ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f0490ffb700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c1041c0 0x7f048c19ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f0490ffb700 1 -- 172.21.15.113:0/1003933970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c1a0660 con 0x7f048c102620 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f048bfff700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f048c103820 0x7f048c198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f0490ffb700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f048c102620 0x7f048c198a00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f048400b9c0 tx=0x7f048400bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:38:11.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f0489ffb700 1 -- 172.21.15.113:0/1003933970 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484019070 con 0x7f048c102620 2024-04-02T14:38:11.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f049325f700 1 -- 172.21.15.113:0/1003933970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048c1a0950 con 0x7f048c102620 2024-04-02T14:38:11.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f0489ffb700 1 -- 172.21.15.113:0/1003933970 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0484014440 con 0x7f048c102620 2024-04-02T14:38:11.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f0489ffb700 1 -- 172.21.15.113:0/1003933970 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04840125a0 con 0x7f048c102620 2024-04-02T14:38:11.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.108+0000 7f049325f700 1 -- 172.21.15.113:0/1003933970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048c1a0f10 con 0x7f048c102620 2024-04-02T14:38:11.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.109+0000 7f049325f700 1 -- 172.21.15.113:0/1003933970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048c0692b0 con 0x7f048c102620 2024-04-02T14:38:11.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.110+0000 7f0489ffb700 1 -- 172.21.15.113:0/1003933970 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f048401c070 con 0x7f048c102620 2024-04-02T14:38:11.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.111+0000 7f0489ffb700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04780a3c40 0x7f04780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:38:11.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.111+0000 7f0489ffb700 1 -- 172.21.15.113:0/1003933970 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7f04840ceb50 con 0x7f048c102620 2024-04-02T14:38:11.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.111+0000 7f048bfff700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04780a3c40 0x7f04780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:38:11.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.112+0000 7f048bfff700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04780a3c40 0x7f04780a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f04740099d0 tx=0x7f0474008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:38:11.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.116+0000 7f0489ffb700 1 -- 172.21.15.113:0/1003933970 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f048409c240 con 0x7f048c102620 2024-04-02T14:38:11.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:11.318+0000 7f049325f700 1 -- 172.21.15.113:0/1003933970 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f048c062a30 con 0x7f04780a3c40 2024-04-02T14:38:11.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:11 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:38:11.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:11 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:38:11.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:11 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:11.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:11 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:38:11.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:11 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:38:11.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:11 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:11.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:11 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:38:11.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:11 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:38:11.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:11 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:13.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:13.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:38:13.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi062 2024-04-02T14:38:13.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi062 to 5697M 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:13.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.105 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:13.105 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:38:13.105 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi062 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi062 to 5697M 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:13.106 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi062 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi062 to 5697M 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:13.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:13.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:13.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:13.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:13.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:13.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:15.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:14 smithi204 ceph-mon[35948]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:15.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:14 smithi204 ceph-mon[35948]: from='client.? 172.21.15.113:0/1276531256' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3097ff07-ddef-4070-88b8-d3002833f1fb"}]: dispatch 2024-04-02T14:38:15.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:14 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3097ff07-ddef-4070-88b8-d3002833f1fb"}]: dispatch 2024-04-02T14:38:15.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:14 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3097ff07-ddef-4070-88b8-d3002833f1fb"}]': finished 2024-04-02T14:38:15.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:14 smithi204 ceph-mon[35948]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T14:38:15.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:14 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:15.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:14 smithi204 ceph-mon[35948]: from='client.? 172.21.15.113:0/2047717975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:38:15.330 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:14 smithi113 ceph-mon[36135]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:15.330 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:14 smithi113 ceph-mon[36135]: from='client.? 172.21.15.113:0/1276531256' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3097ff07-ddef-4070-88b8-d3002833f1fb"}]: dispatch 2024-04-02T14:38:15.330 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:14 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3097ff07-ddef-4070-88b8-d3002833f1fb"}]: dispatch 2024-04-02T14:38:15.330 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:14 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3097ff07-ddef-4070-88b8-d3002833f1fb"}]': finished 2024-04-02T14:38:15.330 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:14 smithi113 ceph-mon[36135]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T14:38:15.331 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:14 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:15.331 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:14 smithi113 ceph-mon[36135]: from='client.? 172.21.15.113:0/2047717975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:38:15.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:14 smithi062 ceph-mon[32116]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:15.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:14 smithi062 ceph-mon[32116]: from='client.? 172.21.15.113:0/1276531256' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3097ff07-ddef-4070-88b8-d3002833f1fb"}]: dispatch 2024-04-02T14:38:15.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:14 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3097ff07-ddef-4070-88b8-d3002833f1fb"}]: dispatch 2024-04-02T14:38:15.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:14 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3097ff07-ddef-4070-88b8-d3002833f1fb"}]': finished 2024-04-02T14:38:15.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:14 smithi062 ceph-mon[32116]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T14:38:15.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:14 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:15.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:14 smithi062 ceph-mon[32116]: from='client.? 172.21.15.113:0/2047717975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:38:17.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:16 smithi113 ceph-mon[36135]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:17.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:16 smithi204 ceph-mon[35948]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:17.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:16 smithi062 ceph-mon[32116]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:19.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:18 smithi113 ceph-mon[36135]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:19.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:18 smithi204 ceph-mon[35948]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:19.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:18 smithi062 ceph-mon[32116]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:21.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:20 smithi113 ceph-mon[36135]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:21.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:20 smithi204 ceph-mon[35948]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:21.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:20 smithi062 ceph-mon[32116]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:22.180 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T14:38:22.181 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:22.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T14:38:22.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:22.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T14:38:22.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:23.140 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:22 smithi113 ceph-mon[36135]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:23.140 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:22 smithi113 ceph-mon[36135]: Deploying daemon osd.7 on smithi113 2024-04-02T14:38:23.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:22 smithi204 ceph-mon[35948]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:23.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:22 smithi204 ceph-mon[35948]: Deploying daemon osd.7 on smithi113 2024-04-02T14:38:23.278 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:22 smithi062 ceph-mon[32116]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:23.278 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:22 smithi062 ceph-mon[32116]: Deploying daemon osd.7 on smithi113 2024-04-02T14:38:25.070 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:24 smithi113 ceph-mon[36135]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:25.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:24 smithi204 ceph-mon[35948]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:25.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:24 smithi062 ceph-mon[32116]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:27.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:26 smithi113 ceph-mon[36135]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:27.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:27.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:27.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:27.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:26 smithi204 ceph-mon[35948]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:27.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:27.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:27.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:27.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:26 smithi062 ceph-mon[32116]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:27.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:27.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:27.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:28.950 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:28 smithi113 ceph-mon[36135]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:29.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:28 smithi204 ceph-mon[35948]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:29.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:28 smithi062 ceph-mon[32116]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:30.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.551+0000 7f0489ffb700 1 -- 172.21.15.113:0/1003933970 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f048c062a30 con 0x7f04780a3c40 2024-04-02T14:38:30.552 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 7 on host 'smithi113' 2024-04-02T14:38:30.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.552+0000 7f049325f700 1 -- 172.21.15.113:0/1003933970 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04780a3c40 msgr2=0x7f04780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:30.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.552+0000 7f049325f700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04780a3c40 0x7f04780a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f04740099d0 tx=0x7f0474008040 comp rx=0 tx=0).stop 2024-04-02T14:38:30.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.553+0000 7f049325f700 1 -- 172.21.15.113:0/1003933970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f048c102620 msgr2=0x7f048c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:30.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.553+0000 7f049325f700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f048c102620 0x7f048c198a00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f048400b9c0 tx=0x7f048400bd80 comp rx=0 tx=0).stop 2024-04-02T14:38:30.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.553+0000 7f049325f700 1 -- 172.21.15.113:0/1003933970 shutdown_connections 2024-04-02T14:38:30.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.553+0000 7f049325f700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04780a3c40 0x7f04780a6100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:30.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.553+0000 7f049325f700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f048c102620 0x7f048c198a00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:30.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.553+0000 7f049325f700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f048c1041c0 0x7f048c19ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:30.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.553+0000 7f049325f700 1 --2- 172.21.15.113:0/1003933970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f048c103820 0x7f048c198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:30.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.553+0000 7f049325f700 1 -- 172.21.15.113:0/1003933970 >> 172.21.15.113:0/1003933970 conn(0x7f048c0fdc00 msgr2=0x7f048c106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:38:30.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.553+0000 7f049325f700 1 -- 172.21.15.113:0/1003933970 shutdown_connections 2024-04-02T14:38:30.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:38:30.553+0000 7f049325f700 1 -- 172.21.15.113:0/1003933970 wait complete. 2024-04-02T14:38:30.714 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:30 smithi113 ceph-mon[36135]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:30.714 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:30.714 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:30.714 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:30.714 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:30.715 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:30.715 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:30.715 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:30.715 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:31.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:30 smithi204 ceph-mon[35948]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:31.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:31.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:30 smithi062 ceph-mon[32116]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:31.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:31.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:31.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:31.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:31.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:31.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:31.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:31.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:31.254 DEBUG:teuthology.orchestra.run.smithi113:osd.7> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.7.service 2024-04-02T14:38:31.256 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-04-02T14:38:31.257 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T14:38:31.503 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:38:31.798 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:31 smithi113 ceph-mon[36135]: from='osd.7 [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T14:38:31.799 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:31 smithi113 ceph-mon[36135]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T14:38:31.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:31 smithi204 ceph-mon[35948]: from='osd.7 [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T14:38:31.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:31 smithi204 ceph-mon[35948]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T14:38:32.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:31 smithi062 ceph-mon[32116]: from='osd.7 [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T14:38:32.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:31 smithi062 ceph-mon[32116]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T14:38:33.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:32 smithi204 ceph-mon[35948]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:33.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:32 smithi204 ceph-mon[35948]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T14:38:33.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:32 smithi204 ceph-mon[35948]: from='osd.7 [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:38:33.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:32 smithi204 ceph-mon[35948]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T14:38:33.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:33.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:32 smithi204 ceph-mon[35948]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:38:33.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:32 smithi062 ceph-mon[32116]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:33.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:32 smithi062 ceph-mon[32116]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T14:38:33.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:32 smithi062 ceph-mon[32116]: from='osd.7 [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:38:33.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:32 smithi062 ceph-mon[32116]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T14:38:33.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:33.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:32 smithi062 ceph-mon[32116]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:38:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:32 smithi113 ceph-mon[36135]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:32 smithi113 ceph-mon[36135]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T14:38:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:32 smithi113 ceph-mon[36135]: from='osd.7 [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:38:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:32 smithi113 ceph-mon[36135]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T14:38:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:33.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:32 smithi113 ceph-mon[36135]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:38:33.122 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:38:32 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:38:32.710+0000 7fe912dc8700 -1 osd.7 0 waiting for initial osdmap 2024-04-02T14:38:33.123 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:38:32 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:38:32.720+0000 7fe90ebbb700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:38:33.316 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T14:38:33.316 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T14:38:33.317 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-04-02T14:38:33.317 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-04-02T14:38:33.317 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113272 s, 926 MB/s 2024-04-02T14:38:33.317 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-04-02T14:38:33.841 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:33 smithi113 ceph-mon[36135]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:38:33.841 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:33 smithi113 ceph-mon[36135]: osdmap e52: 8 total, 7 up, 8 in 2024-04-02T14:38:33.841 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:33.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:33 smithi204 ceph-mon[35948]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:38:33.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:33 smithi204 ceph-mon[35948]: osdmap e52: 8 total, 7 up, 8 in 2024-04-02T14:38:33.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:33.951 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-04-02T14:38:34.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:33 smithi062 ceph-mon[32116]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:38:34.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:33 smithi062 ceph-mon[32116]: osdmap e52: 8 total, 7 up, 8 in 2024-04-02T14:38:34.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:34.193 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:38:35.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:38:35.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: osd.7 [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396] boot 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi113 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:35.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:35.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.051 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:38:35.051 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:38:35.051 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: osd.7 [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396] boot 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi113 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:35.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:35.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:38:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:38:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-04-02T14:38:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: osd.7 [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396] boot 2024-04-02T14:38:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T14:38:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:38:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi113 2024-04-02T14:38:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:35.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:35.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:38:35.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:35.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:38:35.123 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:35.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.727+0000 7fd76f780700 1 -- 172.21.15.204:0/760105620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd768102790 msgr2=0x7fd768102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:35.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.727+0000 7fd76f780700 1 --2- 172.21.15.204:0/760105620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd768102790 0x7fd768102bf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd758009990 tx=0x7fd758009ca0 comp rx=0 tx=0).stop 2024-04-02T14:38:35.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.727+0000 7fd76f780700 1 -- 172.21.15.204:0/760105620 shutdown_connections 2024-04-02T14:38:35.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.727+0000 7fd76f780700 1 --2- 172.21.15.204:0/760105620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd768101590 0x7fd7681019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:35.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.727+0000 7fd76f780700 1 --2- 172.21.15.204:0/760105620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768103130 0x7fd7681099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:35.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.727+0000 7fd76f780700 1 --2- 172.21.15.204:0/760105620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd768102790 0x7fd768102bf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:35.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.727+0000 7fd76f780700 1 -- 172.21.15.204:0/760105620 >> 172.21.15.204:0/760105620 conn(0x7fd7680fcb50 msgr2=0x7fd7680fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:38:35.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.727+0000 7fd76f780700 1 -- 172.21.15.204:0/760105620 shutdown_connections 2024-04-02T14:38:35.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.727+0000 7fd76f780700 1 -- 172.21.15.204:0/760105620 wait complete. 2024-04-02T14:38:35.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76f780700 1 Processor -- start 2024-04-02T14:38:35.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76f780700 1 -- start start 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76f780700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd768101590 0x7fd768198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76f780700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd768102790 0x7fd768198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76f780700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768103130 0x7fd76819fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76f780700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd76810d2c0 con 0x7fd768103130 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76f780700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd76810d140 con 0x7fd768102790 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76f780700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd76810d440 con 0x7fd768101590 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76d51c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd768101590 0x7fd768198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76d51c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd768101590 0x7fd768198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:52870/0 (socket says 172.21.15.204:52870) 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76d51c700 1 -- 172.21.15.204:0/743678473 learned_addr learned my addr 172.21.15.204:0/743678473 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76cd1b700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd768102790 0x7fd768198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76d51c700 1 -- 172.21.15.204:0/743678473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd768102790 msgr2=0x7fd768198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76d51c700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd768102790 0x7fd768198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:35.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76d51c700 1 -- 172.21.15.204:0/743678473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768103130 msgr2=0x7fd76819fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:35.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76d51c700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768103130 0x7fd76819fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:35.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.728+0000 7fd76d51c700 1 -- 172.21.15.204:0/743678473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7681a0590 con 0x7fd768101590 2024-04-02T14:38:35.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.729+0000 7fd76d51c700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd768101590 0x7fd768198930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c920 tx=0x7fd76400cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:38:35.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.729+0000 7fd75e7fc700 1 -- 172.21.15.204:0/743678473 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764004220 con 0x7fd768101590 2024-04-02T14:38:35.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.729+0000 7fd76f780700 1 -- 172.21.15.204:0/743678473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7681a0880 con 0x7fd768101590 2024-04-02T14:38:35.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.729+0000 7fd75e7fc700 1 -- 172.21.15.204:0/743678473 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd764007dc0 con 0x7fd768101590 2024-04-02T14:38:35.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.729+0000 7fd75e7fc700 1 -- 172.21.15.204:0/743678473 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764003e10 con 0x7fd768101590 2024-04-02T14:38:35.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.729+0000 7fd76f780700 1 -- 172.21.15.204:0/743678473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7681a0dc0 con 0x7fd768101590 2024-04-02T14:38:35.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.730+0000 7fd76f780700 1 -- 172.21.15.204:0/743678473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7680692b0 con 0x7fd768101590 2024-04-02T14:38:35.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.731+0000 7fd75e7fc700 1 -- 172.21.15.204:0/743678473 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd7640078c0 con 0x7fd768101590 2024-04-02T14:38:35.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.731+0000 7fd75e7fc700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd7540a3c40 0x7fd7540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:38:35.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.732+0000 7fd75e7fc700 1 -- 172.21.15.204:0/743678473 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7fd7640cf840 con 0x7fd768101590 2024-04-02T14:38:35.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.732+0000 7fd76cd1b700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd7540a3c40 0x7fd7540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:38:35.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.733+0000 7fd76cd1b700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd7540a3c40 0x7fd7540a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b5c0 tx=0x7fd758005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:38:35.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.735+0000 7fd75e7fc700 1 -- 172.21.15.204:0/743678473 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd764014030 con 0x7fd768101590 2024-04-02T14:38:35.950 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:35.947+0000 7fd76f780700 1 -- 172.21.15.204:0/743678473 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd7680fec40 con 0x7fd7540a3c40 2024-04-02T14:38:36.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:35 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi113 to 5693M 2024-04-02T14:38:36.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:35 smithi204 ceph-mon[35948]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T14:38:36.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:35 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi113 to 5693M 2024-04-02T14:38:36.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:35 smithi062 ceph-mon[32116]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T14:38:36.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:35 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi113 to 5693M 2024-04-02T14:38:36.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:35 smithi113 ceph-mon[36135]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T14:38:36.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:36 smithi204 ceph-mon[35948]: pgmap v202: 1 pgs: 1 unknown; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:36.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:36 smithi204 ceph-mon[35948]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-02T14:38:36.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:36 smithi204 ceph-mon[35948]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T14:38:36.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:36 smithi204 ceph-mon[35948]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:38:36.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:36 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:38:36.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:36 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:38:36.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:36 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:37.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:36 smithi062 ceph-mon[32116]: pgmap v202: 1 pgs: 1 unknown; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:37.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:36 smithi062 ceph-mon[32116]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-02T14:38:37.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:36 smithi062 ceph-mon[32116]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T14:38:37.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:36 smithi062 ceph-mon[32116]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:38:37.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:36 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:38:37.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:36 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:38:37.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:36 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:37.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:36 smithi113 ceph-mon[36135]: pgmap v202: 1 pgs: 1 unknown; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:37.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:36 smithi113 ceph-mon[36135]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-02T14:38:37.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:36 smithi113 ceph-mon[36135]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T14:38:37.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:36 smithi113 ceph-mon[36135]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:38:37.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:36 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:38:37.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:36 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:38:37.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:36 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:37.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:37 smithi204 ceph-mon[35948]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T14:38:38.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:37 smithi062 ceph-mon[32116]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T14:38:38.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:37 smithi113 ceph-mon[36135]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T14:38:39.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:38 smithi062 ceph-mon[32116]: pgmap v205: 1 pgs: 1 unknown; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:39.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:38 smithi062 ceph-mon[32116]: osdmap e57: 8 total, 8 up, 8 in 2024-04-02T14:38:39.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:38 smithi062 ceph-mon[32116]: from='client.? 172.21.15.204:0/2670248455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ef880a-ec50-4a20-9edd-f25fcf4ac28e"}]: dispatch 2024-04-02T14:38:39.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:38 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ef880a-ec50-4a20-9edd-f25fcf4ac28e"}]: dispatch 2024-04-02T14:38:39.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:38 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12ef880a-ec50-4a20-9edd-f25fcf4ac28e"}]': finished 2024-04-02T14:38:39.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:38 smithi062 ceph-mon[32116]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T14:38:39.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:38 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:39.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:38 smithi113 ceph-mon[36135]: pgmap v205: 1 pgs: 1 unknown; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:39.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:38 smithi113 ceph-mon[36135]: osdmap e57: 8 total, 8 up, 8 in 2024-04-02T14:38:39.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:38 smithi113 ceph-mon[36135]: from='client.? 172.21.15.204:0/2670248455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ef880a-ec50-4a20-9edd-f25fcf4ac28e"}]: dispatch 2024-04-02T14:38:39.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:38 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ef880a-ec50-4a20-9edd-f25fcf4ac28e"}]: dispatch 2024-04-02T14:38:39.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:38 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12ef880a-ec50-4a20-9edd-f25fcf4ac28e"}]': finished 2024-04-02T14:38:39.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:38 smithi113 ceph-mon[36135]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T14:38:39.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:38 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:39.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:38 smithi204 ceph-mon[35948]: pgmap v205: 1 pgs: 1 unknown; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:39.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:38 smithi204 ceph-mon[35948]: osdmap e57: 8 total, 8 up, 8 in 2024-04-02T14:38:39.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:38 smithi204 ceph-mon[35948]: from='client.? 172.21.15.204:0/2670248455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ef880a-ec50-4a20-9edd-f25fcf4ac28e"}]: dispatch 2024-04-02T14:38:39.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:38 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ef880a-ec50-4a20-9edd-f25fcf4ac28e"}]: dispatch 2024-04-02T14:38:39.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:38 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12ef880a-ec50-4a20-9edd-f25fcf4ac28e"}]': finished 2024-04-02T14:38:39.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:38 smithi204 ceph-mon[35948]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T14:38:39.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:38 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:40.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:39 smithi062 ceph-mon[32116]: from='client.? 172.21.15.204:0/3743341925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:38:40.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:39 smithi113 ceph-mon[36135]: from='client.? 172.21.15.204:0/3743341925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:38:40.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:39 smithi204 ceph-mon[35948]: from='client.? 172.21.15.204:0/3743341925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:38:41.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:40 smithi062 ceph-mon[32116]: pgmap v208: 1 pgs: 1 unknown; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:41.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:40 smithi113 ceph-mon[36135]: pgmap v208: 1 pgs: 1 unknown; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:41.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:40 smithi204 ceph-mon[35948]: pgmap v208: 1 pgs: 1 unknown; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:42.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:41 smithi113 ceph-mon[36135]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-02T14:38:42.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:41 smithi113 ceph-mon[36135]: Cluster is now healthy 2024-04-02T14:38:42.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:41 smithi204 ceph-mon[35948]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-02T14:38:42.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:41 smithi204 ceph-mon[35948]: Cluster is now healthy 2024-04-02T14:38:42.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:41 smithi062 ceph-mon[32116]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-02T14:38:42.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:41 smithi062 ceph-mon[32116]: Cluster is now healthy 2024-04-02T14:38:43.108 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:42 smithi204 ceph-mon[35948]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:43.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:42 smithi113 ceph-mon[36135]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:43.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:42 smithi062 ceph-mon[32116]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:45.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:44 smithi204 ceph-mon[35948]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:45.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:44 smithi113 ceph-mon[36135]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:45.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:44 smithi062 ceph-mon[32116]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:47.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:46 smithi204 ceph-mon[35948]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:47.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T14:38:47.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:46 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:47.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:46 smithi204 ceph-mon[35948]: Deploying daemon osd.8 on smithi204 2024-04-02T14:38:47.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:46 smithi113 ceph-mon[36135]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:47.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T14:38:47.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:46 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:47.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:46 smithi113 ceph-mon[36135]: Deploying daemon osd.8 on smithi204 2024-04-02T14:38:47.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:46 smithi062 ceph-mon[32116]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:47.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T14:38:47.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:46 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:38:47.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:46 smithi062 ceph-mon[32116]: Deploying daemon osd.8 on smithi204 2024-04-02T14:38:48.927 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:48 smithi204 ceph-mon[35948]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:49.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:48 smithi113 ceph-mon[36135]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:49.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:48 smithi062 ceph-mon[32116]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:51.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:50 smithi204 ceph-mon[35948]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:51.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:50 smithi113 ceph-mon[36135]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:51.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:50 smithi062 ceph-mon[32116]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:53.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:52 smithi204 ceph-mon[35948]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:53.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:52 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:53.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:52 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:53.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:52 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:53.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:52 smithi062 ceph-mon[32116]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:53.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:53.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:53.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:53.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:52 smithi113 ceph-mon[36135]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:53.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:52 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:53.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:52 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:53.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:52 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:38:54.810 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:54 smithi062 ceph-mon[32116]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:54.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:54 smithi204 ceph-mon[35948]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:54.984 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.982+0000 7fd75e7fc700 1 -- 172.21.15.204:0/743678473 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd7680fec40 con 0x7fd7540a3c40 2024-04-02T14:38:54.984 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-04-02T14:38:54.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.984+0000 7fd76f780700 1 -- 172.21.15.204:0/743678473 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd7540a3c40 msgr2=0x7fd7540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:54.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.984+0000 7fd76f780700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd7540a3c40 0x7fd7540a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b5c0 tx=0x7fd758005fd0 comp rx=0 tx=0).stop 2024-04-02T14:38:54.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.984+0000 7fd76f780700 1 -- 172.21.15.204:0/743678473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd768101590 msgr2=0x7fd768198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:38:54.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.984+0000 7fd76f780700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd768101590 0x7fd768198930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c920 tx=0x7fd76400cc30 comp rx=0 tx=0).stop 2024-04-02T14:38:54.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.984+0000 7fd76f780700 1 -- 172.21.15.204:0/743678473 shutdown_connections 2024-04-02T14:38:54.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.984+0000 7fd76f780700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd7540a3c40 0x7fd7540a6100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:54.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.984+0000 7fd76f780700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768103130 0x7fd76819fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:54.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.984+0000 7fd76f780700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd768101590 0x7fd768198930 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:54.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.984+0000 7fd76f780700 1 --2- 172.21.15.204:0/743678473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd768102790 0x7fd768198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:38:54.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.984+0000 7fd76f780700 1 -- 172.21.15.204:0/743678473 >> 172.21.15.204:0/743678473 conn(0x7fd7680fcb50 msgr2=0x7fd7680fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:38:54.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.985+0000 7fd76f780700 1 -- 172.21.15.204:0/743678473 shutdown_connections 2024-04-02T14:38:54.987 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:38:54.985+0000 7fd76f780700 1 -- 172.21.15.204:0/743678473 wait complete. 2024-04-02T14:38:55.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:54 smithi113 ceph-mon[36135]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:55.628 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.8.service 2024-04-02T14:38:55.631 INFO:tasks.cephadm:Deploying osd.9 on smithi062 with /dev/vg_nvme/lv_1... 2024-04-02T14:38:55.631 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T14:38:56.091 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:38:56.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:55 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:56.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:55 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:56.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:55 smithi062 ceph-mon[32116]: from='osd.8 [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T14:38:56.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:55 smithi062 ceph-mon[32116]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T14:38:56.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:55 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:56.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:55 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:56.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:55 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:56.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:55 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:56.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:55 smithi204 ceph-mon[35948]: from='osd.8 [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T14:38:56.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:55 smithi204 ceph-mon[35948]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T14:38:56.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:55 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:56.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:55 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:56.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:55 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:56.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:55 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:56.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:55 smithi113 ceph-mon[36135]: from='osd.8 [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T14:38:56.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:55 smithi113 ceph-mon[36135]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T14:38:56.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:55 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:56.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:55 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:38:57.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:56 smithi204 ceph-mon[35948]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:57.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:56 smithi204 ceph-mon[35948]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T14:38:57.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:56 smithi204 ceph-mon[35948]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T14:38:57.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:56 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:57.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:56 smithi204 ceph-mon[35948]: from='osd.8 [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:38:57.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:56 smithi204 ceph-mon[35948]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:38:57.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:56 smithi062 ceph-mon[32116]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:57.273 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:56 smithi062 ceph-mon[32116]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T14:38:57.273 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:56 smithi062 ceph-mon[32116]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T14:38:57.273 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:56 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:57.273 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:56 smithi062 ceph-mon[32116]: from='osd.8 [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:38:57.273 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:56 smithi062 ceph-mon[32116]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:38:57.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:56 smithi113 ceph-mon[36135]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:57.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:56 smithi113 ceph-mon[36135]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T14:38:57.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:56 smithi113 ceph-mon[36135]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T14:38:57.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:56 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:57.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:56 smithi113 ceph-mon[36135]: from='osd.8 [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:38:57.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:56 smithi113 ceph-mon[36135]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:38:57.993 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:38:57 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:38:57.876+0000 7fdf5d022700 -1 osd.8 0 waiting for initial osdmap 2024-04-02T14:38:58.225 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:57 smithi062 ceph-mon[32116]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:38:58.225 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:57 smithi062 ceph-mon[32116]: osdmap e60: 9 total, 8 up, 9 in 2024-04-02T14:38:58.225 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:57 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:58.225 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:57 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:58.225 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:57 smithi062 ceph-mon[32116]: from='osd.8 ' entity='osd.8' 2024-04-02T14:38:58.225 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:57 smithi062 ceph-mon[32116]: osdmap e61: 9 total, 8 up, 9 in 2024-04-02T14:38:58.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:57 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:58.260 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:38:58 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:38:58.021+0000 7fdf5460c700 -1 osd.8 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:38:58.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:57 smithi204 ceph-mon[35948]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:38:58.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:57 smithi204 ceph-mon[35948]: osdmap e60: 9 total, 8 up, 9 in 2024-04-02T14:38:58.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:57 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:58.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:57 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:58.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:57 smithi204 ceph-mon[35948]: from='osd.8 ' entity='osd.8' 2024-04-02T14:38:58.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:57 smithi204 ceph-mon[35948]: osdmap e61: 9 total, 8 up, 9 in 2024-04-02T14:38:58.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:57 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:58.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:57 smithi113 ceph-mon[36135]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:38:58.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:57 smithi113 ceph-mon[36135]: osdmap e60: 9 total, 8 up, 9 in 2024-04-02T14:38:58.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:57 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:58.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:57 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:58.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:57 smithi113 ceph-mon[36135]: from='osd.8 ' entity='osd.8' 2024-04-02T14:38:58.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:57 smithi113 ceph-mon[36135]: osdmap e61: 9 total, 8 up, 9 in 2024-04-02T14:38:58.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:57 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:59.130 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T14:38:59.130 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T14:38:59.131 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-02T14:38:59.131 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-02T14:38:59.131 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0367438 s, 285 MB/s 2024-04-02T14:38:59.131 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 2024-04-02T14:38:59.131 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-02T14:38:59.182 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:58 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:38:59.182 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:58 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:38:59.182 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:58 smithi204 ceph-mon[35948]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:59.182 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:58 smithi204 ceph-mon[35948]: osd.8 [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626] boot 2024-04-02T14:38:59.182 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:58 smithi204 ceph-mon[35948]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T14:38:59.182 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:38:58 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:59.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:58 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:38:59.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:58 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:38:59.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:58 smithi062 ceph-mon[32116]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:59.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:58 smithi062 ceph-mon[32116]: osd.8 [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626] boot 2024-04-02T14:38:59.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:58 smithi062 ceph-mon[32116]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T14:38:59.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:38:58 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:38:59.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:58 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:38:59.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:58 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:38:59.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:58 smithi113 ceph-mon[36135]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:38:59.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:58 smithi113 ceph-mon[36135]: osd.8 [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626] boot 2024-04-02T14:38:59.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:58 smithi113 ceph-mon[36135]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T14:38:59.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:38:58 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:39:00.151 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi062:vg_nvme/lv_1 2024-04-02T14:39:00.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi204 2024-04-02T14:39:00.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:00.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:00.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:00.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi204 to 5693M 2024-04-02T14:39:00.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:00.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:00.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:39:00.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:00.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:00.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:00.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:00 smithi062 ceph-mon[32116]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi204 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi204 to 5693M 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:39:00.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:00.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:00.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:00.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:00 smithi204 ceph-mon[35948]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T14:39:00.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi204 2024-04-02T14:39:00.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:00.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:00.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:00.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi204 to 5693M 2024-04-02T14:39:00.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:00.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:00.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-04-02T14:39:00.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:00.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:00.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:00.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:00.623 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:00 smithi113 ceph-mon[36135]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T14:39:00.627 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:39:02.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:01 smithi204 ceph-mon[35948]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T14:39:02.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:01 smithi062 ceph-mon[32116]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T14:39:02.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:01 smithi113 ceph-mon[36135]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T14:39:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.924+0000 7f990d800700 1 -- 172.21.15.62:0/2615857378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9908100180 msgr2=0x7f99081005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:39:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.924+0000 7f990d800700 1 --2- 172.21.15.62:0/2615857378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9908100180 0x7f99081005a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98f80098f0 tx=0x7f98f8009c00 comp rx=0 tx=0).stop 2024-04-02T14:39:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.924+0000 7f990d800700 1 -- 172.21.15.62:0/2615857378 shutdown_connections 2024-04-02T14:39:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.924+0000 7f990d800700 1 --2- 172.21.15.62:0/2615857378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9908108290 0x7f990810a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.924+0000 7f990d800700 1 --2- 172.21.15.62:0/2615857378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9908100180 0x7f99081005a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.924+0000 7f990d800700 1 --2- 172.21.15.62:0/2615857378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9908100b70 0x7f9908107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.924+0000 7f990d800700 1 -- 172.21.15.62:0/2615857378 >> 172.21.15.62:0/2615857378 conn(0x7f99080fbd60 msgr2=0x7f99080fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:39:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.924+0000 7f990d800700 1 -- 172.21.15.62:0/2615857378 shutdown_connections 2024-04-02T14:39:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.924+0000 7f990d800700 1 -- 172.21.15.62:0/2615857378 wait complete. 2024-04-02T14:39:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.925+0000 7f990d800700 1 Processor -- start 2024-04-02T14:39:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.925+0000 7f990d800700 1 -- start start 2024-04-02T14:39:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.925+0000 7f990d800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9908100180 0x7f990819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f990d800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9908100b70 0x7f990819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f990d800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9908108290 0x7f99081a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f990d800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f990810e2d0 con 0x7f9908100180 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f990d800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f990810e150 con 0x7f9908100b70 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f990d800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f990810e450 con 0x7f9908108290 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f9907e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9908100180 0x7f990819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f9907e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9908100180 0x7f990819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47110/0 (socket says 172.21.15.62:47110) 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f9907e00700 1 -- 172.21.15.62:0/3550067199 learned_addr learned my addr 172.21.15.62:0/3550067199 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f9907400700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9908100b70 0x7f990819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f990ce00700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9908108290 0x7f99081a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f9907e00700 1 -- 172.21.15.62:0/3550067199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9908108290 msgr2=0x7f99081a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:39:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f9907e00700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9908108290 0x7f99081a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f9907e00700 1 -- 172.21.15.62:0/3550067199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9908100b70 msgr2=0x7f990819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:39:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f9907e00700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9908100b70 0x7f990819aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f9907e00700 1 -- 172.21.15.62:0/3550067199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99081a2710 con 0x7f9908100180 2024-04-02T14:39:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.926+0000 7f9907400700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9908100b70 0x7f990819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:39:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.927+0000 7f9907e00700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9908100180 0x7f990819aab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009230 tx=0x7f98f8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:39:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.927+0000 7f9904c00700 1 -- 172.21.15.62:0/3550067199 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8025070 con 0x7f9908100180 2024-04-02T14:39:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.927+0000 7f9904c00700 1 -- 172.21.15.62:0/3550067199 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f98f800ba60 con 0x7f9908100180 2024-04-02T14:39:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.927+0000 7f9904c00700 1 -- 172.21.15.62:0/3550067199 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8003da0 con 0x7f9908100180 2024-04-02T14:39:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.927+0000 7f990d800700 1 -- 172.21.15.62:0/3550067199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99081a29a0 con 0x7f9908100180 2024-04-02T14:39:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.927+0000 7f990d800700 1 -- 172.21.15.62:0/3550067199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99081a2e50 con 0x7f9908100180 2024-04-02T14:39:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.928+0000 7f990d800700 1 -- 172.21.15.62:0/3550067199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99080692b0 con 0x7f9908100180 2024-04-02T14:39:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.931+0000 7f9904c00700 1 -- 172.21.15.62:0/3550067199 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f98f8028070 con 0x7f9908100180 2024-04-02T14:39:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.932+0000 7f9904c00700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98ec0a3cc0 0x7f98ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:39:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.932+0000 7f9904c00700 1 -- 172.21.15.62:0/3550067199 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5829+0+0 (secure 0 0 0) 0x7f98f80d0620 con 0x7f9908100180 2024-04-02T14:39:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.932+0000 7f9907400700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98ec0a3cc0 0x7f98ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:39:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.933+0000 7f9907400700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98ec0a3cc0 0x7f98ec0a6180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009750 tx=0x7f98f4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:39:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:02.935+0000 7f9904c00700 1 -- 172.21.15.62:0/3550067199 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f98f802f020 con 0x7f9908100180 2024-04-02T14:39:03.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:03.182+0000 7f990d800700 1 -- 172.21.15.62:0/3550067199 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f99080fd860 con 0x7f98ec0a3cc0 2024-04-02T14:39:03.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:03 smithi204 ceph-mon[35948]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:03.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:03 smithi204 ceph-mon[35948]: osdmap e65: 9 total, 9 up, 9 in 2024-04-02T14:39:03.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:03 smithi062 ceph-mon[32116]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:03.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:03 smithi062 ceph-mon[32116]: osdmap e65: 9 total, 9 up, 9 in 2024-04-02T14:39:03.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:03 smithi113 ceph-mon[36135]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:03.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:03 smithi113 ceph-mon[36135]: osdmap e65: 9 total, 9 up, 9 in 2024-04-02T14:39:04.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:04 smithi204 ceph-mon[35948]: osdmap e66: 9 total, 9 up, 9 in 2024-04-02T14:39:04.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:04 smithi204 ceph-mon[35948]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:39:04.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:39:04.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:39:04.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:04.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:04 smithi062 ceph-mon[32116]: osdmap e66: 9 total, 9 up, 9 in 2024-04-02T14:39:04.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:04 smithi062 ceph-mon[32116]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:39:04.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:39:04.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:39:04.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:04.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:04 smithi113 ceph-mon[36135]: osdmap e66: 9 total, 9 up, 9 in 2024-04-02T14:39:04.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:04 smithi113 ceph-mon[36135]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:39:04.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:39:04.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:39:04.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:05.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:05 smithi062 ceph-mon[32116]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:05.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:05 smithi113 ceph-mon[36135]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:05.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:05 smithi204 ceph-mon[35948]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:07.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:07 smithi062 ceph-mon[32116]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-02T14:39:07.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:07 smithi113 ceph-mon[36135]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-02T14:39:07.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:07 smithi204 ceph-mon[35948]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-02T14:39:08.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:08 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1990071970' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f2258db-f7e1-432b-be11-a2299b46c30f"}]: dispatch 2024-04-02T14:39:08.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:08 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f2258db-f7e1-432b-be11-a2299b46c30f"}]: dispatch 2024-04-02T14:39:08.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:08 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f2258db-f7e1-432b-be11-a2299b46c30f"}]': finished 2024-04-02T14:39:08.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:08 smithi062 ceph-mon[32116]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T14:39:08.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:08 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:08.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:08 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1990071970' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f2258db-f7e1-432b-be11-a2299b46c30f"}]: dispatch 2024-04-02T14:39:08.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:08 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f2258db-f7e1-432b-be11-a2299b46c30f"}]: dispatch 2024-04-02T14:39:08.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:08 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f2258db-f7e1-432b-be11-a2299b46c30f"}]': finished 2024-04-02T14:39:08.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:08 smithi113 ceph-mon[36135]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T14:39:08.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:08 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:08.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:08 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1990071970' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f2258db-f7e1-432b-be11-a2299b46c30f"}]: dispatch 2024-04-02T14:39:08.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:08 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f2258db-f7e1-432b-be11-a2299b46c30f"}]: dispatch 2024-04-02T14:39:08.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:08 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f2258db-f7e1-432b-be11-a2299b46c30f"}]': finished 2024-04-02T14:39:08.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:08 smithi204 ceph-mon[35948]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T14:39:08.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:08 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:09.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:09 smithi062 ceph-mon[32116]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-02T14:39:09.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:09 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2461045687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:39:09.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:09 smithi113 ceph-mon[36135]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-02T14:39:09.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:09 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2461045687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:39:09.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:09 smithi204 ceph-mon[35948]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-02T14:39:09.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:09 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2461045687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:39:11.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:11 smithi062 ceph-mon[32116]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-02T14:39:11.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:11 smithi113 ceph-mon[36135]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-02T14:39:11.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:11 smithi204 ceph-mon[35948]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-02T14:39:13.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:13 smithi062 ceph-mon[32116]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-02T14:39:13.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:13 smithi113 ceph-mon[36135]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-02T14:39:13.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:13 smithi204 ceph-mon[35948]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-02T14:39:15.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:15 smithi062 ceph-mon[32116]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T14:39:15.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:15 smithi113 ceph-mon[36135]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T14:39:15.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:15 smithi204 ceph-mon[35948]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T14:39:17.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:17 smithi062 ceph-mon[32116]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:17.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:17 smithi113 ceph-mon[36135]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:17.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:17 smithi204 ceph-mon[35948]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:18.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:18 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T14:39:18.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:18 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:18.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:18 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T14:39:18.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:18 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:18.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:18 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T14:39:18.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:18 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:19.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:19 smithi062 ceph-mon[32116]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:19.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:19 smithi062 ceph-mon[32116]: Deploying daemon osd.9 on smithi062 2024-04-02T14:39:19.704 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:19 smithi204 ceph-mon[35948]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:19.704 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:19 smithi204 ceph-mon[35948]: Deploying daemon osd.9 on smithi062 2024-04-02T14:39:19.704 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:19 smithi113 ceph-mon[36135]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:19.704 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:19 smithi113 ceph-mon[36135]: Deploying daemon osd.9 on smithi062 2024-04-02T14:39:21.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:21 smithi062 ceph-mon[32116]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:21.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:21 smithi113 ceph-mon[36135]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:21.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:21 smithi204 ceph-mon[35948]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:23.289 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:23 smithi062 ceph-mon[32116]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:23.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:23 smithi113 ceph-mon[36135]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:23.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:23 smithi204 ceph-mon[35948]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:25.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:25 smithi062 ceph-mon[32116]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:25.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:25 smithi113 ceph-mon[36135]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:25.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:25 smithi204 ceph-mon[35948]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:27.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:26 smithi062 ceph-mon[32116]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:27.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:27.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:27.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:27.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:26 smithi204 ceph-mon[35948]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:27.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:27.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:27.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:27.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:26 smithi113 ceph-mon[36135]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:27.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:27.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:27.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:29.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:28 smithi204 ceph-mon[35948]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:29.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:28 smithi062 ceph-mon[32116]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:29.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:28 smithi113 ceph-mon[36135]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:31.006 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:30 smithi062 ceph-mon[32116]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:31.006 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:31.006 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:30 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:31.006 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:30 smithi062 ceph-mon[32116]: from='osd.9 [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T14:39:31.006 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:30 smithi062 ceph-mon[32116]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T14:39:31.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:30 smithi113 ceph-mon[36135]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:30 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:30 smithi113 ceph-mon[36135]: from='osd.9 [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T14:39:31.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:30 smithi113 ceph-mon[36135]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T14:39:31.137 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 9 on host 'smithi062' 2024-04-02T14:39:31.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.134+0000 7f9904c00700 1 -- 172.21.15.62:0/3550067199 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f99080fd860 con 0x7f98ec0a3cc0 2024-04-02T14:39:31.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.139+0000 7f990d800700 1 -- 172.21.15.62:0/3550067199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98ec0a3cc0 msgr2=0x7f98ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:39:31.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.139+0000 7f990d800700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98ec0a3cc0 0x7f98ec0a6180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009750 tx=0x7f98f4009310 comp rx=0 tx=0).stop 2024-04-02T14:39:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.139+0000 7f990d800700 1 -- 172.21.15.62:0/3550067199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9908100180 msgr2=0x7f990819aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:39:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.139+0000 7f990d800700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9908100180 0x7f990819aab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009230 tx=0x7f98f8005040 comp rx=0 tx=0).stop 2024-04-02T14:39:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.139+0000 7f990d800700 1 -- 172.21.15.62:0/3550067199 shutdown_connections 2024-04-02T14:39:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.139+0000 7f990d800700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98ec0a3cc0 0x7f98ec0a6180 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.139+0000 7f990d800700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9908108290 0x7f99081a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.139+0000 7f990d800700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9908100180 0x7f990819aab0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.139+0000 7f990d800700 1 --2- 172.21.15.62:0/3550067199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9908100b70 0x7f990819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.139+0000 7f990d800700 1 -- 172.21.15.62:0/3550067199 >> 172.21.15.62:0/3550067199 conn(0x7f99080fbd60 msgr2=0x7f99080fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:39:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.139+0000 7f990d800700 1 -- 172.21.15.62:0/3550067199 shutdown_connections 2024-04-02T14:39:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:39:31.140+0000 7f990d800700 1 -- 172.21.15.62:0/3550067199 wait complete. 2024-04-02T14:39:31.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:30 smithi204 ceph-mon[35948]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:31.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:31.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:30 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:31.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:30 smithi204 ceph-mon[35948]: from='osd.9 [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T14:39:31.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:30 smithi204 ceph-mon[35948]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T14:39:31.996 DEBUG:teuthology.orchestra.run.smithi062:osd.9> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.9.service 2024-04-02T14:39:32.000 INFO:tasks.cephadm:Deploying osd.10 on smithi113 with /dev/vg_nvme/lv_1... 2024-04-02T14:39:32.000 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T14:39:32.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:31 smithi113 ceph-mon[36135]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T14:39:32.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:31 smithi113 ceph-mon[36135]: from='osd.9 [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:39:32.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:31 smithi113 ceph-mon[36135]: osdmap e68: 10 total, 9 up, 10 in 2024-04-02T14:39:32.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:31 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:32.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:31 smithi113 ceph-mon[36135]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:39:32.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:31 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:32.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:31 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:32.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:31 smithi062 ceph-mon[32116]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T14:39:32.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:31 smithi062 ceph-mon[32116]: from='osd.9 [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:39:32.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:31 smithi062 ceph-mon[32116]: osdmap e68: 10 total, 9 up, 10 in 2024-04-02T14:39:32.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:31 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:32.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:31 smithi062 ceph-mon[32116]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:39:32.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:31 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:32.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:31 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:32.248 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:39:32.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:31 smithi204 ceph-mon[35948]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T14:39:32.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:31 smithi204 ceph-mon[35948]: from='osd.9 [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:39:32.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:31 smithi204 ceph-mon[35948]: osdmap e68: 10 total, 9 up, 10 in 2024-04-02T14:39:32.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:31 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:32.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:31 smithi204 ceph-mon[35948]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-02T14:39:32.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:31 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:32.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:31 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:33.025 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:32 smithi113 ceph-mon[36135]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:33.025 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:32 smithi113 ceph-mon[36135]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:39:33.025 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:32 smithi113 ceph-mon[36135]: osdmap e69: 10 total, 9 up, 10 in 2024-04-02T14:39:33.025 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:33.026 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:32 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:33.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:32 smithi204 ceph-mon[35948]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:33.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:32 smithi204 ceph-mon[35948]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:39:33.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:32 smithi204 ceph-mon[35948]: osdmap e69: 10 total, 9 up, 10 in 2024-04-02T14:39:33.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:33.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:32 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:33.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:32 smithi062 ceph-mon[32116]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:33.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:32 smithi062 ceph-mon[32116]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-02T14:39:33.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:32 smithi062 ceph-mon[32116]: osdmap e69: 10 total, 9 up, 10 in 2024-04-02T14:39:33.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:33.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:32 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:33.347 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:39:32 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:39:32.933+0000 7ffbe3a00700 -1 osd.9 0 waiting for initial osdmap 2024-04-02T14:39:33.347 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:39:32 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:39:32.969+0000 7ffbd9800700 -1 osd.9 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:39:34.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:33 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:39:34.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:33 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:39:34.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:33 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:34.095 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:33 smithi062 ceph-mon[32116]: from='osd.9 ' entity='osd.9' 2024-04-02T14:39:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:33 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:39:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:33 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:39:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:33 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:34.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:33 smithi113 ceph-mon[36135]: from='osd.9 ' entity='osd.9' 2024-04-02T14:39:34.242 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T14:39:34.243 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T14:39:34.243 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-04-02T14:39:34.243 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-04-02T14:39:34.243 INFO:teuthology.orchestra.run.smithi113.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0094186 s, 1.1 GB/s 2024-04-02T14:39:34.243 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-04-02T14:39:34.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:33 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:39:34.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:33 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:39:34.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:33 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:34.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:33 smithi204 ceph-mon[35948]: from='osd.9 ' entity='osd.9' 2024-04-02T14:39:34.900 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi113:vg_nvme/lv_1 2024-04-02T14:39:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:34 smithi113 ceph-mon[36135]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:34 smithi113 ceph-mon[36135]: osd.9 [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320] boot 2024-04-02T14:39:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:34 smithi113 ceph-mon[36135]: osdmap e70: 10 total, 10 up, 10 in 2024-04-02T14:39:35.122 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:34 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:35.140 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:39:35.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:34 smithi204 ceph-mon[35948]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:35.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:35.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:34 smithi204 ceph-mon[35948]: osd.9 [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320] boot 2024-04-02T14:39:35.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:34 smithi204 ceph-mon[35948]: osdmap e70: 10 total, 10 up, 10 in 2024-04-02T14:39:35.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:34 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:35.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:34 smithi062 ceph-mon[32116]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-02T14:39:35.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:35.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:34 smithi062 ceph-mon[32116]: osd.9 [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320] boot 2024-04-02T14:39:35.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:34 smithi062 ceph-mon[32116]: osdmap e70: 10 total, 10 up, 10 in 2024-04-02T14:39:35.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:34 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:39:36.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:35 smithi204 ceph-mon[35948]: osdmap e71: 10 total, 10 up, 10 in 2024-04-02T14:39:36.298 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:35 smithi113 ceph-mon[36135]: osdmap e71: 10 total, 10 up, 10 in 2024-04-02T14:39:36.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:35 smithi062 ceph-mon[32116]: osdmap e71: 10 total, 10 up, 10 in 2024-04-02T14:39:36.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.778+0000 7feda89b4700 1 -- 172.21.15.113:0/2111336835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda41008e0 msgr2=0x7feda4100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:39:36.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.778+0000 7feda89b4700 1 --2- 172.21.15.113:0/2111336835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda41008e0 0x7feda4100d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed94009990 tx=0x7fed94009ca0 comp rx=0 tx=0).stop 2024-04-02T14:39:36.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.778+0000 7feda89b4700 1 -- 172.21.15.113:0/2111336835 shutdown_connections 2024-04-02T14:39:36.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.778+0000 7feda89b4700 1 --2- 172.21.15.113:0/2111336835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda41008e0 0x7feda4100d00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:36.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.778+0000 7feda89b4700 1 --2- 172.21.15.113:0/2111336835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda4104260 0x7feda410ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:36.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.778+0000 7feda89b4700 1 --2- 172.21.15.113:0/2111336835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feda41038c0 0x7feda4103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:36.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.778+0000 7feda89b4700 1 -- 172.21.15.113:0/2111336835 >> 172.21.15.113:0/2111336835 conn(0x7feda40fc200 msgr2=0x7feda40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:39:36.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.778+0000 7feda89b4700 1 -- 172.21.15.113:0/2111336835 shutdown_connections 2024-04-02T14:39:36.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.778+0000 7feda89b4700 1 -- 172.21.15.113:0/2111336835 wait complete. 2024-04-02T14:39:36.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda89b4700 1 Processor -- start 2024-04-02T14:39:36.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda89b4700 1 -- start start 2024-04-02T14:39:36.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda89b4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda41008e0 0x7feda4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:39:36.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda89b4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda41038c0 0x7feda4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:39:36.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda89b4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feda4104260 0x7feda419ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:39:36.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda89b4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feda410e3b0 con 0x7feda41008e0 2024-04-02T14:39:36.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda89b4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feda410e230 con 0x7feda41038c0 2024-04-02T14:39:36.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda89b4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feda410e530 con 0x7feda4104260 2024-04-02T14:39:36.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda1d9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda41038c0 0x7feda4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:39:36.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda1d9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda41038c0 0x7feda4198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58932/0 (socket says 172.21.15.113:58932) 2024-04-02T14:39:36.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda1d9b700 1 -- 172.21.15.113:0/2492709184 learned_addr learned my addr 172.21.15.113:0/2492709184 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda2d9d700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feda4104260 0x7feda419ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda1d9b700 1 -- 172.21.15.113:0/2492709184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feda4104260 msgr2=0x7feda419ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda1d9b700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feda4104260 0x7feda419ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda1d9b700 1 -- 172.21.15.113:0/2492709184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda41008e0 msgr2=0x7feda4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda1d9b700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda41008e0 0x7feda4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.779+0000 7feda1d9b700 1 -- 172.21.15.113:0/2492709184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda41a05f0 con 0x7feda41038c0 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.780+0000 7feda1d9b700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda41038c0 0x7feda4198ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fed9800e9c0 tx=0x7fed9800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.780+0000 7fed937fe700 1 -- 172.21.15.113:0/2492709184 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9800cbd0 con 0x7feda41038c0 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.780+0000 7fed937fe700 1 -- 172.21.15.113:0/2492709184 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fed9800cd70 con 0x7feda41038c0 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.780+0000 7fed937fe700 1 -- 172.21.15.113:0/2492709184 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98010430 con 0x7feda41038c0 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.780+0000 7feda89b4700 1 -- 172.21.15.113:0/2492709184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda41a08e0 con 0x7feda41038c0 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.780+0000 7feda89b4700 1 -- 172.21.15.113:0/2492709184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda41a0e50 con 0x7feda41038c0 2024-04-02T14:39:36.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.781+0000 7feda89b4700 1 -- 172.21.15.113:0/2492709184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda40692b0 con 0x7feda41038c0 2024-04-02T14:39:36.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.782+0000 7fed937fe700 1 -- 172.21.15.113:0/2492709184 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fed98004e50 con 0x7feda41038c0 2024-04-02T14:39:36.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.782+0000 7fed937fe700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fed8c0a3c40 0x7fed8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:39:36.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.782+0000 7fed937fe700 1 -- 172.21.15.113:0/2492709184 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6261+0+0 (secure 0 0 0) 0x7fed980cfef0 con 0x7feda41038c0 2024-04-02T14:39:36.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.783+0000 7feda259c700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fed8c0a3c40 0x7fed8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:39:36.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.784+0000 7feda259c700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fed8c0a3c40 0x7fed8c0a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fed940098e0 tx=0x7fed94005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:39:36.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.787+0000 7fed937fe700 1 -- 172.21.15.113:0/2492709184 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fed9809d0d0 con 0x7feda41038c0 2024-04-02T14:39:36.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:36.995+0000 7feda89b4700 1 -- 172.21.15.113:0/2492709184 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7feda4062a30 con 0x7fed8c0a3c40 2024-04-02T14:39:37.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:36 smithi204 ceph-mon[35948]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:37.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:36 smithi062 ceph-mon[32116]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:37.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:36 smithi113 ceph-mon[36135]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:38.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:38.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:38.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:38.342 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:38.343 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:38.343 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:38.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:39:38.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:39:38.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:39:38.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:38.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:38.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:38.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:39:38.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:38.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:38.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:38.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:39.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:38 smithi062 ceph-mon[32116]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:39.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:38 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi062 2024-04-02T14:39:39.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:38 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi062 to 4272M 2024-04-02T14:39:39.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:38 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:39.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:38 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:39.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:38 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:39.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:38 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:39.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:38 smithi204 ceph-mon[35948]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:39.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:38 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi062 2024-04-02T14:39:39.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:38 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi062 to 4272M 2024-04-02T14:39:39.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:38 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:39.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:38 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:39.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:38 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:39.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:38 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:39.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:38 smithi113 ceph-mon[36135]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:38 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi062 2024-04-02T14:39:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:38 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi062 to 4272M 2024-04-02T14:39:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:38 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:38 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:38 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:39:39.372 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:38 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:40.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:39 smithi204 ceph-mon[35948]: from='client.? 172.21.15.113:0/3642015060' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ccb9dfd-397e-4996-a37d-1161181a34cd"}]: dispatch 2024-04-02T14:39:40.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:39 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ccb9dfd-397e-4996-a37d-1161181a34cd"}]: dispatch 2024-04-02T14:39:40.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:39 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ccb9dfd-397e-4996-a37d-1161181a34cd"}]': finished 2024-04-02T14:39:40.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:39 smithi204 ceph-mon[35948]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T14:39:40.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:39 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:39 smithi062 ceph-mon[32116]: from='client.? 172.21.15.113:0/3642015060' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ccb9dfd-397e-4996-a37d-1161181a34cd"}]: dispatch 2024-04-02T14:39:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:39 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ccb9dfd-397e-4996-a37d-1161181a34cd"}]: dispatch 2024-04-02T14:39:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:39 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ccb9dfd-397e-4996-a37d-1161181a34cd"}]': finished 2024-04-02T14:39:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:39 smithi062 ceph-mon[32116]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T14:39:40.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:39 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:40.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:39 smithi113 ceph-mon[36135]: from='client.? 172.21.15.113:0/3642015060' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ccb9dfd-397e-4996-a37d-1161181a34cd"}]: dispatch 2024-04-02T14:39:40.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:39 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ccb9dfd-397e-4996-a37d-1161181a34cd"}]: dispatch 2024-04-02T14:39:40.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:39 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ccb9dfd-397e-4996-a37d-1161181a34cd"}]': finished 2024-04-02T14:39:40.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:39 smithi113 ceph-mon[36135]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T14:39:40.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:39 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:41.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:40 smithi204 ceph-mon[35948]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:41.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:40 smithi204 ceph-mon[35948]: from='client.? 172.21.15.113:0/1027916111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:39:41.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:40 smithi062 ceph-mon[32116]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:41.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:40 smithi062 ceph-mon[32116]: from='client.? 172.21.15.113:0/1027916111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:39:41.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:40 smithi113 ceph-mon[36135]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:41.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:40 smithi113 ceph-mon[36135]: from='client.? 172.21.15.113:0/1027916111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:39:43.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:42 smithi204 ceph-mon[35948]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:43.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:42 smithi062 ceph-mon[32116]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:43.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:42 smithi113 ceph-mon[36135]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:45.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:45 smithi204 ceph-mon[35948]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:45.295 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:45 smithi113 ceph-mon[36135]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:45.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:45 smithi062 ceph-mon[32116]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:47.260 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:47 smithi204 ceph-mon[35948]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:47.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:47 smithi062 ceph-mon[32116]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:47.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:47 smithi113 ceph-mon[36135]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:48.270 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:48 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T14:39:48.270 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:48 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:48.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:48 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T14:39:48.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:48 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:48.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:48 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T14:39:48.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:48 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:39:49.248 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:49 smithi113 ceph-mon[36135]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:49.248 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:49 smithi113 ceph-mon[36135]: Deploying daemon osd.10 on smithi113 2024-04-02T14:39:49.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:49 smithi062 ceph-mon[32116]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:49.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:49 smithi062 ceph-mon[32116]: Deploying daemon osd.10 on smithi113 2024-04-02T14:39:49.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:49 smithi204 ceph-mon[35948]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:49.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:49 smithi204 ceph-mon[35948]: Deploying daemon osd.10 on smithi113 2024-04-02T14:39:51.189 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:51 smithi113 ceph-mon[36135]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:51.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:51 smithi062 ceph-mon[32116]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:51.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:51 smithi204 ceph-mon[35948]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:53.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:53 smithi062 ceph-mon[32116]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:53.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:53 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:53.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:53 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:53.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:53 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:53.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:53 smithi113 ceph-mon[36135]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:53.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:53 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:53.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:53 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:53.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:53 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:53.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:53 smithi204 ceph-mon[35948]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:53.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:53 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:53.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:53 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:53.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:53 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:39:55.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:55 smithi062 ceph-mon[32116]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:55.258 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:55 smithi113 ceph-mon[36135]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:55.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:55 smithi204 ceph-mon[35948]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:56.481 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 10 on host 'smithi113' 2024-04-02T14:39:56.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.480+0000 7fed937fe700 1 -- 172.21.15.113:0/2492709184 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7feda4062a30 con 0x7fed8c0a3c40 2024-04-02T14:39:56.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 -- 172.21.15.113:0/2492709184 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fed8c0a3c40 msgr2=0x7fed8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:39:56.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fed8c0a3c40 0x7fed8c0a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fed940098e0 tx=0x7fed94005fb0 comp rx=0 tx=0).stop 2024-04-02T14:39:56.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 -- 172.21.15.113:0/2492709184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda41038c0 msgr2=0x7feda4198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:39:56.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda41038c0 0x7feda4198ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fed9800e9c0 tx=0x7fed9800ecd0 comp rx=0 tx=0).stop 2024-04-02T14:39:56.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 -- 172.21.15.113:0/2492709184 shutdown_connections 2024-04-02T14:39:56.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fed8c0a3c40 0x7fed8c0a6100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:56.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feda4104260 0x7feda419ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:56.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda41008e0 0x7feda4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:56.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 --2- 172.21.15.113:0/2492709184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda41038c0 0x7feda4198ed0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:39:56.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 -- 172.21.15.113:0/2492709184 >> 172.21.15.113:0/2492709184 conn(0x7feda40fc200 msgr2=0x7feda4106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:39:56.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 -- 172.21.15.113:0/2492709184 shutdown_connections 2024-04-02T14:39:56.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-02T14:39:56.482+0000 7feda89b4700 1 -- 172.21.15.113:0/2492709184 wait complete. 2024-04-02T14:39:57.061 DEBUG:teuthology.orchestra.run.smithi113:osd.10> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.10.service 2024-04-02T14:39:57.064 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-04-02T14:39:57.064 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T14:39:57.315 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:39:57.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:57 smithi062 ceph-mon[32116]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:57.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:57 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:57.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:57 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:57.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:57 smithi062 ceph-mon[32116]: from='osd.10 [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T14:39:57.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:57 smithi062 ceph-mon[32116]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T14:39:57.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:57 smithi113 ceph-mon[36135]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:57.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:57 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:57.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:57 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:57.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:57 smithi113 ceph-mon[36135]: from='osd.10 [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T14:39:57.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:57 smithi113 ceph-mon[36135]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T14:39:57.436 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:57 smithi204 ceph-mon[35948]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:57.436 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:57 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:57.437 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:57 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:57.437 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:57 smithi204 ceph-mon[35948]: from='osd.10 [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T14:39:57.437 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:57 smithi204 ceph-mon[35948]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T14:39:58.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:58 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:58.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:58 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:58.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:58 smithi204 ceph-mon[35948]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T14:39:58.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:58 smithi204 ceph-mon[35948]: osdmap e73: 11 total, 10 up, 11 in 2024-04-02T14:39:58.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:58 smithi204 ceph-mon[35948]: from='osd.10 [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:39:58.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:58 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:58.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:58 smithi204 ceph-mon[35948]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:39:58.482 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:58 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:58.483 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:58 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:58.483 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:58 smithi113 ceph-mon[36135]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T14:39:58.483 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:58 smithi113 ceph-mon[36135]: osdmap e73: 11 total, 10 up, 11 in 2024-04-02T14:39:58.483 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:58 smithi113 ceph-mon[36135]: from='osd.10 [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:39:58.483 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:58 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:58.483 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:58 smithi113 ceph-mon[36135]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:39:58.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:58 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:58.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:58 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:39:58.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:58 smithi062 ceph-mon[32116]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T14:39:58.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:58 smithi062 ceph-mon[32116]: osdmap e73: 11 total, 10 up, 11 in 2024-04-02T14:39:58.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:58 smithi062 ceph-mon[32116]: from='osd.10 [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:39:58.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:58 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:58.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:58 smithi062 ceph-mon[32116]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-02T14:39:58.789 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:39:58 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:39:58.480+0000 7f937bade700 -1 osd.10 0 waiting for initial osdmap 2024-04-02T14:39:58.790 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:39:58 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:39:58.494+0000 7f93778d1700 -1 osd.10 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:39:59.282 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:59 smithi204 ceph-mon[35948]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:59.283 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:59 smithi204 ceph-mon[35948]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:39:59.283 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:59 smithi204 ceph-mon[35948]: osdmap e74: 11 total, 10 up, 11 in 2024-04-02T14:39:59.283 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:59 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:59.283 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:39:59 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:59.340 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T14:39:59.340 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T14:39:59.340 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-04-02T14:39:59.340 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-04-02T14:39:59.340 INFO:teuthology.orchestra.run.smithi204.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112134 s, 935 MB/s 2024-04-02T14:39:59.340 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-04-02T14:39:59.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:59 smithi113 ceph-mon[36135]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:59.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:59 smithi113 ceph-mon[36135]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:39:59.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:59 smithi113 ceph-mon[36135]: osdmap e74: 11 total, 10 up, 11 in 2024-04-02T14:39:59.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:59 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:59.371 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:39:59 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:59.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:59 smithi062 ceph-mon[32116]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:39:59.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:59 smithi062 ceph-mon[32116]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-02T14:39:59.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:59 smithi062 ceph-mon[32116]: osdmap e74: 11 total, 10 up, 11 in 2024-04-02T14:39:59.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:59 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:59.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:39:59 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:39:59.981 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-04-02T14:40:00.229 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:40:00.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:00 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:40:00.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:00 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:40:00.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:00 smithi204 ceph-mon[35948]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:40:00.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:00 smithi204 ceph-mon[35948]: osd.10 [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940] boot 2024-04-02T14:40:00.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:00 smithi204 ceph-mon[35948]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T14:40:00.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:40:00.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:00 smithi204 ceph-mon[35948]: overall HEALTH_OK 2024-04-02T14:40:00.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:00 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:40:00.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:00 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:40:00.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:00 smithi062 ceph-mon[32116]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:40:00.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:00 smithi062 ceph-mon[32116]: osd.10 [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940] boot 2024-04-02T14:40:00.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:00 smithi062 ceph-mon[32116]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T14:40:00.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:40:00.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:00 smithi062 ceph-mon[32116]: overall HEALTH_OK 2024-04-02T14:40:00.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:00 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:40:00.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:00 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:40:00.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:00 smithi113 ceph-mon[36135]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 894 GiB / 894 GiB avail 2024-04-02T14:40:00.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:00 smithi113 ceph-mon[36135]: osd.10 [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940] boot 2024-04-02T14:40:00.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:00 smithi113 ceph-mon[36135]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T14:40:00.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:40:00.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:00 smithi113 ceph-mon[36135]: overall HEALTH_OK 2024-04-02T14:40:01.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.724+0000 7f5aaf28a700 1 -- 172.21.15.204:0/4281033257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8105a30 msgr2=0x7f5aa8107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.724+0000 7f5aaf28a700 1 --2- 172.21.15.204:0/4281033257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8105a30 0x7f5aa8107e20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009990 tx=0x7f5a98009ca0 comp rx=0 tx=0).stop 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 -- 172.21.15.204:0/4281033257 shutdown_connections 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 --2- 172.21.15.204:0/4281033257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aa8103100 0x7f5aa81054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 --2- 172.21.15.204:0/4281033257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa8108360 0x7f5aa810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 --2- 172.21.15.204:0/4281033257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8105a30 0x7f5aa8107e20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 -- 172.21.15.204:0/4281033257 >> 172.21.15.204:0/4281033257 conn(0x7f5aa80fcb10 msgr2=0x7f5aa80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 -- 172.21.15.204:0/4281033257 shutdown_connections 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 -- 172.21.15.204:0/4281033257 wait complete. 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 Processor -- start 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 -- start start 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8103100 0x7f5aa8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:01.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aa8105a30 0x7f5aa8198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa8108360 0x7f5aa819fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa810e400 con 0x7f5aa8108360 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa810e280 con 0x7f5aa8105a30 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aaf28a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa810e580 con 0x7f5aa8103100 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.725+0000 7f5aad026700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8103100 0x7f5aa8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5aad026700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8103100 0x7f5aa8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43550/0 (socket says 172.21.15.204:43550) 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5aad026700 1 -- 172.21.15.204:0/989380508 learned_addr learned my addr 172.21.15.204:0/989380508 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5aac825700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aa8105a30 0x7f5aa8198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5aad026700 1 -- 172.21.15.204:0/989380508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aa8105a30 msgr2=0x7f5aa8198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5aad026700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aa8105a30 0x7f5aa8198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5aad026700 1 -- 172.21.15.204:0/989380508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa8108360 msgr2=0x7f5aa819fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5aad026700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa8108360 0x7f5aa819fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5aad026700 1 -- 172.21.15.204:0/989380508 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa81a0590 con 0x7f5aa8103100 2024-04-02T14:40:01.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5aad026700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8103100 0x7f5aa8198930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c9e0 tx=0x7f5aa400cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:01.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5a9e7fc700 1 -- 172.21.15.204:0/989380508 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4007ac0 con 0x7f5aa8103100 2024-04-02T14:40:01.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5aaf28a700 1 -- 172.21.15.204:0/989380508 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa81a0880 con 0x7f5aa8103100 2024-04-02T14:40:01.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5a9e7fc700 1 -- 172.21.15.204:0/989380508 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5aa4007c60 con 0x7f5aa8103100 2024-04-02T14:40:01.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5a9e7fc700 1 -- 172.21.15.204:0/989380508 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4018760 con 0x7f5aa8103100 2024-04-02T14:40:01.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.726+0000 7f5aaf28a700 1 -- 172.21.15.204:0/989380508 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa81a0e10 con 0x7f5aa8103100 2024-04-02T14:40:01.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.727+0000 7f5aaf28a700 1 -- 172.21.15.204:0/989380508 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa80692b0 con 0x7f5aa8103100 2024-04-02T14:40:01.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.728+0000 7f5a9e7fc700 1 -- 172.21.15.204:0/989380508 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5aa400f450 con 0x7f5aa8103100 2024-04-02T14:40:01.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.729+0000 7f5a9e7fc700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5a940a3c40 0x7f5a940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:01.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.729+0000 7f5a9e7fc700 1 -- 172.21.15.204:0/989380508 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6723+0+0 (secure 0 0 0) 0x7f5aa40cfff0 con 0x7f5aa8103100 2024-04-02T14:40:01.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.729+0000 7f5aac825700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5a940a3c40 0x7f5a940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:01.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.730+0000 7f5aac825700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5a940a3c40 0x7f5a940a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5a98003ad0 tx=0x7f5a98005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:01.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.733+0000 7f5a9e7fc700 1 -- 172.21.15.204:0/989380508 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5aa4014030 con 0x7f5aa8103100 2024-04-02T14:40:01.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T14:40:01.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi113 2024-04-02T14:40:01.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi113 to 4270M 2024-04-02T14:40:01.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:01.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:01.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:01.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:01.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:01.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:01 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T14:40:01.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi113 2024-04-02T14:40:01.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi113 to 4270M 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:01.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:01 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.872 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi113 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi113 to 4270M 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:01.873 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.874 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:01.874 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:01.874 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:01.874 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:01 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:01.943 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:01.942+0000 7f5aaf28a700 1 -- 172.21.15.204:0/989380508 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5aa8062a30 con 0x7f5a940a3c40 2024-04-02T14:40:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:02 smithi204 ceph-mon[35948]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:02 smithi204 ceph-mon[35948]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T14:40:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:02 smithi204 ceph-mon[35948]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:40:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:40:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:40:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:02.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:02 smithi062 ceph-mon[32116]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:02.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:02 smithi062 ceph-mon[32116]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T14:40:02.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:02 smithi062 ceph-mon[32116]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:40:02.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:40:02.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:40:02.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:02.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:02 smithi113 ceph-mon[36135]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:02.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:02 smithi113 ceph-mon[36135]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T14:40:02.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:02 smithi113 ceph-mon[36135]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:40:02.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T14:40:02.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T14:40:02.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:03.799 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:03 smithi204 ceph-mon[35948]: osdmap e78: 11 total, 11 up, 11 in 2024-04-02T14:40:03.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:03 smithi062 ceph-mon[32116]: osdmap e78: 11 total, 11 up, 11 in 2024-04-02T14:40:03.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:03 smithi113 ceph-mon[36135]: osdmap e78: 11 total, 11 up, 11 in 2024-04-02T14:40:04.815 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:04 smithi204 ceph-mon[35948]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:04.815 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:04 smithi204 ceph-mon[35948]: osdmap e79: 11 total, 11 up, 11 in 2024-04-02T14:40:04.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:04 smithi062 ceph-mon[32116]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:04.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:04 smithi062 ceph-mon[32116]: osdmap e79: 11 total, 11 up, 11 in 2024-04-02T14:40:04.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:04 smithi113 ceph-mon[36135]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:04.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:04 smithi113 ceph-mon[36135]: osdmap e79: 11 total, 11 up, 11 in 2024-04-02T14:40:05.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:05 smithi062 ceph-mon[32116]: from='client.? 172.21.15.204:0/2428557327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6f10568-eebc-4be6-92eb-942cc4afa1f6"}]: dispatch 2024-04-02T14:40:05.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:05 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6f10568-eebc-4be6-92eb-942cc4afa1f6"}]: dispatch 2024-04-02T14:40:05.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:05 smithi062 ceph-mon[32116]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6f10568-eebc-4be6-92eb-942cc4afa1f6"}]': finished 2024-04-02T14:40:05.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:05 smithi062 ceph-mon[32116]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T14:40:05.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:05 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:05.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:05 smithi062 ceph-mon[32116]: from='client.? 172.21.15.204:0/2231206275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:40:05.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:05 smithi113 ceph-mon[36135]: from='client.? 172.21.15.204:0/2428557327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6f10568-eebc-4be6-92eb-942cc4afa1f6"}]: dispatch 2024-04-02T14:40:05.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:05 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6f10568-eebc-4be6-92eb-942cc4afa1f6"}]: dispatch 2024-04-02T14:40:05.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:05 smithi113 ceph-mon[36135]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6f10568-eebc-4be6-92eb-942cc4afa1f6"}]': finished 2024-04-02T14:40:05.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:05 smithi113 ceph-mon[36135]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T14:40:05.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:05 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:05.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:05 smithi113 ceph-mon[36135]: from='client.? 172.21.15.204:0/2231206275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:40:06.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:05 smithi204 ceph-mon[35948]: from='client.? 172.21.15.204:0/2428557327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6f10568-eebc-4be6-92eb-942cc4afa1f6"}]: dispatch 2024-04-02T14:40:06.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:05 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6f10568-eebc-4be6-92eb-942cc4afa1f6"}]: dispatch 2024-04-02T14:40:06.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:05 smithi204 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6f10568-eebc-4be6-92eb-942cc4afa1f6"}]': finished 2024-04-02T14:40:06.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:05 smithi204 ceph-mon[35948]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T14:40:06.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:05 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:06.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:05 smithi204 ceph-mon[35948]: from='client.? 172.21.15.204:0/2231206275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T14:40:07.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:06 smithi204 ceph-mon[35948]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T14:40:07.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:06 smithi062 ceph-mon[32116]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T14:40:07.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:06 smithi113 ceph-mon[36135]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T14:40:08.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:08 smithi204 ceph-mon[35948]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-02T14:40:09.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:08 smithi062 ceph-mon[32116]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-02T14:40:09.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:08 smithi113 ceph-mon[36135]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-02T14:40:10.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:10 smithi204 ceph-mon[35948]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-02T14:40:11.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:10 smithi062 ceph-mon[32116]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-02T14:40:11.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:10 smithi113 ceph-mon[36135]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-02T14:40:12.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:12 smithi204 ceph-mon[35948]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T14:40:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T14:40:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:13.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:12 smithi062 ceph-mon[32116]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T14:40:13.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T14:40:13.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:13.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:12 smithi113 ceph-mon[36135]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T14:40:13.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T14:40:13.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:14.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:13 smithi204 ceph-mon[35948]: Deploying daemon osd.11 on smithi204 2024-04-02T14:40:14.022 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:13 smithi113 ceph-mon[36135]: Deploying daemon osd.11 on smithi204 2024-04-02T14:40:14.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:13 smithi062 ceph-mon[32116]: Deploying daemon osd.11 on smithi204 2024-04-02T14:40:15.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:14 smithi204 ceph-mon[35948]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-02T14:40:15.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:14 smithi062 ceph-mon[32116]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-02T14:40:15.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:14 smithi113 ceph-mon[36135]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-02T14:40:16.856 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:16 smithi204 ceph-mon[35948]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-02T14:40:17.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:16 smithi062 ceph-mon[32116]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-02T14:40:17.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:16 smithi113 ceph-mon[36135]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-02T14:40:19.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:18 smithi204 ceph-mon[35948]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:19.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:18 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:19.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:18 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:19.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:18 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:19.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:18 smithi062 ceph-mon[32116]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:19.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:18 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:19.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:18 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:19.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:18 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:19.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:18 smithi113 ceph-mon[36135]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:19.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:18 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:19.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:18 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:19.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:18 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:21.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:20 smithi204 ceph-mon[35948]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:21.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:20 smithi062 ceph-mon[32116]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:21.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:20 smithi113 ceph-mon[36135]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:21.238 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.237+0000 7f5a9e7fc700 1 -- 172.21.15.204:0/989380508 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5aa8062a30 con 0x7f5a940a3c40 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 -- 172.21.15.204:0/989380508 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5a940a3c40 msgr2=0x7f5a940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5a940a3c40 0x7f5a940a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5a98003ad0 tx=0x7f5a98005fd0 comp rx=0 tx=0).stop 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 -- 172.21.15.204:0/989380508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8103100 msgr2=0x7f5aa8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8103100 0x7f5aa8198930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c9e0 tx=0x7f5aa400cda0 comp rx=0 tx=0).stop 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 -- 172.21.15.204:0/989380508 shutdown_connections 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5a940a3c40 0x7f5a940a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa8108360 0x7f5aa819fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8103100 0x7f5aa8198930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 --2- 172.21.15.204:0/989380508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aa8105a30 0x7f5aa8198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 -- 172.21.15.204:0/989380508 >> 172.21.15.204:0/989380508 conn(0x7f5aa80fcb10 msgr2=0x7f5aa8103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:21.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 -- 172.21.15.204:0/989380508 shutdown_connections 2024-04-02T14:40:21.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:21.239+0000 7f5aaf28a700 1 -- 172.21.15.204:0/989380508 wait complete. 2024-04-02T14:40:21.876 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.11.service 2024-04-02T14:40:21.879 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-02T14:40:21.879 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-04-02T14:40:22.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:22.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:22.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:22 smithi062 ceph-mon[32116]: from='osd.11 [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T14:40:22.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:22 smithi062 ceph-mon[32116]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T14:40:22.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:22.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:22.354 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:40:22.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:22.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:22.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:22 smithi204 ceph-mon[35948]: from='osd.11 [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T14:40:22.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:22 smithi204 ceph-mon[35948]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T14:40:22.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:22.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:22.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:22.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:22.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:22 smithi113 ceph-mon[36135]: from='osd.11 [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T14:40:22.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:22 smithi113 ceph-mon[36135]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T14:40:22.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:22.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:23.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:23 smithi204 ceph-mon[35948]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:23.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:23 smithi204 ceph-mon[35948]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T14:40:23.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:23 smithi204 ceph-mon[35948]: from='osd.11 [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:40:23.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:23 smithi204 ceph-mon[35948]: osdmap e81: 12 total, 11 up, 12 in 2024-04-02T14:40:23.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:23 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:23.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:23 smithi204 ceph-mon[35948]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:40:23.365 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:40:23 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:40:23.205+0000 7f41182de700 -1 osd.11 0 waiting for initial osdmap 2024-04-02T14:40:23.365 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:40:23 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:40:23.221+0000 7f41140d1700 -1 osd.11 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T14:40:23.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:23 smithi062 ceph-mon[32116]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:23.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:23 smithi062 ceph-mon[32116]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T14:40:23.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:23 smithi062 ceph-mon[32116]: from='osd.11 [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:40:23.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:23 smithi062 ceph-mon[32116]: osdmap e81: 12 total, 11 up, 12 in 2024-04-02T14:40:23.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:23 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:23.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:23 smithi062 ceph-mon[32116]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:40:23.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:23 smithi113 ceph-mon[36135]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:23.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:23 smithi113 ceph-mon[36135]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T14:40:23.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:23 smithi113 ceph-mon[36135]: from='osd.11 [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:40:23.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:23 smithi113 ceph-mon[36135]: osdmap e81: 12 total, 11 up, 12 in 2024-04-02T14:40:23.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:23 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:23.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:23 smithi113 ceph-mon[36135]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-02T14:40:24.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:24 smithi062 ceph-mon[32116]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:40:24.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:24 smithi062 ceph-mon[32116]: osdmap e82: 12 total, 11 up, 12 in 2024-04-02T14:40:24.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:24 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:24.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:24 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:24.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:24 smithi204 ceph-mon[35948]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:40:24.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:24 smithi204 ceph-mon[35948]: osdmap e82: 12 total, 11 up, 12 in 2024-04-02T14:40:24.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:24 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:24.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:24 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:24.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:24 smithi113 ceph-mon[36135]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-02T14:40:24.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:24 smithi113 ceph-mon[36135]: osdmap e82: 12 total, 11 up, 12 in 2024-04-02T14:40:24.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:24 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:24.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:24 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.694+0000 7ff116600700 1 -- 172.21.15.62:0/2577038310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff110100ea0 msgr2=0x7ff1101032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.694+0000 7ff116600700 1 --2- 172.21.15.62:0/2577038310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff110100ea0 0x7ff1101032d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff100009990 tx=0x7ff100009ca0 comp rx=0 tx=0).stop 2024-04-02T14:40:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.695+0000 7ff116600700 1 -- 172.21.15.62:0/2577038310 shutdown_connections 2024-04-02T14:40:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.695+0000 7ff116600700 1 --2- 172.21.15.62:0/2577038310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1101061e0 0x7ff110106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.695+0000 7ff116600700 1 --2- 172.21.15.62:0/2577038310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff110100ea0 0x7ff1101032d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.695+0000 7ff116600700 1 --2- 172.21.15.62:0/2577038310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff110103810 0x7ff110105ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.695+0000 7ff116600700 1 -- 172.21.15.62:0/2577038310 >> 172.21.15.62:0/2577038310 conn(0x7ff1100fcac0 msgr2=0x7ff1100fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.695+0000 7ff116600700 1 -- 172.21.15.62:0/2577038310 shutdown_connections 2024-04-02T14:40:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.695+0000 7ff116600700 1 -- 172.21.15.62:0/2577038310 wait complete. 2024-04-02T14:40:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.696+0000 7ff116600700 1 Processor -- start 2024-04-02T14:40:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.696+0000 7ff116600700 1 -- start start 2024-04-02T14:40:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.697+0000 7ff116600700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff110100ea0 0x7ff11019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.697+0000 7ff116600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff110103810 0x7ff11019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.697+0000 7ff116600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1101061e0 0x7ff1101a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.697+0000 7ff116600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff110114910 con 0x7ff1101061e0 2024-04-02T14:40:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.697+0000 7ff116600700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff110114790 con 0x7ff110100ea0 2024-04-02T14:40:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.697+0000 7ff116600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff110114a90 con 0x7ff110103810 2024-04-02T14:40:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.697+0000 7ff10f400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff110100ea0 0x7ff11019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.697+0000 7ff10f400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff110100ea0 0x7ff11019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:39628/0 (socket says 172.21.15.62:39628) 2024-04-02T14:40:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.697+0000 7ff10f400700 1 -- 172.21.15.62:0/2153629728 learned_addr learned my addr 172.21.15.62:0/2153629728 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.697+0000 7ff106a00700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff110103810 0x7ff11019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.697+0000 7ff10fe00700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1101061e0 0x7ff1101a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.698+0000 7ff10f400700 1 -- 172.21.15.62:0/2153629728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff110103810 msgr2=0x7ff11019f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.698+0000 7ff10f400700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff110103810 0x7ff11019f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.698+0000 7ff10f400700 1 -- 172.21.15.62:0/2153629728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1101061e0 msgr2=0x7ff1101a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.698+0000 7ff10f400700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1101061e0 0x7ff1101a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.698+0000 7ff10f400700 1 -- 172.21.15.62:0/2153629728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1101a6b50 con 0x7ff110100ea0 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.698+0000 7ff106a00700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff110103810 0x7ff11019f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.698+0000 7ff10f400700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff110100ea0 0x7ff11019eef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff100005910 tx=0x7ff10000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.698+0000 7ff10cc00700 1 -- 172.21.15.62:0/2153629728 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100026070 con 0x7ff110100ea0 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.698+0000 7ff116600700 1 -- 172.21.15.62:0/2153629728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1101a6de0 con 0x7ff110100ea0 2024-04-02T14:40:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.698+0000 7ff10cc00700 1 -- 172.21.15.62:0/2153629728 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff10000fde0 con 0x7ff110100ea0 2024-04-02T14:40:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.698+0000 7ff10cc00700 1 -- 172.21.15.62:0/2153629728 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100017650 con 0x7ff110100ea0 2024-04-02T14:40:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.699+0000 7ff116600700 1 -- 172.21.15.62:0/2153629728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1101a7240 con 0x7ff110100ea0 2024-04-02T14:40:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.700+0000 7ff116600700 1 -- 172.21.15.62:0/2153629728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1100692b0 con 0x7ff110100ea0 2024-04-02T14:40:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.702+0000 7ff10cc00700 1 -- 172.21.15.62:0/2153629728 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff100029070 con 0x7ff110100ea0 2024-04-02T14:40:24.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.703+0000 7ff10cc00700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0f40a3c40 0x7ff0f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.704+0000 7ff10cc00700 1 -- 172.21.15.62:0/2153629728 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ff1000d0d90 con 0x7ff110100ea0 2024-04-02T14:40:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.704+0000 7ff106a00700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0f40a3c40 0x7ff0f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.705+0000 7ff106a00700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0f40a3c40 0x7ff0f40a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009680 tx=0x7ff0f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.712+0000 7ff10cc00700 1 -- 172.21.15.62:0/2153629728 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff10005df20 con 0x7ff110100ea0 2024-04-02T14:40:24.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.951+0000 7ff116600700 1 -- 172.21.15.62:0/2153629728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff110063200 con 0x7ff110100ea0 2024-04-02T14:40:24.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.952+0000 7ff10cc00700 1 -- 172.21.15.62:0/2153629728 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7ff100013070 con 0x7ff110100ea0 2024-04-02T14:40:24.952 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:24.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.955+0000 7ff116600700 1 -- 172.21.15.62:0/2153629728 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0f40a3c40 msgr2=0x7ff0f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:24.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.955+0000 7ff116600700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0f40a3c40 0x7ff0f40a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009680 tx=0x7ff0f8009260 comp rx=0 tx=0).stop 2024-04-02T14:40:24.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.955+0000 7ff116600700 1 -- 172.21.15.62:0/2153629728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff110100ea0 msgr2=0x7ff11019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:24.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.955+0000 7ff116600700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff110100ea0 0x7ff11019eef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff100005910 tx=0x7ff10000f690 comp rx=0 tx=0).stop 2024-04-02T14:40:24.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.955+0000 7ff116600700 1 -- 172.21.15.62:0/2153629728 shutdown_connections 2024-04-02T14:40:24.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.955+0000 7ff116600700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0f40a3c40 0x7ff0f40a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:24.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.955+0000 7ff116600700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff110100ea0 0x7ff11019eef0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:24.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.955+0000 7ff116600700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1101061e0 0x7ff1101a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:24.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.955+0000 7ff116600700 1 --2- 172.21.15.62:0/2153629728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff110103810 0x7ff11019f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:24.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.955+0000 7ff116600700 1 -- 172.21.15.62:0/2153629728 >> 172.21.15.62:0/2153629728 conn(0x7ff1100fcac0 msgr2=0x7ff110104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:24.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.956+0000 7ff116600700 1 -- 172.21.15.62:0/2153629728 shutdown_connections 2024-04-02T14:40:24.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:24.956+0000 7ff116600700 1 -- 172.21.15.62:0/2153629728 wait complete. 2024-04-02T14:40:25.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:25 smithi204 ceph-mon[35948]: purged_snaps scrub starts 2024-04-02T14:40:25.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:25 smithi204 ceph-mon[35948]: purged_snaps scrub ok 2024-04-02T14:40:25.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:25 smithi204 ceph-mon[35948]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:25.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:25 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:25.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:25 smithi204 ceph-mon[35948]: osd.11 [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038] boot 2024-04-02T14:40:25.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:25 smithi204 ceph-mon[35948]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T14:40:25.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:25 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:25.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:25 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2153629728' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T14:40:25.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:25 smithi062 ceph-mon[32116]: purged_snaps scrub starts 2024-04-02T14:40:25.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:25 smithi062 ceph-mon[32116]: purged_snaps scrub ok 2024-04-02T14:40:25.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:25 smithi062 ceph-mon[32116]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:25.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:25 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:25.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:25 smithi062 ceph-mon[32116]: osd.11 [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038] boot 2024-04-02T14:40:25.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:25 smithi062 ceph-mon[32116]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T14:40:25.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:25 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:25.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:25 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2153629728' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T14:40:25.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:25 smithi113 ceph-mon[36135]: purged_snaps scrub starts 2024-04-02T14:40:25.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:25 smithi113 ceph-mon[36135]: purged_snaps scrub ok 2024-04-02T14:40:25.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:25 smithi113 ceph-mon[36135]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T14:40:25.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:25 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:25.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:25 smithi113 ceph-mon[36135]: osd.11 [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038] boot 2024-04-02T14:40:25.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:25 smithi113 ceph-mon[36135]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T14:40:25.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:25 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:40:25.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:25 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2153629728' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T14:40:25.800 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1712068824,"num_in_osds":12,"osd_in_since":1712068804,"num_remapped_pgs":0} 2024-04-02T14:40:25.801 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T14:40:26.271 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:40:26.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T14:40:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: Detected new or changed devices on smithi204 2024-04-02T14:40:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi204 to 4270M 2024-04-02T14:40:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:26.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:26.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:26.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:26.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:26.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:26 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T14:40:26.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: Detected new or changed devices on smithi204 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi204 to 4270M 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:26.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:26.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:26.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:26 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T14:40:26.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:26.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: Detected new or changed devices on smithi204 2024-04-02T14:40:26.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:40:26.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi204 to 4270M 2024-04-02T14:40:26.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:26.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:26.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:26.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:26.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:26.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:26.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:26 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:28.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.474+0000 7f8add000700 1 -- 172.21.15.62:0/3860568525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad810cac0 msgr2=0x7f8ad810ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:28.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.474+0000 7f8add000700 1 --2- 172.21.15.62:0/3860568525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad810cac0 0x7f8ad810ef80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000b0a0 tx=0x7f8ad000b3b0 comp rx=0 tx=0).stop 2024-04-02T14:40:28.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.474+0000 7f8add000700 1 -- 172.21.15.62:0/3860568525 shutdown_connections 2024-04-02T14:40:28.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.474+0000 7f8add000700 1 --2- 172.21.15.62:0/3860568525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad810cac0 0x7f8ad810ef80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:28.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.474+0000 7f8add000700 1 --2- 172.21.15.62:0/3860568525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ad8103690 0x7f8ad8103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:28.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.474+0000 7f8add000700 1 --2- 172.21.15.62:0/3860568525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad8103fb0 0x7f8ad810c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:28.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.474+0000 7f8add000700 1 -- 172.21.15.62:0/3860568525 >> 172.21.15.62:0/3860568525 conn(0x7f8ad80fcec0 msgr2=0x7f8ad80ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:28.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.474+0000 7f8add000700 1 -- 172.21.15.62:0/3860568525 shutdown_connections 2024-04-02T14:40:28.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.474+0000 7f8add000700 1 -- 172.21.15.62:0/3860568525 wait complete. 2024-04-02T14:40:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.476+0000 7f8add000700 1 Processor -- start 2024-04-02T14:40:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.476+0000 7f8add000700 1 -- start start 2024-04-02T14:40:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.476+0000 7f8add000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad8103690 0x7f8ad819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.476+0000 7f8add000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad8103fb0 0x7f8ad819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.476+0000 7f8add000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ad810cac0 0x7f8ad8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.476+0000 7f8add000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8112940 con 0x7f8ad8103690 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.476+0000 7f8add000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81127c0 con 0x7f8ad8103fb0 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.476+0000 7f8add000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8112ac0 con 0x7f8ad810cac0 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad7400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad8103690 0x7f8ad819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad6a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad8103fb0 0x7f8ad819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad6a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad8103fb0 0x7f8ad819ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:39646/0 (socket says 172.21.15.62:39646) 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad6a00700 1 -- 172.21.15.62:0/4030763767 learned_addr learned my addr 172.21.15.62:0/4030763767 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad7400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad8103690 0x7f8ad819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33364/0 (socket says 172.21.15.62:33364) 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad7e00700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ad810cac0 0x7f8ad8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad7400700 1 -- 172.21.15.62:0/4030763767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ad810cac0 msgr2=0x7f8ad8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad7400700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ad810cac0 0x7f8ad8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad7400700 1 -- 172.21.15.62:0/4030763767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad8103fb0 msgr2=0x7f8ad819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad7400700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad8103fb0 0x7f8ad819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad7400700 1 -- 172.21.15.62:0/4030763767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad8199240 con 0x7f8ad8103690 2024-04-02T14:40:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.477+0000 7f8ad6a00700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad8103fb0 0x7f8ad819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:40:28.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.478+0000 7f8ad7400700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad8103690 0x7f8ad819e920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800d750 tx=0x7f8ac800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:28.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.478+0000 7f8acfe00700 1 -- 172.21.15.62:0/4030763767 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8014030 con 0x7f8ad8103690 2024-04-02T14:40:28.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.478+0000 7f8add000700 1 -- 172.21.15.62:0/4030763767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad8199530 con 0x7f8ad8103690 2024-04-02T14:40:28.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.478+0000 7f8acfe00700 1 -- 172.21.15.62:0/4030763767 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8ac8004520 con 0x7f8ad8103690 2024-04-02T14:40:28.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.478+0000 7f8acfe00700 1 -- 172.21.15.62:0/4030763767 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8005490 con 0x7f8ad8103690 2024-04-02T14:40:28.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.478+0000 7f8add000700 1 -- 172.21.15.62:0/4030763767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad81a57f0 con 0x7f8ad8103690 2024-04-02T14:40:28.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.479+0000 7f8add000700 1 -- 172.21.15.62:0/4030763767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad80692b0 con 0x7f8ad8103690 2024-04-02T14:40:28.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.481+0000 7f8acfe00700 1 -- 172.21.15.62:0/4030763767 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8ac8004b50 con 0x7f8ad8103690 2024-04-02T14:40:28.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.483+0000 7f8acfe00700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8abc0a3c40 0x7f8abc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:28.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.483+0000 7f8acfe00700 1 -- 172.21.15.62:0/4030763767 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f8ac80cf850 con 0x7f8ad8103690 2024-04-02T14:40:28.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.483+0000 7f8ad6a00700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8abc0a3c40 0x7f8abc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:28.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.484+0000 7f8ad6a00700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8abc0a3c40 0x7f8abc0a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8ad810de00 tx=0x7f8ac4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:28.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.487+0000 7f8acfe00700 1 -- 172.21.15.62:0/4030763767 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8ac809c630 con 0x7f8ad8103690 2024-04-02T14:40:28.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:28 smithi062 ceph-mon[32116]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:28.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.722+0000 7f8add000700 1 -- 172.21.15.62:0/4030763767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ad81a5a20 con 0x7f8ad8103690 2024-04-02T14:40:28.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.725+0000 7f8acfe00700 1 -- 172.21.15.62:0/4030763767 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18434 (secure 0 0 0) 0x7f8ac809bf30 con 0x7f8ad8103690 2024-04-02T14:40:28.726 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:28.726 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":84,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:40:25.221043+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:40:28.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.728+0000 7f8add000700 1 -- 172.21.15.62:0/4030763767 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8abc0a3c40 msgr2=0x7f8abc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:28.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.728+0000 7f8add000700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8abc0a3c40 0x7f8abc0a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8ad810de00 tx=0x7f8ac4009260 comp rx=0 tx=0).stop 2024-04-02T14:40:28.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.728+0000 7f8add000700 1 -- 172.21.15.62:0/4030763767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad8103690 msgr2=0x7f8ad819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:28.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.728+0000 7f8add000700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad8103690 0x7f8ad819e920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800d750 tx=0x7f8ac800da60 comp rx=0 tx=0).stop 2024-04-02T14:40:28.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.728+0000 7f8add000700 1 -- 172.21.15.62:0/4030763767 shutdown_connections 2024-04-02T14:40:28.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.728+0000 7f8add000700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8abc0a3c40 0x7f8abc0a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:28.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.728+0000 7f8add000700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ad810cac0 0x7f8ad8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:28.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.728+0000 7f8add000700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad8103690 0x7f8ad819e920 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:28.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.728+0000 7f8add000700 1 --2- 172.21.15.62:0/4030763767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad8103fb0 0x7f8ad819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:28.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.728+0000 7f8add000700 1 -- 172.21.15.62:0/4030763767 >> 172.21.15.62:0/4030763767 conn(0x7f8ad80fcec0 msgr2=0x7f8ad80fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:28.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.729+0000 7f8add000700 1 -- 172.21.15.62:0/4030763767 shutdown_connections 2024-04-02T14:40:28.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:28.729+0000 7f8add000700 1 -- 172.21.15.62:0/4030763767 wait complete. 2024-04-02T14:40:28.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:28 smithi204 ceph-mon[35948]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:28.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:28 smithi113 ceph-mon[36135]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:29.533 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:40:29.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-04-02T14:40:29.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:29 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/4030763767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:40:29.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:29 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/4030763767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:40:29.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:29 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/4030763767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:40:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:40:30.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:30 smithi062 ceph-mon[32116]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:30.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:30 smithi204 ceph-mon[35948]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:30.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:30 smithi113 ceph-mon[36135]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:32.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.218+0000 7fe64b400700 1 -- 172.21.15.62:0/1264799601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe64406ab30 msgr2=0x7fe64406af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:32.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.218+0000 7fe64b400700 1 --2- 172.21.15.62:0/1264799601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe64406ab30 0x7fe64406af10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe638009990 tx=0x7fe638009ca0 comp rx=0 tx=0).stop 2024-04-02T14:40:32.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.218+0000 7fe64b400700 1 -- 172.21.15.62:0/1264799601 shutdown_connections 2024-04-02T14:40:32.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.218+0000 7fe64b400700 1 --2- 172.21.15.62:0/1264799601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe64410f0d0 0x7fe6441114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:32.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.218+0000 7fe64b400700 1 --2- 172.21.15.62:0/1264799601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe64406ab30 0x7fe64406af10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:32.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.218+0000 7fe64b400700 1 --2- 172.21.15.62:0/1264799601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe64406b450 0x7fe64410eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:32.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.218+0000 7fe64b400700 1 -- 172.21.15.62:0/1264799601 >> 172.21.15.62:0/1264799601 conn(0x7fe644077540 msgr2=0x7fe644077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:32.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.219+0000 7fe64b400700 1 -- 172.21.15.62:0/1264799601 shutdown_connections 2024-04-02T14:40:32.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.219+0000 7fe64b400700 1 -- 172.21.15.62:0/1264799601 wait complete. 2024-04-02T14:40:32.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.221+0000 7fe64b400700 1 Processor -- start 2024-04-02T14:40:32.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.221+0000 7fe64b400700 1 -- start start 2024-04-02T14:40:32.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.222+0000 7fe64b400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe64406ab30 0x7fe64419b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:32.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.222+0000 7fe64b400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe64406b450 0x7fe64419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.222+0000 7fe64b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe64410f0d0 0x7fe64419f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.222+0000 7fe64b400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe644114c90 con 0x7fe64410f0d0 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.222+0000 7fe64b400700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe644114b10 con 0x7fe64406b450 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.222+0000 7fe64b400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe644114e10 con 0x7fe64406ab30 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.222+0000 7fe648a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe64406ab30 0x7fe64419b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.222+0000 7fe648a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe64406ab30 0x7fe64419b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:51482/0 (socket says 172.21.15.62:51482) 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.222+0000 7fe648a00700 1 -- 172.21.15.62:0/2679475180 learned_addr learned my addr 172.21.15.62:0/2679475180 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.222+0000 7fe643e00700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe64406b450 0x7fe64419b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.223+0000 7fe649400700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe64410f0d0 0x7fe64419f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.223+0000 7fe648a00700 1 -- 172.21.15.62:0/2679475180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe64406b450 msgr2=0x7fe64419b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.223+0000 7fe648a00700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe64406b450 0x7fe64419b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.223+0000 7fe648a00700 1 -- 172.21.15.62:0/2679475180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe64410f0d0 msgr2=0x7fe64419f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.223+0000 7fe648a00700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe64410f0d0 0x7fe64419f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.223+0000 7fe648a00700 1 -- 172.21.15.62:0/2679475180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6441a00b0 con 0x7fe64406ab30 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.223+0000 7fe649400700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe64410f0d0 0x7fe64419f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.223+0000 7fe648a00700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe64406ab30 0x7fe64419b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe638000c00 tx=0x7fe638005590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.224+0000 7fe641600700 1 -- 172.21.15.62:0/2679475180 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638027070 con 0x7fe64406ab30 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.224+0000 7fe641600700 1 -- 172.21.15.62:0/2679475180 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe6380056b0 con 0x7fe64406ab30 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.224+0000 7fe64b400700 1 -- 172.21.15.62:0/2679475180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6441a0340 con 0x7fe64406ab30 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.224+0000 7fe641600700 1 -- 172.21.15.62:0/2679475180 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638017650 con 0x7fe64406ab30 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.224+0000 7fe64b400700 1 -- 172.21.15.62:0/2679475180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6441a7c20 con 0x7fe64406ab30 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.225+0000 7fe64b400700 1 -- 172.21.15.62:0/2679475180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe644101b00 con 0x7fe64406ab30 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.227+0000 7fe641600700 1 -- 172.21.15.62:0/2679475180 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe63802a070 con 0x7fe64406ab30 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.228+0000 7fe641600700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe62c0a3c40 0x7fe62c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.229+0000 7fe641600700 1 -- 172.21.15.62:0/2679475180 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe6380d1140 con 0x7fe64406ab30 2024-04-02T14:40:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.229+0000 7fe643e00700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe62c0a3c40 0x7fe62c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.229+0000 7fe643e00700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe62c0a3c40 0x7fe62c0a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe6441007f0 tx=0x7fe630009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.231+0000 7fe641600700 1 -- 172.21.15.62:0/2679475180 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe638031080 con 0x7fe64406ab30 2024-04-02T14:40:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.461+0000 7fe64b400700 1 -- 172.21.15.62:0/2679475180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe6441a7df0 con 0x7fe64406ab30 2024-04-02T14:40:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.462+0000 7fe641600700 1 -- 172.21.15.62:0/2679475180 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fe638013070 con 0x7fe64406ab30 2024-04-02T14:40:32.463 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:40:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.464+0000 7fe64b400700 1 -- 172.21.15.62:0/2679475180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe62c0a3c40 msgr2=0x7fe62c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.464+0000 7fe64b400700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe62c0a3c40 0x7fe62c0a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe6441007f0 tx=0x7fe630009260 comp rx=0 tx=0).stop 2024-04-02T14:40:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.465+0000 7fe64b400700 1 -- 172.21.15.62:0/2679475180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe64406ab30 msgr2=0x7fe64419b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.465+0000 7fe64b400700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe64406ab30 0x7fe64419b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe638000c00 tx=0x7fe638005590 comp rx=0 tx=0).stop 2024-04-02T14:40:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.465+0000 7fe64b400700 1 -- 172.21.15.62:0/2679475180 shutdown_connections 2024-04-02T14:40:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.465+0000 7fe64b400700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe62c0a3c40 0x7fe62c0a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.465+0000 7fe64b400700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe64410f0d0 0x7fe64419f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.465+0000 7fe64b400700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe64406ab30 0x7fe64419b0a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.465+0000 7fe64b400700 1 --2- 172.21.15.62:0/2679475180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe64406b450 0x7fe64419b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.465+0000 7fe64b400700 1 -- 172.21.15.62:0/2679475180 >> 172.21.15.62:0/2679475180 conn(0x7fe644077540 msgr2=0x7fe644110420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.465+0000 7fe64b400700 1 -- 172.21.15.62:0/2679475180 shutdown_connections 2024-04-02T14:40:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:32.465+0000 7fe64b400700 1 -- 172.21.15.62:0/2679475180 wait complete. 2024-04-02T14:40:32.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:32 smithi204 ceph-mon[35948]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:32.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:32 smithi062 ceph-mon[32116]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:32.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:32 smithi113 ceph-mon[36135]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:33.305 INFO:tasks.cephadm:Adding mds.a on smithi062 2024-04-02T14:40:33.306 INFO:tasks.cephadm:Adding mds.d on smithi062 2024-04-02T14:40:33.306 INFO:tasks.cephadm:Adding mds.g on smithi062 2024-04-02T14:40:33.306 INFO:tasks.cephadm:Adding mds.j on smithi062 2024-04-02T14:40:33.306 INFO:tasks.cephadm:Adding mds.b on smithi113 2024-04-02T14:40:33.306 INFO:tasks.cephadm:Adding mds.e on smithi113 2024-04-02T14:40:33.306 INFO:tasks.cephadm:Adding mds.h on smithi113 2024-04-02T14:40:33.306 INFO:tasks.cephadm:Adding mds.k on smithi113 2024-04-02T14:40:33.306 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-04-02T14:40:33.306 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-04-02T14:40:33.306 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-04-02T14:40:33.306 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-04-02T14:40:33.307 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph orch apply mds all '12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi113=b;smithi113=e;smithi113=h;smithi113=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-04-02T14:40:33.560 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:40:33.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:33 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2679475180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:40:33.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:33 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2679475180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:40:33.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:33 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2679475180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:40:34.675 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:34 smithi204 ceph-mon[35948]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:34.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:34 smithi062 ceph-mon[32116]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:34.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:34 smithi113 ceph-mon[36135]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:35.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.126+0000 7fb31b7ea700 1 -- 172.21.15.204:0/3122981801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb314102720 msgr2=0x7fb314102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:35.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.126+0000 7fb31b7ea700 1 --2- 172.21.15.204:0/3122981801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb314102720 0x7fb314102b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb310009990 tx=0x7fb310009ca0 comp rx=0 tx=0).stop 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.126+0000 7fb31b7ea700 1 -- 172.21.15.204:0/3122981801 shutdown_connections 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.126+0000 7fb31b7ea700 1 --2- 172.21.15.204:0/3122981801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb314101520 0x7fb314101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.126+0000 7fb31b7ea700 1 --2- 172.21.15.204:0/3122981801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3141030c0 0x7fb314109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.126+0000 7fb31b7ea700 1 --2- 172.21.15.204:0/3122981801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb314102720 0x7fb314102b80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.126+0000 7fb31b7ea700 1 -- 172.21.15.204:0/3122981801 >> 172.21.15.204:0/3122981801 conn(0x7fb3140fcae0 msgr2=0x7fb3140fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb31b7ea700 1 -- 172.21.15.204:0/3122981801 shutdown_connections 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb31b7ea700 1 -- 172.21.15.204:0/3122981801 wait complete. 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb31b7ea700 1 Processor -- start 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb31b7ea700 1 -- start start 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb31b7ea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb314101520 0x7fb314198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb31b7ea700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb314102720 0x7fb314198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb31b7ea700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3141030c0 0x7fb31419ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:35.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb31b7ea700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb31410d2c0 con 0x7fb314101520 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb31b7ea700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb31410d140 con 0x7fb314102720 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb31b7ea700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb31410d440 con 0x7fb3141030c0 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb319d87700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3141030c0 0x7fb31419ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb319d87700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3141030c0 0x7fb31419ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:52470/0 (socket says 172.21.15.204:52470) 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.127+0000 7fb319d87700 1 -- 172.21.15.204:0/1952035943 learned_addr learned my addr 172.21.15.204:0/1952035943 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb318d85700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb314102720 0x7fb314198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb319d87700 1 -- 172.21.15.204:0/1952035943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb314102720 msgr2=0x7fb314198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb319d87700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb314102720 0x7fb314198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb319d87700 1 -- 172.21.15.204:0/1952035943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb314101520 msgr2=0x7fb314198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb319d87700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb314101520 0x7fb314198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb319d87700 1 -- 172.21.15.204:0/1952035943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3141a05d0 con 0x7fb3141030c0 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb318d85700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb314102720 0x7fb314198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:40:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb319d87700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3141030c0 0x7fb31419ff30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00b9a0 tx=0x7fb30c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:35.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb30a7fc700 1 -- 172.21.15.204:0/1952035943 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c019070 con 0x7fb3141030c0 2024-04-02T14:40:35.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb31b7ea700 1 -- 172.21.15.204:0/1952035943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3141a08c0 con 0x7fb3141030c0 2024-04-02T14:40:35.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb30a7fc700 1 -- 172.21.15.204:0/1952035943 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb30c014440 con 0x7fb3141030c0 2024-04-02T14:40:35.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb30a7fc700 1 -- 172.21.15.204:0/1952035943 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0075d0 con 0x7fb3141030c0 2024-04-02T14:40:35.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.128+0000 7fb31b7ea700 1 -- 172.21.15.204:0/1952035943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3141a0e00 con 0x7fb3141030c0 2024-04-02T14:40:35.132 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.129+0000 7fb31b7ea700 1 -- 172.21.15.204:0/1952035943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3140692b0 con 0x7fb3141030c0 2024-04-02T14:40:35.132 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.130+0000 7fb30a7fc700 1 -- 172.21.15.204:0/1952035943 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb30c01c070 con 0x7fb3141030c0 2024-04-02T14:40:35.132 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.131+0000 7fb30a7fc700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb3000a3c40 0x7fb3000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:35.132 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.131+0000 7fb30a7fc700 1 -- 172.21.15.204:0/1952035943 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb30c0cf870 con 0x7fb3141030c0 2024-04-02T14:40:35.132 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.131+0000 7fb319586700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb3000a3c40 0x7fb3000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:35.133 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.132+0000 7fb319586700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb3000a3c40 0x7fb3000a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb314102580 tx=0x7fb304009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:35.135 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.134+0000 7fb30a7fc700 1 -- 172.21.15.204:0/1952035943 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb30c09c780 con 0x7fb3141030c0 2024-04-02T14:40:35.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.341+0000 7fb31b7ea700 1 -- 172.21.15.204:0/1952035943 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi113=b;smithi113=e;smithi113=h;smithi113=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) -- 0x7fb3140fedc0 con 0x7fb3000a3c40 2024-04-02T14:40:35.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.359+0000 7fb30a7fc700 1 -- 172.21.15.204:0/1952035943 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fb3140fedc0 con 0x7fb3000a3c40 2024-04-02T14:40:35.360 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 -- 172.21.15.204:0/1952035943 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb3000a3c40 msgr2=0x7fb3000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb3000a3c40 0x7fb3000a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb314102580 tx=0x7fb304009470 comp rx=0 tx=0).stop 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 -- 172.21.15.204:0/1952035943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3141030c0 msgr2=0x7fb31419ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3141030c0 0x7fb31419ff30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00b9a0 tx=0x7fb30c00bd60 comp rx=0 tx=0).stop 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 -- 172.21.15.204:0/1952035943 shutdown_connections 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb3000a3c40 0x7fb3000a6100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3141030c0 0x7fb31419ff30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb314101520 0x7fb314198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 --2- 172.21.15.204:0/1952035943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb314102720 0x7fb314198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 -- 172.21.15.204:0/1952035943 >> 172.21.15.204:0/1952035943 conn(0x7fb3140fcae0 msgr2=0x7fb3140fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 -- 172.21.15.204:0/1952035943 shutdown_connections 2024-04-02T14:40:35.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T14:40:35.361+0000 7fb31b7ea700 1 -- 172.21.15.204:0/1952035943 wait complete. 2024-04-02T14:40:36.014 DEBUG:teuthology.orchestra.run.smithi062:mds.a> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.a.service 2024-04-02T14:40:36.022 DEBUG:teuthology.orchestra.run.smithi062:mds.d> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.d.service 2024-04-02T14:40:36.069 DEBUG:teuthology.orchestra.run.smithi062:mds.g> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.g.service 2024-04-02T14:40:36.114 DEBUG:teuthology.orchestra.run.smithi062:mds.j> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.j.service 2024-04-02T14:40:36.160 DEBUG:teuthology.orchestra.run.smithi113:mds.b> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.b.service 2024-04-02T14:40:36.163 DEBUG:teuthology.orchestra.run.smithi113:mds.e> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.e.service 2024-04-02T14:40:36.207 DEBUG:teuthology.orchestra.run.smithi113:mds.h> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.h.service 2024-04-02T14:40:36.250 DEBUG:teuthology.orchestra.run.smithi113:mds.k> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.k.service 2024-04-02T14:40:36.293 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.c.service 2024-04-02T14:40:36.296 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.f.service 2024-04-02T14:40:36.339 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.i.service 2024-04-02T14:40:36.382 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.l.service 2024-04-02T14:40:36.425 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-02T14:40:36.426 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:40:36.514 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:36 smithi062 ceph-mon[32116]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:36.514 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:36 smithi062 ceph-mon[32116]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi113=b;smithi113=e;smithi113=h;smithi113=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:40:36.514 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:36 smithi062 ceph-mon[32116]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi113=b;smithi113=e;smithi113=h;smithi113=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-04-02T14:40:36.514 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:36 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:36.515 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:36 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:36.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:36 smithi113 ceph-mon[36135]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:36.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:36 smithi113 ceph-mon[36135]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi113=b;smithi113=e;smithi113=h;smithi113=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:40:36.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:36 smithi113 ceph-mon[36135]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi113=b;smithi113=e;smithi113=h;smithi113=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-04-02T14:40:36.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:36 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:36.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:36 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:36.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.672+0000 7f85abe00640 1 -- 172.21.15.62:0/2403433958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4104f30 msgr2=0x7f85a41053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:36.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.672+0000 7f85abe00640 1 --2- 172.21.15.62:0/2403433958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4104f30 0x7f85a41053b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8590009900 tx=0x7f859002bc50 comp rx=0 tx=0).stop 2024-04-02T14:40:36.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.672+0000 7f85abe00640 1 -- 172.21.15.62:0/2403433958 shutdown_connections 2024-04-02T14:40:36.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.672+0000 7f85abe00640 1 --2- 172.21.15.62:0/2403433958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a41058f0 0x7f85a410c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:36.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.672+0000 7f85abe00640 1 --2- 172.21.15.62:0/2403433958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4104f30 0x7f85a41053b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:36.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.672+0000 7f85abe00640 1 --2- 172.21.15.62:0/2403433958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a4103d40 0x7f85a4104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:36.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.672+0000 7f85abe00640 1 -- 172.21.15.62:0/2403433958 >> 172.21.15.62:0/2403433958 conn(0x7f85a40ff2b0 msgr2=0x7f85a41016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:36.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.672+0000 7f85abe00640 1 -- 172.21.15.62:0/2403433958 shutdown_connections 2024-04-02T14:40:36.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.672+0000 7f85abe00640 1 -- 172.21.15.62:0/2403433958 wait complete. 2024-04-02T14:40:36.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.673+0000 7f85abe00640 1 Processor -- start 2024-04-02T14:40:36.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.673+0000 7f85abe00640 1 -- start start 2024-04-02T14:40:36.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.673+0000 7f85abe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4103d40 0x7f85a419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:36.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.673+0000 7f85abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4104f30 0x7f85a419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:36.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85abe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a41058f0 0x7f85a41a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:36.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85abe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f85a410faf0 con 0x7f85a4104f30 2024-04-02T14:40:36.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85abe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f85a410f970 con 0x7f85a41058f0 2024-04-02T14:40:36.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85abe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f85a410fc70 con 0x7f85a4103d40 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85a9400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4103d40 0x7f85a419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85a9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a41058f0 0x7f85a41a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85a9400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4103d40 0x7f85a419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:37388/0 (socket says 172.21.15.62:37388) 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85a9400640 1 -- 172.21.15.62:0/4026519232 learned_addr learned my addr 172.21.15.62:0/4026519232 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85a8a00640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4104f30 0x7f85a419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85a9e00640 1 -- 172.21.15.62:0/4026519232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4103d40 msgr2=0x7f85a419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85a9e00640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4103d40 0x7f85a419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85a9e00640 1 -- 172.21.15.62:0/4026519232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4104f30 msgr2=0x7f85a419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85a9e00640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4104f30 0x7f85a419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85a9e00640 1 -- 172.21.15.62:0/4026519232 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a41a1d30 con 0x7f85a41058f0 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.674+0000 7f85a9400640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4103d40 0x7f85a419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.675+0000 7f85a9e00640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a41058f0 0x7f85a41a16c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f859c00e940 tx=0x7f859c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.675+0000 7f85a2000640 1 -- 172.21.15.62:0/4026519232 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c004d20 con 0x7f85a41058f0 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.675+0000 7f85a2000640 1 -- 172.21.15.62:0/4026519232 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f859c00cb20 con 0x7f85a41058f0 2024-04-02T14:40:36.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.675+0000 7f85abe00640 1 -- 172.21.15.62:0/4026519232 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a41a1fc0 con 0x7f85a41058f0 2024-04-02T14:40:36.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.675+0000 7f85a2000640 1 -- 172.21.15.62:0/4026519232 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0116b0 con 0x7f85a41058f0 2024-04-02T14:40:36.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.675+0000 7f85abe00640 1 -- 172.21.15.62:0/4026519232 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a41a24e0 con 0x7f85a41058f0 2024-04-02T14:40:36.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.676+0000 7f85abe00640 1 -- 172.21.15.62:0/4026519232 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a410b000 con 0x7f85a41058f0 2024-04-02T14:40:36.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.678+0000 7f85a2000640 1 -- 172.21.15.62:0/4026519232 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f859c005020 con 0x7f85a41058f0 2024-04-02T14:40:36.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.679+0000 7f85a2000640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3c30 0x7f858c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:36.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.680+0000 7f85a2000640 1 -- 172.21.15.62:0/4026519232 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f859c0cfff0 con 0x7f85a41058f0 2024-04-02T14:40:36.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.680+0000 7f85a9400640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3c30 0x7f858c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:36.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.680+0000 7f85a9400640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3c30 0x7f858c0a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f85980058c0 tx=0x7f8598005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.684+0000 7f85a2000640 1 -- 172.21.15.62:0/4026519232 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f859c016030 con 0x7f85a41058f0 2024-04-02T14:40:36.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:36 smithi204 ceph-mon[35948]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:36.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:36 smithi204 ceph-mon[35948]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi113=b;smithi113=e;smithi113=h;smithi113=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:40:36.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:36 smithi204 ceph-mon[35948]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi113=b;smithi113=e;smithi113=h;smithi113=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-04-02T14:40:36.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:36 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:36.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:36 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:40:36.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.895+0000 7f85abe00640 1 -- 172.21.15.62:0/4026519232 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85a4062700 con 0x7f85a41058f0 2024-04-02T14:40:36.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.896+0000 7f85a2000640 1 -- 172.21.15.62:0/4026519232 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18434 (secure 0 0 0) 0x7f859c09c780 con 0x7f85a41058f0 2024-04-02T14:40:36.896 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:36.897 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":84,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:40:25.221043+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 -- 172.21.15.62:0/4026519232 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3c30 msgr2=0x7f858c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3c30 0x7f858c0a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f85980058c0 tx=0x7f8598005f90 comp rx=0 tx=0).stop 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 -- 172.21.15.62:0/4026519232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a41058f0 msgr2=0x7f85a41a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a41058f0 0x7f85a41a16c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f859c00e940 tx=0x7f859c00ee00 comp rx=0 tx=0).stop 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 -- 172.21.15.62:0/4026519232 shutdown_connections 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3c30 0x7f858c0a60f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a41058f0 0x7f85a41a16c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4104f30 0x7f85a419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 --2- 172.21.15.62:0/4026519232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4103d40 0x7f85a419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 -- 172.21.15.62:0/4026519232 >> 172.21.15.62:0/4026519232 conn(0x7f85a40ff2b0 msgr2=0x7f85a41081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 -- 172.21.15.62:0/4026519232 shutdown_connections 2024-04-02T14:40:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:36.899+0000 7f85abe00640 1 -- 172.21.15.62:0/4026519232 wait complete. 2024-04-02T14:40:36.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:40:36.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:40:37.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.154+0000 7fd167200640 1 -- 172.21.15.62:0/1420218926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd168107f20 msgr2=0x7fd168108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:37.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.154+0000 7fd167200640 1 --2- 172.21.15.62:0/1420218926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd168107f20 0x7fd168108340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd158009900 tx=0x7fd15802bc50 comp rx=0 tx=0).stop 2024-04-02T14:40:37.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.154+0000 7fd167200640 1 -- 172.21.15.62:0/1420218926 shutdown_connections 2024-04-02T14:40:37.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.154+0000 7fd167200640 1 --2- 172.21.15.62:0/1420218926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd168109a30 0x7fd168110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.154+0000 7fd167200640 1 --2- 172.21.15.62:0/1420218926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168109070 0x7fd1681094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.154+0000 7fd167200640 1 --2- 172.21.15.62:0/1420218926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd168107f20 0x7fd168108340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.154+0000 7fd167200640 1 -- 172.21.15.62:0/1420218926 >> 172.21.15.62:0/1420218926 conn(0x7fd168076ec0 msgr2=0x7fd168079300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:37.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.154+0000 7fd167200640 1 -- 172.21.15.62:0/1420218926 shutdown_connections 2024-04-02T14:40:37.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.154+0000 7fd167200640 1 -- 172.21.15.62:0/1420218926 wait complete. 2024-04-02T14:40:37.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.155+0000 7fd167200640 1 Processor -- start 2024-04-02T14:40:37.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.155+0000 7fd167200640 1 -- start start 2024-04-02T14:40:37.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd167200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168107f20 0x7fd16819e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:37.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd167200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd168109070 0x7fd16819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd167200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd168109a30 0x7fd1681a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd167200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd168113cf0 con 0x7fd168107f20 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd167200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd168113b70 con 0x7fd168109070 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd167200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd168113e70 con 0x7fd168109a30 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd165e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168107f20 0x7fd16819e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd166800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd168109a30 0x7fd1681a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd166800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd168109a30 0x7fd1681a5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:37410/0 (socket says 172.21.15.62:37410) 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd15fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd168109070 0x7fd16819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd166800640 1 -- 172.21.15.62:0/3509164645 learned_addr learned my addr 172.21.15.62:0/3509164645 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd166800640 1 -- 172.21.15.62:0/3509164645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd168109070 msgr2=0x7fd16819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd166800640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd168109070 0x7fd16819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.156+0000 7fd166800640 1 -- 172.21.15.62:0/3509164645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168107f20 msgr2=0x7fd16819e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.157+0000 7fd166800640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168107f20 0x7fd16819e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.157+0000 7fd166800640 1 -- 172.21.15.62:0/3509164645 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1681a5ef0 con 0x7fd168109a30 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.157+0000 7fd15fe00640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd168109070 0x7fd16819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.157+0000 7fd165e00640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168107f20 0x7fd16819e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.157+0000 7fd166800640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd168109a30 0x7fd1681a5880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd16000eb10 tx=0x7fd16000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.157+0000 7fd15ea00640 1 -- 172.21.15.62:0/3509164645 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160004b20 con 0x7fd168109a30 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.157+0000 7fd15ea00640 1 -- 172.21.15.62:0/3509164645 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd160004cc0 con 0x7fd168109a30 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.157+0000 7fd167200640 1 -- 172.21.15.62:0/3509164645 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1681a61e0 con 0x7fd168109a30 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.157+0000 7fd15ea00640 1 -- 172.21.15.62:0/3509164645 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1600107f0 con 0x7fd168109a30 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.157+0000 7fd167200640 1 -- 172.21.15.62:0/3509164645 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681a6730 con 0x7fd168109a30 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.158+0000 7fd167200640 1 -- 172.21.15.62:0/3509164645 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd16810f1d0 con 0x7fd168109a30 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.160+0000 7fd15ea00640 1 -- 172.21.15.62:0/3509164645 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd160005020 con 0x7fd168109a30 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.161+0000 7fd15ea00640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd14c0a3bb0 0x7fd14c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.162+0000 7fd15ea00640 1 -- 172.21.15.62:0/3509164645 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fd1600d0530 con 0x7fd168109a30 2024-04-02T14:40:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.162+0000 7fd165e00640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd14c0a3bb0 0x7fd14c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:37.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.162+0000 7fd165e00640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd14c0a3bb0 0x7fd14c0a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd158000c00 tx=0x7fd158005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:37.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.166+0000 7fd15ea00640 1 -- 172.21.15.62:0/3509164645 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd16009d310 con 0x7fd168109a30 2024-04-02T14:40:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.374+0000 7fd167200640 1 -- 172.21.15.62:0/3509164645 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd168062700 con 0x7fd168109a30 2024-04-02T14:40:37.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.375+0000 7fd15ea00640 1 -- 172.21.15.62:0/3509164645 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fd16009cc10 con 0x7fd168109a30 2024-04-02T14:40:37.375 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:40:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.379+0000 7fd167200640 1 -- 172.21.15.62:0/3509164645 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd14c0a3bb0 msgr2=0x7fd14c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.379+0000 7fd167200640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd14c0a3bb0 0x7fd14c0a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd158000c00 tx=0x7fd158005fd0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.379+0000 7fd167200640 1 -- 172.21.15.62:0/3509164645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd168109a30 msgr2=0x7fd1681a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.379+0000 7fd167200640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd168109a30 0x7fd1681a5880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd16000eb10 tx=0x7fd16000c2d0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.380+0000 7fd167200640 1 -- 172.21.15.62:0/3509164645 shutdown_connections 2024-04-02T14:40:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.380+0000 7fd167200640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fd14c0a3bb0 0x7fd14c0a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.380+0000 7fd167200640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd168109a30 0x7fd1681a5880 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.380+0000 7fd167200640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd168109070 0x7fd16819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.380+0000 7fd167200640 1 --2- 172.21.15.62:0/3509164645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168107f20 0x7fd16819e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.380+0000 7fd167200640 1 -- 172.21.15.62:0/3509164645 >> 172.21.15.62:0/3509164645 conn(0x7fd168076ec0 msgr2=0x7fd16810c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.380+0000 7fd167200640 1 -- 172.21.15.62:0/3509164645 shutdown_connections 2024-04-02T14:40:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.380+0000 7fd167200640 1 -- 172.21.15.62:0/3509164645 wait complete. 2024-04-02T14:40:37.399 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:40:37.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:37.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:37.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:37.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:37.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:37.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:37 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:37.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:37 smithi204 ceph-mon[35948]: Deploying daemon mds.l on smithi204 2024-04-02T14:40:37.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:37 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/4026519232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:40:37.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:37.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:37.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:37.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:37.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:37.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:37 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:37.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:37 smithi113 ceph-mon[36135]: Deploying daemon mds.l on smithi204 2024-04-02T14:40:37.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:37 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/4026519232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:40:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.623+0000 7f9a2e200640 1 -- 172.21.15.62:0/49096452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a28109aa0 msgr2=0x7f9a281103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.623+0000 7f9a2e200640 1 --2- 172.21.15.62:0/49096452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a28109aa0 0x7f9a281103c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009ca0 tx=0x7f9a2002c100 comp rx=0 tx=0).stop 2024-04-02T14:40:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.623+0000 7f9a2e200640 1 -- 172.21.15.62:0/49096452 shutdown_connections 2024-04-02T14:40:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.623+0000 7f9a2e200640 1 --2- 172.21.15.62:0/49096452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a28109aa0 0x7f9a281103c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.623+0000 7f9a2e200640 1 --2- 172.21.15.62:0/49096452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a281090e0 0x7f9a28109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.623+0000 7f9a2e200640 1 --2- 172.21.15.62:0/49096452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a28107f10 0x7f9a28108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.623+0000 7f9a2e200640 1 -- 172.21.15.62:0/49096452 >> 172.21.15.62:0/49096452 conn(0x7f9a28076f10 msgr2=0x7f9a28079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.623+0000 7f9a2e200640 1 -- 172.21.15.62:0/49096452 shutdown_connections 2024-04-02T14:40:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.623+0000 7f9a2e200640 1 -- 172.21.15.62:0/49096452 wait complete. 2024-04-02T14:40:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.624+0000 7f9a2e200640 1 Processor -- start 2024-04-02T14:40:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.624+0000 7f9a2e200640 1 -- start start 2024-04-02T14:40:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.624+0000 7f9a2e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a28107f10 0x7f9a2819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.624+0000 7f9a2e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a281090e0 0x7f9a2819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.624+0000 7f9a2e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a28109aa0 0x7f9a281a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.624+0000 7f9a2e200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9a28113d10 con 0x7f9a28107f10 2024-04-02T14:40:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.624+0000 7f9a2e200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9a28113b90 con 0x7f9a28109aa0 2024-04-02T14:40:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.624+0000 7f9a2e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9a28113e90 con 0x7f9a281090e0 2024-04-02T14:40:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a27e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a281090e0 0x7f9a2819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a2d800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a28109aa0 0x7f9a281a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a27e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a281090e0 0x7f9a2819e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:37430/0 (socket says 172.21.15.62:37430) 2024-04-02T14:40:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a2d800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a28109aa0 0x7f9a281a5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41414/0 (socket says 172.21.15.62:41414) 2024-04-02T14:40:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a27e00640 1 -- 172.21.15.62:0/71392454 learned_addr learned my addr 172.21.15.62:0/71392454 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a2ce00640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a28107f10 0x7f9a2819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a27e00640 1 -- 172.21.15.62:0/71392454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a28109aa0 msgr2=0x7f9a281a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a27e00640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a28109aa0 0x7f9a281a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a27e00640 1 -- 172.21.15.62:0/71392454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a28107f10 msgr2=0x7f9a2819e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a27e00640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a28107f10 0x7f9a2819e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a27e00640 1 -- 172.21.15.62:0/71392454 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a281a5ee0 con 0x7f9a281090e0 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a2d800640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a28109aa0 0x7f9a281a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a2ce00640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a28107f10 0x7f9a2819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.625+0000 7f9a27e00640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a281090e0 0x7f9a2819e800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400e9f0 tx=0x7f9a1400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.626+0000 7f9a25600640 1 -- 172.21.15.62:0/71392454 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400cd80 con 0x7f9a281090e0 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.626+0000 7f9a25600640 1 -- 172.21.15.62:0/71392454 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a14010930 con 0x7f9a281090e0 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.626+0000 7f9a2e200640 1 -- 172.21.15.62:0/71392454 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a281a61d0 con 0x7f9a281090e0 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.626+0000 7f9a25600640 1 -- 172.21.15.62:0/71392454 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14018b00 con 0x7f9a281090e0 2024-04-02T14:40:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.626+0000 7f9a2e200640 1 -- 172.21.15.62:0/71392454 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a281a6720 con 0x7f9a281090e0 2024-04-02T14:40:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.627+0000 7f9a2e200640 1 -- 172.21.15.62:0/71392454 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a2810f2a0 con 0x7f9a281090e0 2024-04-02T14:40:37.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.629+0000 7f9a25600640 1 -- 172.21.15.62:0/71392454 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9a14018ca0 con 0x7f9a281090e0 2024-04-02T14:40:37.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.631+0000 7f9a25600640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f9a100a3bb0 0x7f9a100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:37.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.631+0000 7f9a25600640 1 -- 172.21.15.62:0/71392454 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f9a140d0420 con 0x7f9a281090e0 2024-04-02T14:40:37.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.631+0000 7f9a2ce00640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f9a100a3bb0 0x7f9a100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:37.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.632+0000 7f9a2ce00640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f9a100a3bb0 0x7f9a100a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9a28107c30 tx=0x7f9a1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:37.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.635+0000 7f9a25600640 1 -- 172.21.15.62:0/71392454 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9a1409d200 con 0x7f9a281090e0 2024-04-02T14:40:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:40:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:37 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:37 smithi062 ceph-mon[32116]: Deploying daemon mds.l on smithi204 2024-04-02T14:40:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:37 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/4026519232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:40:37.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.887+0000 7f9a2e200640 1 -- 172.21.15.62:0/71392454 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a28062700 con 0x7f9a281090e0 2024-04-02T14:40:37.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.888+0000 7f9a25600640 1 -- 172.21.15.62:0/71392454 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f9a1409cb00 con 0x7f9a281090e0 2024-04-02T14:40:37.888 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:37.888 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-02T14:40:37.888 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 1 2024-04-02T14:40:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.890+0000 7f9a2e200640 1 -- 172.21.15.62:0/71392454 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f9a100a3bb0 msgr2=0x7f9a100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.890+0000 7f9a2e200640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f9a100a3bb0 0x7f9a100a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9a28107c30 tx=0x7f9a1c005f90 comp rx=0 tx=0).stop 2024-04-02T14:40:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.890+0000 7f9a2e200640 1 -- 172.21.15.62:0/71392454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a281090e0 msgr2=0x7f9a2819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.890+0000 7f9a2e200640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a281090e0 0x7f9a2819e800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400e9f0 tx=0x7f9a1400eeb0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.891+0000 7f9a2e200640 1 -- 172.21.15.62:0/71392454 shutdown_connections 2024-04-02T14:40:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.891+0000 7f9a2e200640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f9a100a3bb0 0x7f9a100a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.891+0000 7f9a2e200640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a28109aa0 0x7f9a281a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.891+0000 7f9a2e200640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a281090e0 0x7f9a2819e800 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.891+0000 7f9a2e200640 1 --2- 172.21.15.62:0/71392454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a28107f10 0x7f9a2819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.891+0000 7f9a2e200640 1 -- 172.21.15.62:0/71392454 >> 172.21.15.62:0/71392454 conn(0x7f9a28076f10 msgr2=0x7f9a2810c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.891+0000 7f9a2e200640 1 -- 172.21.15.62:0/71392454 shutdown_connections 2024-04-02T14:40:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:37.891+0000 7f9a2e200640 1 -- 172.21.15.62:0/71392454 wait complete. 2024-04-02T14:40:38.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:38 smithi204 ceph-mon[35948]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:38.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:38 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3509164645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:40:38.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:38 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/71392454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:38.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:38 smithi113 ceph-mon[36135]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:38.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:38 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3509164645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:40:38.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:38 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/71392454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:38.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:38 smithi062 ceph-mon[32116]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:38.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:38 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3509164645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:40:38.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:38 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/71392454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:38.946 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:40:38 smithi204 systemd[1]: Starting Ceph mds.l for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:40:39.198 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:40:38 smithi204 podman[66681]: 2024-04-02 14:40:38.848829763 +0000 UTC m=+0.017561532 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:40:39.198 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:40:38 smithi204 podman[66681]: 2024-04-02 14:40:38.970353062 +0000 UTC m=+0.139084856 container create c1ddf547fb051b9324dcfcb5068d989dfead323ff0ae50acf3efc4af40e69ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-l, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-02T14:40:39.198 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:40:39 smithi204 podman[66681]: 2024-04-02 14:40:39.172855458 +0000 UTC m=+0.341587227 container init c1ddf547fb051b9324dcfcb5068d989dfead323ff0ae50acf3efc4af40e69ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-l, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-02T14:40:39.198 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:40:39 smithi204 podman[66681]: 2024-04-02 14:40:39.175468164 +0000 UTC m=+0.344199920 container start c1ddf547fb051b9324dcfcb5068d989dfead323ff0ae50acf3efc4af40e69ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-l, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-02T14:40:39.509 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:40:39 smithi204 ceph-mds[66700]: -- 172.21.15.204:0/4216814820 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563755079a40 con 0x56375504d000 2024-04-02T14:40:39.509 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:40:39 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-l[66696]: starting mds.l at 2024-04-02T14:40:39.509 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:40:39 smithi204 bash[66681]: c1ddf547fb051b9324dcfcb5068d989dfead323ff0ae50acf3efc4af40e69ae1 2024-04-02T14:40:39.509 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:40:39 smithi204 systemd[1]: Started Ceph mds.l for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:40:39.913 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:40:40.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.147+0000 7fe361a00640 1 -- 172.21.15.62:0/636825627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe35c1036d0 msgr2=0x7fe35c109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:40.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.147+0000 7fe361a00640 1 --2- 172.21.15.62:0/636825627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe35c1036d0 0x7fe35c109f60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe354009ca0 tx=0x7fe35402c0f0 comp rx=0 tx=0).stop 2024-04-02T14:40:40.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.147+0000 7fe361a00640 1 -- 172.21.15.62:0/636825627 shutdown_connections 2024-04-02T14:40:40.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.147+0000 7fe361a00640 1 --2- 172.21.15.62:0/636825627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe35c1036d0 0x7fe35c109f60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.147+0000 7fe361a00640 1 --2- 172.21.15.62:0/636825627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe35c102d10 0x7fe35c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.147+0000 7fe361a00640 1 --2- 172.21.15.62:0/636825627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c101b20 0x7fe35c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.147+0000 7fe361a00640 1 -- 172.21.15.62:0/636825627 >> 172.21.15.62:0/636825627 conn(0x7fe35c0fd0b0 msgr2=0x7fe35c0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.147+0000 7fe361a00640 1 -- 172.21.15.62:0/636825627 shutdown_connections 2024-04-02T14:40:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.147+0000 7fe361a00640 1 -- 172.21.15.62:0/636825627 wait complete. 2024-04-02T14:40:40.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.148+0000 7fe361a00640 1 Processor -- start 2024-04-02T14:40:40.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.148+0000 7fe361a00640 1 -- start start 2024-04-02T14:40:40.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe361a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c101b20 0x7fe35c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:40.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe361a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe35c102d10 0x7fe35c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe361a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe35c1036d0 0x7fe35c19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe361a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c10d8d0 con 0x7fe35c102d10 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe361a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c10d750 con 0x7fe35c1036d0 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe361a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c10da50 con 0x7fe35c101b20 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe353e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe35c102d10 0x7fe35c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe35b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe35c1036d0 0x7fe35c19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe353e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe35c102d10 0x7fe35c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41504/0 (socket says 172.21.15.62:41504) 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe353e00640 1 -- 172.21.15.62:0/1106675272 learned_addr learned my addr 172.21.15.62:0/1106675272 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe35aa00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c101b20 0x7fe35c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe353e00640 1 -- 172.21.15.62:0/1106675272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c101b20 msgr2=0x7fe35c197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe353e00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c101b20 0x7fe35c197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe353e00640 1 -- 172.21.15.62:0/1106675272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe35c1036d0 msgr2=0x7fe35c19f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:40.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe353e00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe35c1036d0 0x7fe35c19f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:40.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.149+0000 7fe353e00640 1 -- 172.21.15.62:0/1106675272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c19fb00 con 0x7fe35c102d10 2024-04-02T14:40:40.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.150+0000 7fe35aa00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c101b20 0x7fe35c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:40:40.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.150+0000 7fe353e00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe35c102d10 0x7fe35c198420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe34400ea60 tx=0x7fe34400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:40.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.150+0000 7fe353400640 1 -- 172.21.15.62:0/1106675272 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34400cde0 con 0x7fe35c102d10 2024-04-02T14:40:40.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.150+0000 7fe353400640 1 -- 172.21.15.62:0/1106675272 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe3440035f0 con 0x7fe35c102d10 2024-04-02T14:40:40.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.150+0000 7fe353400640 1 -- 172.21.15.62:0/1106675272 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344010720 con 0x7fe35c102d10 2024-04-02T14:40:40.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.150+0000 7fe361a00640 1 -- 172.21.15.62:0/1106675272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35c19fdf0 con 0x7fe35c102d10 2024-04-02T14:40:40.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.150+0000 7fe361a00640 1 -- 172.21.15.62:0/1106675272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35c1a0260 con 0x7fe35c102d10 2024-04-02T14:40:40.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.151+0000 7fe361a00640 1 -- 172.21.15.62:0/1106675272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe35c108de0 con 0x7fe35c102d10 2024-04-02T14:40:40.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.153+0000 7fe353400640 1 -- 172.21.15.62:0/1106675272 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe3440037a0 con 0x7fe35c102d10 2024-04-02T14:40:40.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.155+0000 7fe353400640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe3400a3c30 0x7fe3400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:40.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.155+0000 7fe353400640 1 -- 172.21.15.62:0/1106675272 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe3440d0320 con 0x7fe35c102d10 2024-04-02T14:40:40.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.155+0000 7fe35aa00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe3400a3c30 0x7fe3400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:40.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.156+0000 7fe35aa00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe3400a3c30 0x7fe3400a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe35c102b70 tx=0x7fe34c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:40.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.160+0000 7fe353400640 1 -- 172.21.15.62:0/1106675272 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe34409d100 con 0x7fe35c102d10 2024-04-02T14:40:40.278 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:40 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.204:6832/3512482618,v1:172.21.15.204:6833/3512482618] up:boot 2024-04-02T14:40:40.278 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:40 smithi113 ceph-mon[36135]: fsmap 1 up:standby 2024-04-02T14:40:40.278 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T14:40:40.278 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:40.278 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:40.278 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:40.279 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:40.279 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:40 smithi113 ceph-mon[36135]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:40.279 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:40.279 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:40 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:40.279 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:40 smithi113 ceph-mon[36135]: Deploying daemon mds.k on smithi113 2024-04-02T14:40:40.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:40 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.204:6832/3512482618,v1:172.21.15.204:6833/3512482618] up:boot 2024-04-02T14:40:40.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:40 smithi062 ceph-mon[32116]: fsmap 1 up:standby 2024-04-02T14:40:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T14:40:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:40 smithi062 ceph-mon[32116]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:40 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:40.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:40 smithi062 ceph-mon[32116]: Deploying daemon mds.k on smithi113 2024-04-02T14:40:40.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.416+0000 7fe361a00640 1 -- 172.21.15.62:0/1106675272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe35c02d370 con 0x7fe35c102d10 2024-04-02T14:40:40.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.417+0000 7fe353400640 1 -- 172.21.15.62:0/1106675272 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fe34409ca00 con 0x7fe35c102d10 2024-04-02T14:40:40.417 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:40.417 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T14:40:40.417 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 2 2024-04-02T14:40:40.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 -- 172.21.15.62:0/1106675272 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe3400a3c30 msgr2=0x7fe3400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:40.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe3400a3c30 0x7fe3400a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe35c102b70 tx=0x7fe34c005f90 comp rx=0 tx=0).stop 2024-04-02T14:40:40.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 -- 172.21.15.62:0/1106675272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe35c102d10 msgr2=0x7fe35c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:40.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe35c102d10 0x7fe35c198420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe34400ea60 tx=0x7fe34400ef20 comp rx=0 tx=0).stop 2024-04-02T14:40:40.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 -- 172.21.15.62:0/1106675272 shutdown_connections 2024-04-02T14:40:40.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe3400a3c30 0x7fe3400a60f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe35c1036d0 0x7fe35c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe35c102d10 0x7fe35c198420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 --2- 172.21.15.62:0/1106675272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c101b20 0x7fe35c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 -- 172.21.15.62:0/1106675272 >> 172.21.15.62:0/1106675272 conn(0x7fe35c0fd0b0 msgr2=0x7fe35c105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 -- 172.21.15.62:0/1106675272 shutdown_connections 2024-04-02T14:40:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:40.420+0000 7fe361a00640 1 -- 172.21.15.62:0/1106675272 wait complete. 2024-04-02T14:40:40.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:40 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.204:6832/3512482618,v1:172.21.15.204:6833/3512482618] up:boot 2024-04-02T14:40:40.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:40 smithi204 ceph-mon[35948]: fsmap 1 up:standby 2024-04-02T14:40:40.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T14:40:40.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:40.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:40.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:40.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:40.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:40 smithi204 ceph-mon[35948]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:40.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:40.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:40 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:40.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:40 smithi204 ceph-mon[35948]: Deploying daemon mds.k on smithi113 2024-04-02T14:40:41.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:41 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1106675272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:41.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:41 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1106675272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:41.620 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:40:41 smithi113 systemd[1]: Starting Ceph mds.k for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:40:41.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:41 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1106675272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:42.045 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:40:41 smithi113 podman[66195]: 2024-04-02 14:40:41.640819793 +0000 UTC m=+0.017984166 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:40:42.045 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:40:41 smithi113 podman[66195]: 2024-04-02 14:40:41.796978136 +0000 UTC m=+0.174142495 container create 5be66e02d69448a83d68ed8fd9a18c5d22b3bfeaf296e6e9ac63340a4154e894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-02T14:40:42.045 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:40:42 smithi113 podman[66195]: 2024-04-02 14:40:42.045705241 +0000 UTC m=+0.422869608 container init 5be66e02d69448a83d68ed8fd9a18c5d22b3bfeaf296e6e9ac63340a4154e894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:40:42.371 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:40:42 smithi113 podman[66195]: 2024-04-02 14:40:42.048597755 +0000 UTC m=+0.425762120 container start 5be66e02d69448a83d68ed8fd9a18c5d22b3bfeaf296e6e9ac63340a4154e894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-02T14:40:42.371 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:40:42 smithi113 ceph-mds[66216]: -- 172.21.15.113:0/2030686130 <== mon.2 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e29e7a7a40 con 0x55e29e77ac00 2024-04-02T14:40:42.371 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:40:42 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k[66212]: starting mds.k at 2024-04-02T14:40:42.372 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:40:42 smithi113 bash[66195]: 5be66e02d69448a83d68ed8fd9a18c5d22b3bfeaf296e6e9ac63340a4154e894 2024-04-02T14:40:42.372 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:40:42 smithi113 systemd[1]: Started Ceph mds.k for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:40:42.439 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:40:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.642+0000 7f0482e00640 1 -- 172.21.15.62:0/651749993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c1023d0 msgr2=0x7f047c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.642+0000 7f0482e00640 1 --2- 172.21.15.62:0/651749993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c1023d0 0x7f047c1047c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0470009900 tx=0x7f047002bc40 comp rx=0 tx=0).stop 2024-04-02T14:40:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.642+0000 7f0482e00640 1 -- 172.21.15.62:0/651749993 shutdown_connections 2024-04-02T14:40:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.642+0000 7f0482e00640 1 --2- 172.21.15.62:0/651749993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c107630 0x7f047c109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.642+0000 7f0482e00640 1 --2- 172.21.15.62:0/651749993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c104d00 0x7f047c1070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.642+0000 7f0482e00640 1 --2- 172.21.15.62:0/651749993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c1023d0 0x7f047c1047c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.642+0000 7f0482e00640 1 -- 172.21.15.62:0/651749993 >> 172.21.15.62:0/651749993 conn(0x7f047c0fbde0 msgr2=0x7f047c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.642+0000 7f0482e00640 1 -- 172.21.15.62:0/651749993 shutdown_connections 2024-04-02T14:40:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.642+0000 7f0482e00640 1 -- 172.21.15.62:0/651749993 wait complete. 2024-04-02T14:40:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.643+0000 7f0482e00640 1 Processor -- start 2024-04-02T14:40:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.643+0000 7f0482e00640 1 -- start start 2024-04-02T14:40:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.644+0000 7f0482e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c1023d0 0x7f047c101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.644+0000 7f0482e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c104d00 0x7f047c100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.644+0000 7f0482e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c107630 0x7f047c1008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.644+0000 7f0482e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f047c10d700 con 0x7f047c104d00 2024-04-02T14:40:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.644+0000 7f0482e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f047c10d580 con 0x7f047c1023d0 2024-04-02T14:40:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.644+0000 7f0482e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f047c10d880 con 0x7f047c107630 2024-04-02T14:40:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.644+0000 7f0481000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c104d00 0x7f047c100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.644+0000 7f0481000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c104d00 0x7f047c100380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37822/0 (socket says 172.21.15.62:37822) 2024-04-02T14:40:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.644+0000 7f0481000640 1 -- 172.21.15.62:0/2711202545 learned_addr learned my addr 172.21.15.62:0/2711202545 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.644+0000 7f0482400640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c107630 0x7f047c1008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.644+0000 7f0481a00640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c1023d0 0x7f047c101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f0482400640 1 -- 172.21.15.62:0/2711202545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c1023d0 msgr2=0x7f047c101ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f0482400640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c1023d0 0x7f047c101ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f0482400640 1 -- 172.21.15.62:0/2711202545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c104d00 msgr2=0x7f047c100380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f0482400640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c104d00 0x7f047c100380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f0482400640 1 -- 172.21.15.62:0/2711202545 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c1010c0 con 0x7f047c107630 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f0481a00640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c1023d0 0x7f047c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f0482400640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c107630 0x7f047c1008c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0474009fd0 tx=0x7f047400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f047a000640 1 -- 172.21.15.62:0/2711202545 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047400ce20 con 0x7f047c107630 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f047a000640 1 -- 172.21.15.62:0/2711202545 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0474004b20 con 0x7f047c107630 2024-04-02T14:40:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f0482e00640 1 -- 172.21.15.62:0/2711202545 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047c1a1ed0 con 0x7f047c107630 2024-04-02T14:40:42.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f047a000640 1 -- 172.21.15.62:0/2711202545 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474010790 con 0x7f047c107630 2024-04-02T14:40:42.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.645+0000 7f0482e00640 1 -- 172.21.15.62:0/2711202545 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047c1a2340 con 0x7f047c107630 2024-04-02T14:40:42.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.647+0000 7f0482e00640 1 -- 172.21.15.62:0/2711202545 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f047c0fd8d0 con 0x7f047c107630 2024-04-02T14:40:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.648+0000 7f047a000640 1 -- 172.21.15.62:0/2711202545 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0474005020 con 0x7f047c107630 2024-04-02T14:40:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.650+0000 7f047a000640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04640a3bb0 0x7f04640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.650+0000 7f047a000640 1 -- 172.21.15.62:0/2711202545 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f04740d0720 con 0x7f047c107630 2024-04-02T14:40:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.650+0000 7f0481a00640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04640a3bb0 0x7f04640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.650+0000 7f0481a00640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04640a3bb0 0x7f04640a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f047002c250 tx=0x7f047002c9e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:42.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.654+0000 7f047a000640 1 -- 172.21.15.62:0/2711202545 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f047409d500 con 0x7f047c107630 2024-04-02T14:40:42.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.903+0000 7f0482e00640 1 -- 172.21.15.62:0/2711202545 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f047c0627a0 con 0x7f047c107630 2024-04-02T14:40:42.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.903+0000 7f047a000640 1 -- 172.21.15.62:0/2711202545 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f047409ce00 con 0x7f047c107630 2024-04-02T14:40:42.903 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:42.903 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T14:40:42.904 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 3 2024-04-02T14:40:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 -- 172.21.15.62:0/2711202545 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04640a3bb0 msgr2=0x7f04640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04640a3bb0 0x7f04640a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f047002c250 tx=0x7f047002c9e0 comp rx=0 tx=0).stop 2024-04-02T14:40:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 -- 172.21.15.62:0/2711202545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c107630 msgr2=0x7f047c1008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c107630 0x7f047c1008c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0474009fd0 tx=0x7f047400ef10 comp rx=0 tx=0).stop 2024-04-02T14:40:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 -- 172.21.15.62:0/2711202545 shutdown_connections 2024-04-02T14:40:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f04640a3bb0 0x7f04640a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c107630 0x7f047c1008c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c104d00 0x7f047c100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 --2- 172.21.15.62:0/2711202545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c1023d0 0x7f047c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 -- 172.21.15.62:0/2711202545 >> 172.21.15.62:0/2711202545 conn(0x7f047c0fbde0 msgr2=0x7f047c102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 -- 172.21.15.62:0/2711202545 shutdown_connections 2024-04-02T14:40:42.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:42.906+0000 7f0482e00640 1 -- 172.21.15.62:0/2711202545 wait complete. 2024-04-02T14:40:43.455 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:43 smithi113 ceph-mon[36135]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:43.455 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:43 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:43.455 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:43 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:43.455 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:43 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:43.455 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:43 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:43.455 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:43 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:43.455 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:43 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:43.455 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:43 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.113:6832/1862883972,v1:172.21.15.113:6833/1862883972] up:boot 2024-04-02T14:40:43.455 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:43 smithi113 ceph-mon[36135]: fsmap 2 up:standby 2024-04-02T14:40:43.455 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:43 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:40:43.456 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:43 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2711202545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:43.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:43 smithi062 ceph-mon[32116]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:43.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:43 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:43.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:43 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:43.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:43 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:43.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:43 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:43.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:43 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:43.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:43 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:43.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:43 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.113:6832/1862883972,v1:172.21.15.113:6833/1862883972] up:boot 2024-04-02T14:40:43.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:43 smithi062 ceph-mon[32116]: fsmap 2 up:standby 2024-04-02T14:40:43.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:43 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:40:43.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:43 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2711202545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:43.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:43 smithi204 ceph-mon[35948]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:43.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:43 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:43.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:43 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:43.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:43 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:43.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:43 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:43.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:43 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:43.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:43 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:43.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:43 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.113:6832/1862883972,v1:172.21.15.113:6833/1862883972] up:boot 2024-04-02T14:40:43.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:43 smithi204 ceph-mon[35948]: fsmap 2 up:standby 2024-04-02T14:40:43.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:43 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:40:43.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:43 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2711202545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:44.500 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:44 smithi062 ceph-mon[32116]: Deploying daemon mds.j on smithi062 2024-04-02T14:40:44.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:44 smithi204 ceph-mon[35948]: Deploying daemon mds.j on smithi062 2024-04-02T14:40:44.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:44 smithi113 ceph-mon[36135]: Deploying daemon mds.j on smithi062 2024-04-02T14:40:44.925 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:40:45.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.154+0000 7fcdd6600640 1 -- 172.21.15.62:0/3928649607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd00764c0 msgr2=0x7fcdd00768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:45.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.154+0000 7fcdd6600640 1 --2- 172.21.15.62:0/3928649607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd00764c0 0x7fcdd00768e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009900 tx=0x7fcdc002bc40 comp rx=0 tx=0).stop 2024-04-02T14:40:45.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.154+0000 7fcdd6600640 1 -- 172.21.15.62:0/3928649607 shutdown_connections 2024-04-02T14:40:45.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.154+0000 7fcdd6600640 1 --2- 172.21.15.62:0/3928649607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd00780a0 0x7fcdd0078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:45.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.154+0000 7fcdd6600640 1 --2- 172.21.15.62:0/3928649607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdd0077700 0x7fcdd0077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:45.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.154+0000 7fcdd6600640 1 --2- 172.21.15.62:0/3928649607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd00764c0 0x7fcdd00768e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:45.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.154+0000 7fcdd6600640 1 -- 172.21.15.62:0/3928649607 >> 172.21.15.62:0/3928649607 conn(0x7fcdd00ff2b0 msgr2=0x7fcdd01016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:45.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.154+0000 7fcdd6600640 1 -- 172.21.15.62:0/3928649607 shutdown_connections 2024-04-02T14:40:45.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.154+0000 7fcdd6600640 1 -- 172.21.15.62:0/3928649607 wait complete. 2024-04-02T14:40:45.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.155+0000 7fcdd6600640 1 Processor -- start 2024-04-02T14:40:45.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdd6600640 1 -- start start 2024-04-02T14:40:45.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdd6600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd00764c0 0x7fcdd019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:45.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdd6600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0077700 0x7fcdd019ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdd6600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdd00780a0 0x7fcdd01a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdd6600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0113ef0 con 0x7fcdd0077700 2024-04-02T14:40:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdd6600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0113d70 con 0x7fcdd00780a0 2024-04-02T14:40:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdd6600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0114070 con 0x7fcdd00764c0 2024-04-02T14:40:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdcea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0077700 0x7fcdd019ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdcea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0077700 0x7fcdd019ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37834/0 (socket says 172.21.15.62:37834) 2024-04-02T14:40:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdcea00640 1 -- 172.21.15.62:0/2193357775 learned_addr learned my addr 172.21.15.62:0/2193357775 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdcf400640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd00764c0 0x7fcdd019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.156+0000 7fcdcfe00640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdd00780a0 0x7fcdd01a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.157+0000 7fcdcea00640 1 -- 172.21.15.62:0/2193357775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd00764c0 msgr2=0x7fcdd019e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.157+0000 7fcdcea00640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd00764c0 0x7fcdd019e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.157+0000 7fcdcea00640 1 -- 172.21.15.62:0/2193357775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdd00780a0 msgr2=0x7fcdd01a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.157+0000 7fcdcea00640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdd00780a0 0x7fcdd01a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.157+0000 7fcdcea00640 1 -- 172.21.15.62:0/2193357775 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd01a6100 con 0x7fcdd0077700 2024-04-02T14:40:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.157+0000 7fcdcea00640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0077700 0x7fcdd019ea20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00e9f0 tx=0x7fcdbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.157+0000 7fcdc7e00640 1 -- 172.21.15.62:0/2193357775 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc00cd80 con 0x7fcdd0077700 2024-04-02T14:40:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.157+0000 7fcdd6600640 1 -- 172.21.15.62:0/2193357775 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd01a63f0 con 0x7fcdd0077700 2024-04-02T14:40:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.157+0000 7fcdc7e00640 1 -- 172.21.15.62:0/2193357775 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcdbc010930 con 0x7fcdd0077700 2024-04-02T14:40:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.157+0000 7fcdc7e00640 1 -- 172.21.15.62:0/2193357775 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc018b00 con 0x7fcdd0077700 2024-04-02T14:40:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.158+0000 7fcdd6600640 1 -- 172.21.15.62:0/2193357775 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd01a6940 con 0x7fcdd0077700 2024-04-02T14:40:45.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.159+0000 7fcdd6600640 1 -- 172.21.15.62:0/2193357775 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd01107f0 con 0x7fcdd0077700 2024-04-02T14:40:45.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.161+0000 7fcdc7e00640 1 -- 172.21.15.62:0/2193357775 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcdbc018ca0 con 0x7fcdd0077700 2024-04-02T14:40:45.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.163+0000 7fcdc7e00640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcdb40a3bb0 0x7fcdb40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:45.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.163+0000 7fcdc7e00640 1 -- 172.21.15.62:0/2193357775 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fcdbc0d0370 con 0x7fcdd0077700 2024-04-02T14:40:45.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.163+0000 7fcdcf400640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcdb40a3bb0 0x7fcdb40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:45.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.164+0000 7fcdcf400640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcdb40a3bb0 0x7fcdb40a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcdc002c250 tx=0x7fcdc002c9e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:45.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.167+0000 7fcdc7e00640 1 -- 172.21.15.62:0/2193357775 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcdbc09cb00 con 0x7fcdd0077700 2024-04-02T14:40:45.330 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:45 smithi062 ceph-mon[32116]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:45.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.422+0000 7fcdd6600640 1 -- 172.21.15.62:0/2193357775 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdd002d370 con 0x7fcdd0077700 2024-04-02T14:40:45.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.423+0000 7fcdc7e00640 1 -- 172.21.15.62:0/2193357775 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fcdbc0a0e00 con 0x7fcdd0077700 2024-04-02T14:40:45.422 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:45.422 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T14:40:45.423 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 3 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 -- 172.21.15.62:0/2193357775 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcdb40a3bb0 msgr2=0x7fcdb40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcdb40a3bb0 0x7fcdb40a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcdc002c250 tx=0x7fcdc002c9e0 comp rx=0 tx=0).stop 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 -- 172.21.15.62:0/2193357775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0077700 msgr2=0x7fcdd019ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0077700 0x7fcdd019ea20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00e9f0 tx=0x7fcdbc00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 -- 172.21.15.62:0/2193357775 shutdown_connections 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcdb40a3bb0 0x7fcdb40a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdd00780a0 0x7fcdd01a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0077700 0x7fcdd019ea20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 --2- 172.21.15.62:0/2193357775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd00764c0 0x7fcdd019e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 -- 172.21.15.62:0/2193357775 >> 172.21.15.62:0/2193357775 conn(0x7fcdd00ff2b0 msgr2=0x7fcdd0100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 -- 172.21.15.62:0/2193357775 shutdown_connections 2024-04-02T14:40:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:45.425+0000 7fcdd6600640 1 -- 172.21.15.62:0/2193357775 wait complete. 2024-04-02T14:40:45.481 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:45 smithi204 ceph-mon[35948]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:45.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:45 smithi113 ceph-mon[36135]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:46.025 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:40:45 smithi062 podman[70947]: 2024-04-02 14:40:45.710868116 +0000 UTC m=+0.162479785 container create e1c39c3c50789e51f8aaf29f430d6baa7aacdbcfc32e67ef3b81d2d91eb6347c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-02T14:40:46.025 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:40:46 smithi062 podman[70947]: 2024-04-02 14:40:46.023426205 +0000 UTC m=+0.475037877 container init e1c39c3c50789e51f8aaf29f430d6baa7aacdbcfc32e67ef3b81d2d91eb6347c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:40:46.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:46 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2193357775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:46.343 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:40:46 smithi062 podman[70947]: 2024-04-02 14:40:46.034475393 +0000 UTC m=+0.486087041 container start e1c39c3c50789e51f8aaf29f430d6baa7aacdbcfc32e67ef3b81d2d91eb6347c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, ceph=True) 2024-04-02T14:40:46.343 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:40:46 smithi062 ceph-mds[70966]: -- 172.21.15.62:0/2273037953 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556aa58d5a40 con 0x556aa58a8c00 2024-04-02T14:40:46.343 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:40:46 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-j[70962]: starting mds.j at 2024-04-02T14:40:46.343 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:40:46 smithi062 bash[70947]: e1c39c3c50789e51f8aaf29f430d6baa7aacdbcfc32e67ef3b81d2d91eb6347c 2024-04-02T14:40:46.343 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:40:46 smithi062 systemd[1]: Started Ceph mds.j for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:40:46.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:46 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2193357775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:46.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:46 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2193357775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:47.445 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:40:47.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:47 smithi204 ceph-mon[35948]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:47.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:47 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.62:6834/1178681921,v1:172.21.15.62:6835/1178681921] up:boot 2024-04-02T14:40:47.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:47 smithi204 ceph-mon[35948]: fsmap 3 up:standby 2024-04-02T14:40:47.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T14:40:47.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:47.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:47.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:47.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:47.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:47.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:47 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:47.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:47 smithi062 ceph-mon[32116]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:47.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:47 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.62:6834/1178681921,v1:172.21.15.62:6835/1178681921] up:boot 2024-04-02T14:40:47.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:47 smithi062 ceph-mon[32116]: fsmap 3 up:standby 2024-04-02T14:40:47.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T14:40:47.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:47.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:47.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:47.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:47.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:47.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:47 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:47.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:47 smithi113 ceph-mon[36135]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:47.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:47 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.62:6834/1178681921,v1:172.21.15.62:6835/1178681921] up:boot 2024-04-02T14:40:47.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:47 smithi113 ceph-mon[36135]: fsmap 3 up:standby 2024-04-02T14:40:47.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T14:40:47.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:47.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:47.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:47.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:47.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:47.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:47 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:47.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.670+0000 7f0daf000640 1 -- 172.21.15.62:0/2933409719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0da8074500 msgr2=0x7f0da8074960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:47.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.670+0000 7f0daf000640 1 --2- 172.21.15.62:0/2933409719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0da8074500 0x7f0da8074960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009860 tx=0x7f0d9402bc00 comp rx=0 tx=0).stop 2024-04-02T14:40:47.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.670+0000 7f0daf000640 1 -- 172.21.15.62:0/2933409719 shutdown_connections 2024-04-02T14:40:47.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.670+0000 7f0daf000640 1 --2- 172.21.15.62:0/2933409719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da81097f0 0x7f0da810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:47.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.670+0000 7f0daf000640 1 --2- 172.21.15.62:0/2933409719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0da8074500 0x7f0da8074960 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:47.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.670+0000 7f0daf000640 1 --2- 172.21.15.62:0/2933409719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da8075ae0 0x7f0da8073f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:47.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.670+0000 7f0daf000640 1 -- 172.21.15.62:0/2933409719 >> 172.21.15.62:0/2933409719 conn(0x7f0da80fced0 msgr2=0x7f0da80ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:47.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.670+0000 7f0daf000640 1 -- 172.21.15.62:0/2933409719 shutdown_connections 2024-04-02T14:40:47.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.670+0000 7f0daf000640 1 -- 172.21.15.62:0/2933409719 wait complete. 2024-04-02T14:40:47.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.671+0000 7f0daf000640 1 Processor -- start 2024-04-02T14:40:47.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0daf000640 1 -- start start 2024-04-02T14:40:47.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0daf000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0da8074500 0x7f0da8199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:47.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0daf000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da8075ae0 0x7f0da819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:47.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0daf000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da81097f0 0x7f0da81a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:47.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0daf000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0da810f910 con 0x7f0da81097f0 2024-04-02T14:40:47.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0daf000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0da810f790 con 0x7f0da8074500 2024-04-02T14:40:47.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0daf000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0da810fa90 con 0x7f0da8075ae0 2024-04-02T14:40:47.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0dae600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da81097f0 0x7f0da81a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:47.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0dae600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da81097f0 0x7f0da81a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37874/0 (socket says 172.21.15.62:37874) 2024-04-02T14:40:47.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0dae600640 1 -- 172.21.15.62:0/1118746753 learned_addr learned my addr 172.21.15.62:0/1118746753 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:47.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0dadc00640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0da8074500 0x7f0da8199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:47.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.672+0000 7f0dad200640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da8075ae0 0x7f0da819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:47.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.673+0000 7f0dad200640 1 -- 172.21.15.62:0/1118746753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0da8074500 msgr2=0x7f0da8199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:47.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.673+0000 7f0dad200640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0da8074500 0x7f0da8199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:47.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.673+0000 7f0dad200640 1 -- 172.21.15.62:0/1118746753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da81097f0 msgr2=0x7f0da81a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:47.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.673+0000 7f0dad200640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da81097f0 0x7f0da81a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:47.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.673+0000 7f0dad200640 1 -- 172.21.15.62:0/1118746753 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da81a1b20 con 0x7f0da8075ae0 2024-04-02T14:40:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.673+0000 7f0dad200640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da8075ae0 0x7f0da819a440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d9402c140 tx=0x7f0d940346f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.673+0000 7f0da6000640 1 -- 172.21.15.62:0/1118746753 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94041020 con 0x7f0da8075ae0 2024-04-02T14:40:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.673+0000 7f0da6000640 1 -- 172.21.15.62:0/1118746753 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0d9402ce50 con 0x7f0da8075ae0 2024-04-02T14:40:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.673+0000 7f0da6000640 1 -- 172.21.15.62:0/1118746753 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94039040 con 0x7f0da8075ae0 2024-04-02T14:40:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.673+0000 7f0daf000640 1 -- 172.21.15.62:0/1118746753 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da81a1db0 con 0x7f0da8075ae0 2024-04-02T14:40:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.674+0000 7f0daf000640 1 -- 172.21.15.62:0/1118746753 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da81a2240 con 0x7f0da8075ae0 2024-04-02T14:40:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.675+0000 7f0daf000640 1 -- 172.21.15.62:0/1118746753 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da81085b0 con 0x7f0da8075ae0 2024-04-02T14:40:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.677+0000 7f0da6000640 1 -- 172.21.15.62:0/1118746753 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0d94044020 con 0x7f0da8075ae0 2024-04-02T14:40:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.678+0000 7f0da6000640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0d900a3bb0 0x7f0d900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.678+0000 7f0da6000640 1 -- 172.21.15.62:0/1118746753 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f0d940f1560 con 0x7f0da8075ae0 2024-04-02T14:40:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.678+0000 7f0dadc00640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0d900a3bb0 0x7f0d900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:47.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.679+0000 7f0dadc00640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0d900a3bb0 0x7f0d900a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009bd0 tx=0x7f0d9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:47.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.682+0000 7f0da6000640 1 -- 172.21.15.62:0/1118746753 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0d9404b020 con 0x7f0da8075ae0 2024-04-02T14:40:47.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.937+0000 7f0daf000640 1 -- 172.21.15.62:0/1118746753 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0da802d370 con 0x7f0da8075ae0 2024-04-02T14:40:47.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.938+0000 7f0da6000640 1 -- 172.21.15.62:0/1118746753 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f0d940be340 con 0x7f0da8075ae0 2024-04-02T14:40:47.938 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:47.938 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T14:40:47.938 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 4 2024-04-02T14:40:47.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 -- 172.21.15.62:0/1118746753 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0d900a3bb0 msgr2=0x7f0d900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:47.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0d900a3bb0 0x7f0d900a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009bd0 tx=0x7f0d9c005f90 comp rx=0 tx=0).stop 2024-04-02T14:40:47.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 -- 172.21.15.62:0/1118746753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da8075ae0 msgr2=0x7f0da819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:47.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da8075ae0 0x7f0da819a440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d9402c140 tx=0x7f0d940346f0 comp rx=0 tx=0).stop 2024-04-02T14:40:47.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 -- 172.21.15.62:0/1118746753 shutdown_connections 2024-04-02T14:40:47.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0d900a3bb0 0x7f0d900a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:47.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da81097f0 0x7f0da81a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:47.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da8075ae0 0x7f0da819a440 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:47.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 --2- 172.21.15.62:0/1118746753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0da8074500 0x7f0da8199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:47.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 -- 172.21.15.62:0/1118746753 >> 172.21.15.62:0/1118746753 conn(0x7f0da80fced0 msgr2=0x7f0da8105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:47.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 -- 172.21.15.62:0/1118746753 shutdown_connections 2024-04-02T14:40:47.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:47.941+0000 7f0daf000640 1 -- 172.21.15.62:0/1118746753 wait complete. 2024-04-02T14:40:48.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:48 smithi204 ceph-mon[35948]: Deploying daemon mds.d on smithi062 2024-04-02T14:40:48.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:48 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1118746753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:48.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:48 smithi062 ceph-mon[32116]: Deploying daemon mds.d on smithi062 2024-04-02T14:40:48.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:48 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1118746753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:48.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:48 smithi113 ceph-mon[36135]: Deploying daemon mds.d on smithi062 2024-04-02T14:40:48.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:48 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1118746753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:49.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:49 smithi204 ceph-mon[35948]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:49.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:49 smithi062 ceph-mon[32116]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:49.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:49 smithi113 ceph-mon[36135]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:49.961 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:40:50.033 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:40:49 smithi062 systemd[1]: Starting Ceph mds.d for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:40:50.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.165+0000 7fdb67400640 1 -- 172.21.15.62:0/75048728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60109cf0 msgr2=0x7fdb60110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:50.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.165+0000 7fdb67400640 1 --2- 172.21.15.62:0/75048728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60109cf0 0x7fdb60110580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009be0 tx=0x7fdb5802bef0 comp rx=0 tx=0).stop 2024-04-02T14:40:50.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.165+0000 7fdb67400640 1 -- 172.21.15.62:0/75048728 shutdown_connections 2024-04-02T14:40:50.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.165+0000 7fdb67400640 1 --2- 172.21.15.62:0/75048728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60109cf0 0x7fdb60110580 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:50.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.165+0000 7fdb67400640 1 --2- 172.21.15.62:0/75048728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb60109330 0x7fdb601097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:50.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.165+0000 7fdb67400640 1 --2- 172.21.15.62:0/75048728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb60108140 0x7fdb60108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:50.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.165+0000 7fdb67400640 1 -- 172.21.15.62:0/75048728 >> 172.21.15.62:0/75048728 conn(0x7fdb60077110 msgr2=0x7fdb60079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:50.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.166+0000 7fdb67400640 1 -- 172.21.15.62:0/75048728 shutdown_connections 2024-04-02T14:40:50.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.166+0000 7fdb67400640 1 -- 172.21.15.62:0/75048728 wait complete. 2024-04-02T14:40:50.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.167+0000 7fdb67400640 1 Processor -- start 2024-04-02T14:40:50.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.167+0000 7fdb67400640 1 -- start start 2024-04-02T14:40:50.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.167+0000 7fdb67400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb60108140 0x7fdb6019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:50.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.167+0000 7fdb67400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb60109330 0x7fdb6019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:50.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.167+0000 7fdb67400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60109cf0 0x7fdb601a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:50.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.167+0000 7fdb67400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdb60113ef0 con 0x7fdb60109330 2024-04-02T14:40:50.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.167+0000 7fdb67400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb60113d70 con 0x7fdb60108140 2024-04-02T14:40:50.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.167+0000 7fdb67400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdb60114070 con 0x7fdb60109cf0 2024-04-02T14:40:50.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb5fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb60109330 0x7fdb6019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:50.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb65400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60109cf0 0x7fdb601a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:50.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb64a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb60108140 0x7fdb6019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:50.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb5fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb60109330 0x7fdb6019ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37880/0 (socket says 172.21.15.62:37880) 2024-04-02T14:40:50.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb5fe00640 1 -- 172.21.15.62:0/1551112199 learned_addr learned my addr 172.21.15.62:0/1551112199 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:50.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb64a00640 1 -- 172.21.15.62:0/1551112199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60109cf0 msgr2=0x7fdb601a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:50.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb64a00640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60109cf0 0x7fdb601a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:50.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb64a00640 1 -- 172.21.15.62:0/1551112199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb60109330 msgr2=0x7fdb6019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:50.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb64a00640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb60109330 0x7fdb6019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:50.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb64a00640 1 -- 172.21.15.62:0/1551112199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb601a6140 con 0x7fdb60108140 2024-04-02T14:40:50.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb5fe00640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb60109330 0x7fdb6019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:40:50.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.168+0000 7fdb64a00640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb60108140 0x7fdb6019e520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400e9c0 tx=0x7fdb5400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:50.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.169+0000 7fdb5d600640 1 -- 172.21.15.62:0/1551112199 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5400ccc0 con 0x7fdb60108140 2024-04-02T14:40:50.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.169+0000 7fdb5d600640 1 -- 172.21.15.62:0/1551112199 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdb540035f0 con 0x7fdb60108140 2024-04-02T14:40:50.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.169+0000 7fdb67400640 1 -- 172.21.15.62:0/1551112199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb601a6430 con 0x7fdb60108140 2024-04-02T14:40:50.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.169+0000 7fdb5d600640 1 -- 172.21.15.62:0/1551112199 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5400ce60 con 0x7fdb60108140 2024-04-02T14:40:50.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.169+0000 7fdb67400640 1 -- 172.21.15.62:0/1551112199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb601a6930 con 0x7fdb60108140 2024-04-02T14:40:50.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.170+0000 7fdb67400640 1 -- 172.21.15.62:0/1551112199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb6010f390 con 0x7fdb60108140 2024-04-02T14:40:50.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.172+0000 7fdb5d600640 1 -- 172.21.15.62:0/1551112199 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdb540037a0 con 0x7fdb60108140 2024-04-02T14:40:50.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.173+0000 7fdb5d600640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb480a3bb0 0x7fdb480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:50.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.174+0000 7fdb5d600640 1 -- 172.21.15.62:0/1551112199 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fdb540d0340 con 0x7fdb60108140 2024-04-02T14:40:50.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.174+0000 7fdb5fe00640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb480a3bb0 0x7fdb480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:50.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.174+0000 7fdb5fe00640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb480a3bb0 0x7fdb480a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdb6019fa30 tx=0x7fdb4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:50.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.176+0000 7fdb5d600640 1 -- 172.21.15.62:0/1551112199 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdb5409d120 con 0x7fdb60108140 2024-04-02T14:40:50.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:50 smithi062 ceph-mon[32116]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:50.343 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:40:50 smithi062 podman[71288]: 2024-04-02 14:40:50.00513517 +0000 UTC m=+0.039651242 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:40:50.343 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:40:50 smithi062 podman[71288]: 2024-04-02 14:40:50.123361754 +0000 UTC m=+0.157877881 container create dab721609ae75c70b9310df6d2ca345e3033ba603aa169d7054387a1b1b7b84b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-02T14:40:50.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.431+0000 7fdb67400640 1 -- 172.21.15.62:0/1551112199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb6002d370 con 0x7fdb60108140 2024-04-02T14:40:50.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.431+0000 7fdb5d600640 1 -- 172.21.15.62:0/1551112199 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fdb5409ca20 con 0x7fdb60108140 2024-04-02T14:40:50.432 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:50.432 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T14:40:50.432 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 4 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 -- 172.21.15.62:0/1551112199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb480a3bb0 msgr2=0x7fdb480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb480a3bb0 0x7fdb480a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdb6019fa30 tx=0x7fdb4c009260 comp rx=0 tx=0).stop 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 -- 172.21.15.62:0/1551112199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb60108140 msgr2=0x7fdb6019e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb60108140 0x7fdb6019e520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400e9c0 tx=0x7fdb5400ee80 comp rx=0 tx=0).stop 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 -- 172.21.15.62:0/1551112199 shutdown_connections 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdb480a3bb0 0x7fdb480a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60109cf0 0x7fdb601a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb60109330 0x7fdb6019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 --2- 172.21.15.62:0/1551112199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb60108140 0x7fdb6019e520 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 -- 172.21.15.62:0/1551112199 >> 172.21.15.62:0/1551112199 conn(0x7fdb60077110 msgr2=0x7fdb6010c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 -- 172.21.15.62:0/1551112199 shutdown_connections 2024-04-02T14:40:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:50.434+0000 7fdb67400640 1 -- 172.21.15.62:0/1551112199 wait complete. 2024-04-02T14:40:50.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:50 smithi204 ceph-mon[35948]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:50.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:50 smithi113 ceph-mon[36135]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:50.843 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:40:50 smithi062 podman[71288]: 2024-04-02 14:40:50.371733959 +0000 UTC m=+0.406250099 container init dab721609ae75c70b9310df6d2ca345e3033ba603aa169d7054387a1b1b7b84b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T14:40:50.843 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:40:50 smithi062 podman[71288]: 2024-04-02 14:40:50.384411134 +0000 UTC m=+0.418927270 container start dab721609ae75c70b9310df6d2ca345e3033ba603aa169d7054387a1b1b7b84b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD) 2024-04-02T14:40:50.844 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:40:50 smithi062 ceph-mds[71345]: -- 172.21.15.62:0/598635629 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559248e2fa40 con 0x559248e03000 2024-04-02T14:40:50.844 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:40:50 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d[71341]: starting mds.d at 2024-04-02T14:40:50.844 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:40:50 smithi062 bash[71288]: dab721609ae75c70b9310df6d2ca345e3033ba603aa169d7054387a1b1b7b84b 2024-04-02T14:40:50.844 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:40:50 smithi062 systemd[1]: Started Ceph mds.d for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:40:51.505 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:51 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1551112199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:51.505 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:51 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:51.506 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:51 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:51.506 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:51 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:51.506 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:51 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:51.506 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:51 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:51.506 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:51 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:51.506 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:51 smithi113 ceph-mon[36135]: Deploying daemon mds.e on smithi113 2024-04-02T14:40:51.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:51 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1551112199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:51.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:51 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:51.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:51 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:51.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:51 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:51.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:51 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:51.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:51 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:51.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:51 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:51.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:51 smithi204 ceph-mon[35948]: Deploying daemon mds.e on smithi113 2024-04-02T14:40:51.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:51 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1551112199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:51.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:51 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:51.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:51 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:51.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:51 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:51.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:51 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:51.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:51 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:51.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:51 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:51.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:51 smithi062 ceph-mon[32116]: Deploying daemon mds.e on smithi113 2024-04-02T14:40:52.454 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:40:52.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:52 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.62:6836/3805165269,v1:172.21.15.62:6837/3805165269] up:boot 2024-04-02T14:40:52.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:52 smithi204 ceph-mon[35948]: fsmap 4 up:standby 2024-04-02T14:40:52.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:52 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T14:40:52.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:52 smithi204 ceph-mon[35948]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:52.547 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:52 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.62:6836/3805165269,v1:172.21.15.62:6837/3805165269] up:boot 2024-04-02T14:40:52.548 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:52 smithi062 ceph-mon[32116]: fsmap 4 up:standby 2024-04-02T14:40:52.548 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:52 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T14:40:52.548 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:52 smithi062 ceph-mon[32116]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:52.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:52 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.62:6836/3805165269,v1:172.21.15.62:6837/3805165269] up:boot 2024-04-02T14:40:52.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:52 smithi113 ceph-mon[36135]: fsmap 4 up:standby 2024-04-02T14:40:52.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:52 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T14:40:52.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:52 smithi113 ceph-mon[36135]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.689+0000 7f01d5800640 1 -- 172.21.15.62:0/333734504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0107f10 msgr2=0x7f01d0108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.689+0000 7f01d5800640 1 --2- 172.21.15.62:0/333734504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0107f10 0x7f01d0108350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009900 tx=0x7f01c002bc30 comp rx=0 tx=0).stop 2024-04-02T14:40:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.689+0000 7f01d5800640 1 -- 172.21.15.62:0/333734504 shutdown_connections 2024-04-02T14:40:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.689+0000 7f01d5800640 1 --2- 172.21.15.62:0/333734504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01d0109aa0 0x7f01d01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.689+0000 7f01d5800640 1 --2- 172.21.15.62:0/333734504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d01090e0 0x7f01d0109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.689+0000 7f01d5800640 1 --2- 172.21.15.62:0/333734504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0107f10 0x7f01d0108350 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.689+0000 7f01d5800640 1 -- 172.21.15.62:0/333734504 >> 172.21.15.62:0/333734504 conn(0x7f01d0076f10 msgr2=0x7f01d0079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.689+0000 7f01d5800640 1 -- 172.21.15.62:0/333734504 shutdown_connections 2024-04-02T14:40:52.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.689+0000 7f01d5800640 1 -- 172.21.15.62:0/333734504 wait complete. 2024-04-02T14:40:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.690+0000 7f01d5800640 1 Processor -- start 2024-04-02T14:40:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01d5800640 1 -- start start 2024-04-02T14:40:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01d5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0107f10 0x7f01d019e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01d5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01d01090e0 0x7f01d019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01d5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d0109aa0 0x7f01d01a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01d5800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f01d0113cb0 con 0x7f01d01090e0 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01d5800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f01d0113b30 con 0x7f01d0109aa0 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01d5800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01d0113e30 con 0x7f01d0107f10 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01cfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0107f10 0x7f01d019e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01d4e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d0109aa0 0x7f01d01a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01d01090e0 0x7f01d019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01cfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0107f10 0x7f01d019e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:34588/0 (socket says 172.21.15.62:34588) 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01cfe00640 1 -- 172.21.15.62:0/3531521827 learned_addr learned my addr 172.21.15.62:0/3531521827 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.691+0000 7f01d4e00640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d0109aa0 0x7f01d01a5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:33690/0 (socket says 172.21.15.62:33690) 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.692+0000 7f01d4e00640 1 -- 172.21.15.62:0/3531521827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0107f10 msgr2=0x7f01d019e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.692+0000 7f01d4e00640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0107f10 0x7f01d019e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.692+0000 7f01d4e00640 1 -- 172.21.15.62:0/3531521827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01d01090e0 msgr2=0x7f01d019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.692+0000 7f01d4e00640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01d01090e0 0x7f01d019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.692+0000 7f01d4e00640 1 -- 172.21.15.62:0/3531521827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d01a5f70 con 0x7f01d0109aa0 2024-04-02T14:40:52.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.692+0000 7f01d4e00640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d0109aa0 0x7f01d01a5900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01c800ec50 tx=0x7f01c800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:52.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.692+0000 7f01cd600640 1 -- 172.21.15.62:0/3531521827 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8003b70 con 0x7f01d0109aa0 2024-04-02T14:40:52.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.692+0000 7f01cd600640 1 -- 172.21.15.62:0/3531521827 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f01c8003d10 con 0x7f01d0109aa0 2024-04-02T14:40:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.692+0000 7f01d5800640 1 -- 172.21.15.62:0/3531521827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d01a6260 con 0x7f01d0109aa0 2024-04-02T14:40:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.692+0000 7f01cd600640 1 -- 172.21.15.62:0/3531521827 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8010920 con 0x7f01d0109aa0 2024-04-02T14:40:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.693+0000 7f01d5800640 1 -- 172.21.15.62:0/3531521827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d01a67b0 con 0x7f01d0109aa0 2024-04-02T14:40:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.694+0000 7f01d5800640 1 -- 172.21.15.62:0/3531521827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d010f2a0 con 0x7f01d0109aa0 2024-04-02T14:40:52.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.695+0000 7f01cd600640 1 -- 172.21.15.62:0/3531521827 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f01c8004020 con 0x7f01d0109aa0 2024-04-02T14:40:52.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.697+0000 7f01cd600640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f01b40a3c30 0x7f01b40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:52.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.697+0000 7f01cd600640 1 -- 172.21.15.62:0/3531521827 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f01c80d0490 con 0x7f01d0109aa0 2024-04-02T14:40:52.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.697+0000 7f01cfe00640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f01b40a3c30 0x7f01b40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:52.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.698+0000 7f01cfe00640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f01b40a3c30 0x7f01b40a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01c002ca30 tx=0x7f01c0009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:52.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.701+0000 7f01cd600640 1 -- 172.21.15.62:0/3531521827 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f01c809d270 con 0x7f01d0109aa0 2024-04-02T14:40:52.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.954+0000 7f01d5800640 1 -- 172.21.15.62:0/3531521827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01d0062700 con 0x7f01d0109aa0 2024-04-02T14:40:52.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.954+0000 7f01cd600640 1 -- 172.21.15.62:0/3531521827 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f01c809cb70 con 0x7f01d0109aa0 2024-04-02T14:40:52.955 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:52.955 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-02T14:40:52.955 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 5 2024-04-02T14:40:52.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.957+0000 7f01d5800640 1 -- 172.21.15.62:0/3531521827 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f01b40a3c30 msgr2=0x7f01b40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:52.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.957+0000 7f01d5800640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f01b40a3c30 0x7f01b40a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01c002ca30 tx=0x7f01c0009560 comp rx=0 tx=0).stop 2024-04-02T14:40:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.957+0000 7f01d5800640 1 -- 172.21.15.62:0/3531521827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d0109aa0 msgr2=0x7f01d01a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.957+0000 7f01d5800640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d0109aa0 0x7f01d01a5900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01c800ec50 tx=0x7f01c800c6a0 comp rx=0 tx=0).stop 2024-04-02T14:40:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.958+0000 7f01d5800640 1 -- 172.21.15.62:0/3531521827 shutdown_connections 2024-04-02T14:40:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.958+0000 7f01d5800640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f01b40a3c30 0x7f01b40a60f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.958+0000 7f01d5800640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d0109aa0 0x7f01d01a5900 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.958+0000 7f01d5800640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01d01090e0 0x7f01d019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.958+0000 7f01d5800640 1 --2- 172.21.15.62:0/3531521827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0107f10 0x7f01d019e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.958+0000 7f01d5800640 1 -- 172.21.15.62:0/3531521827 >> 172.21.15.62:0/3531521827 conn(0x7f01d0076f10 msgr2=0x7f01d010c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.958+0000 7f01d5800640 1 -- 172.21.15.62:0/3531521827 shutdown_connections 2024-04-02T14:40:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:52.958+0000 7f01d5800640 1 -- 172.21.15.62:0/3531521827 wait complete. 2024-04-02T14:40:53.035 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:40:52 smithi113 systemd[1]: Starting Ceph mds.e for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:40:53.356 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:53 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3531521827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:53.356 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:40:53 smithi113 podman[66522]: 2024-04-02 14:40:52.938245147 +0000 UTC m=+0.017930799 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:40:53.356 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:40:53 smithi113 podman[66522]: 2024-04-02 14:40:53.070840048 +0000 UTC m=+0.150525679 container create e9fd3363bc176939f37739638eaafb45abba031c9ac0645239ecb3fad0fb0d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:40:53.356 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:40:53 smithi113 podman[66522]: 2024-04-02 14:40:53.256434583 +0000 UTC m=+0.336120226 container init e9fd3363bc176939f37739638eaafb45abba031c9ac0645239ecb3fad0fb0d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2) 2024-04-02T14:40:53.356 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:40:53 smithi113 podman[66522]: 2024-04-02 14:40:53.259092085 +0000 UTC m=+0.338777725 container start e9fd3363bc176939f37739638eaafb45abba031c9ac0645239ecb3fad0fb0d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-02T14:40:53.356 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:40:53 smithi113 ceph-mds[66541]: -- 172.21.15.113:0/11258127 <== mon.2 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d399d1a40 con 0x556d399a4800 2024-04-02T14:40:53.356 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:40:53 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e[66537]: starting mds.e at 2024-04-02T14:40:53.356 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:40:53 smithi113 bash[66522]: e9fd3363bc176939f37739638eaafb45abba031c9ac0645239ecb3fad0fb0d58 2024-04-02T14:40:53.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:53 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3531521827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:53.620 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:40:53 smithi113 systemd[1]: Started Ceph mds.e for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:40:53.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:53 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3531521827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:54.677 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:54 smithi113 ceph-mon[36135]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:54.678 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:54 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:54.678 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:54 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:54.678 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:54 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:54.678 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:54 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:54.678 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:54 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:54.678 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:54 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:54.678 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:54 smithi113 ceph-mon[36135]: Deploying daemon mds.f on smithi204 2024-04-02T14:40:54.678 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:54 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.113:6834/360641878,v1:172.21.15.113:6835/360641878] up:boot 2024-04-02T14:40:54.678 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:54 smithi113 ceph-mon[36135]: fsmap 5 up:standby 2024-04-02T14:40:54.678 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:54 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:40:54.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:54 smithi204 ceph-mon[35948]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:54.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:54 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:54.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:54 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:54.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:54 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:54.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:54 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:54.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:54 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:54.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:54 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:54.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:54 smithi204 ceph-mon[35948]: Deploying daemon mds.f on smithi204 2024-04-02T14:40:54.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:54 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.113:6834/360641878,v1:172.21.15.113:6835/360641878] up:boot 2024-04-02T14:40:54.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:54 smithi204 ceph-mon[35948]: fsmap 5 up:standby 2024-04-02T14:40:54.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:54 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:40:54.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:54 smithi062 ceph-mon[32116]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:54.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:54 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:54.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:54 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:54.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:54 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:54.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:54 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:54.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:54 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:54.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:54 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:54.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:54 smithi062 ceph-mon[32116]: Deploying daemon mds.f on smithi204 2024-04-02T14:40:54.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:54 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.113:6834/360641878,v1:172.21.15.113:6835/360641878] up:boot 2024-04-02T14:40:54.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:54 smithi062 ceph-mon[32116]: fsmap 5 up:standby 2024-04-02T14:40:54.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:54 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:40:54.979 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:40:55.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.203+0000 7f5a17200640 1 -- 172.21.15.62:0/1529536723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a10109330 msgr2=0x7f5a101097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:55.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.203+0000 7f5a17200640 1 --2- 172.21.15.62:0/1529536723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a10109330 0x7f5a101097b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009900 tx=0x7f59fc02bc30 comp rx=0 tx=0).stop 2024-04-02T14:40:55.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.203+0000 7f5a17200640 1 -- 172.21.15.62:0/1529536723 shutdown_connections 2024-04-02T14:40:55.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.203+0000 7f5a17200640 1 --2- 172.21.15.62:0/1529536723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a10109cf0 0x7f5a10110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:55.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.203+0000 7f5a17200640 1 --2- 172.21.15.62:0/1529536723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a10109330 0x7f5a101097b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:55.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.203+0000 7f5a17200640 1 --2- 172.21.15.62:0/1529536723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10108140 0x7f5a10108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:55.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.203+0000 7f5a17200640 1 -- 172.21.15.62:0/1529536723 >> 172.21.15.62:0/1529536723 conn(0x7f5a10077110 msgr2=0x7f5a10079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.203+0000 7f5a17200640 1 -- 172.21.15.62:0/1529536723 shutdown_connections 2024-04-02T14:40:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.203+0000 7f5a17200640 1 -- 172.21.15.62:0/1529536723 wait complete. 2024-04-02T14:40:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a17200640 1 Processor -- start 2024-04-02T14:40:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a17200640 1 -- start start 2024-04-02T14:40:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a17200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a10108140 0x7f5a1019e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a17200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10109330 0x7f5a1019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a17200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a10109cf0 0x7f5a101a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a17200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10113ea0 con 0x7f5a10109cf0 2024-04-02T14:40:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a17200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10113d20 con 0x7f5a10108140 2024-04-02T14:40:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a17200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10114020 con 0x7f5a10109330 2024-04-02T14:40:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a15200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a10109cf0 0x7f5a101a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a0fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a10108140 0x7f5a1019e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a15200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a10109cf0 0x7f5a101a5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53620/0 (socket says 172.21.15.62:53620) 2024-04-02T14:40:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.204+0000 7f5a15200640 1 -- 172.21.15.62:0/1746066949 learned_addr learned my addr 172.21.15.62:0/1746066949 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a0f400640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10109330 0x7f5a1019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a15200640 1 -- 172.21.15.62:0/1746066949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10109330 msgr2=0x7f5a1019ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a15200640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10109330 0x7f5a1019ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a15200640 1 -- 172.21.15.62:0/1746066949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a10108140 msgr2=0x7f5a1019e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a15200640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a10108140 0x7f5a1019e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a15200640 1 -- 172.21.15.62:0/1746066949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a101a6120 con 0x7f5a10109cf0 2024-04-02T14:40:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a0fe00640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a10108140 0x7f5a1019e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:40:55.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a15200640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a10109cf0 0x7f5a101a5ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800efd0 tx=0x7f5a0800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:55.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a0cc00640 1 -- 172.21.15.62:0/1746066949 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08003b70 con 0x7f5a10109cf0 2024-04-02T14:40:55.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a0cc00640 1 -- 172.21.15.62:0/1746066949 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5a08003d10 con 0x7f5a10109cf0 2024-04-02T14:40:55.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a0cc00640 1 -- 172.21.15.62:0/1746066949 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08010a90 con 0x7f5a10109cf0 2024-04-02T14:40:55.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.205+0000 7f5a17200640 1 -- 172.21.15.62:0/1746066949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a101a6410 con 0x7f5a10109cf0 2024-04-02T14:40:55.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.206+0000 7f5a17200640 1 -- 172.21.15.62:0/1746066949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a101a6930 con 0x7f5a10109cf0 2024-04-02T14:40:55.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.207+0000 7f5a17200640 1 -- 172.21.15.62:0/1746066949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1010f460 con 0x7f5a10109cf0 2024-04-02T14:40:55.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.209+0000 7f5a0cc00640 1 -- 172.21.15.62:0/1746066949 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5a08004000 con 0x7f5a10109cf0 2024-04-02T14:40:55.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.210+0000 7f5a0cc00640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f59f40a3c30 0x7f59f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:55.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.210+0000 7f5a0cc00640 1 -- 172.21.15.62:0/1746066949 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f5a080d03b0 con 0x7f5a10109cf0 2024-04-02T14:40:55.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.210+0000 7f5a0fe00640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f59f40a3c30 0x7f59f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:55.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.211+0000 7f5a0fe00640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f59f40a3c30 0x7f59f40a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5a10109190 tx=0x7f5a00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:55.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.215+0000 7f5a0cc00640 1 -- 172.21.15.62:0/1746066949 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5a0809d190 con 0x7f5a10109cf0 2024-04-02T14:40:55.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.453+0000 7f5a17200640 1 -- 172.21.15.62:0/1746066949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a1002d370 con 0x7f5a10109cf0 2024-04-02T14:40:55.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.454+0000 7f5a0cc00640 1 -- 172.21.15.62:0/1746066949 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f5a0809ca90 con 0x7f5a10109cf0 2024-04-02T14:40:55.455 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:55.455 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-02T14:40:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 6 2024-04-02T14:40:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.457+0000 7f5a17200640 1 -- 172.21.15.62:0/1746066949 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f59f40a3c30 msgr2=0x7f59f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.457+0000 7f5a17200640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f59f40a3c30 0x7f59f40a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5a10109190 tx=0x7f5a00005f90 comp rx=0 tx=0).stop 2024-04-02T14:40:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.457+0000 7f5a17200640 1 -- 172.21.15.62:0/1746066949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a10109cf0 msgr2=0x7f5a101a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.457+0000 7f5a17200640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a10109cf0 0x7f5a101a5ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800efd0 tx=0x7f5a0800c6a0 comp rx=0 tx=0).stop 2024-04-02T14:40:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.457+0000 7f5a17200640 1 -- 172.21.15.62:0/1746066949 shutdown_connections 2024-04-02T14:40:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.457+0000 7f5a17200640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f59f40a3c30 0x7f59f40a60f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.457+0000 7f5a17200640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a10109cf0 0x7f5a101a5ab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.457+0000 7f5a17200640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10109330 0x7f5a1019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.458+0000 7f5a17200640 1 --2- 172.21.15.62:0/1746066949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a10108140 0x7f5a1019e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.458+0000 7f5a17200640 1 -- 172.21.15.62:0/1746066949 >> 172.21.15.62:0/1746066949 conn(0x7f5a10077110 msgr2=0x7f5a1010c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.458+0000 7f5a17200640 1 -- 172.21.15.62:0/1746066949 shutdown_connections 2024-04-02T14:40:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:55.458+0000 7f5a17200640 1 -- 172.21.15.62:0/1746066949 wait complete. 2024-04-02T14:40:55.759 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:40:55 smithi204 systemd[1]: Starting Ceph mds.f for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:40:56.150 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:40:55 smithi204 podman[66987]: 2024-04-02 14:40:55.764721352 +0000 UTC m=+0.017657219 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:40:56.150 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:40:55 smithi204 podman[66987]: 2024-04-02 14:40:55.881363877 +0000 UTC m=+0.134299738 container create 0719b437b19e38afba04c13a63cac3b4fc0dfd3cfc3a3c0e25199ba230f5239f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:40:56.150 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:40:56 smithi204 podman[66987]: 2024-04-02 14:40:56.150766305 +0000 UTC m=+0.403702165 container init 0719b437b19e38afba04c13a63cac3b4fc0dfd3cfc3a3c0e25199ba230f5239f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-02T14:40:56.413 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:40:56 smithi204 podman[66987]: 2024-04-02 14:40:56.153457737 +0000 UTC m=+0.406393599 container start 0719b437b19e38afba04c13a63cac3b4fc0dfd3cfc3a3c0e25199ba230f5239f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2) 2024-04-02T14:40:56.414 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:40:56 smithi204 ceph-mds[67006]: -- 172.21.15.204:0/2738054573 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5573e6671a40 con 0x5573e6644c00 2024-04-02T14:40:56.414 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:40:56 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f[67002]: starting mds.f at 2024-04-02T14:40:56.414 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:40:56 smithi204 bash[66987]: 0719b437b19e38afba04c13a63cac3b4fc0dfd3cfc3a3c0e25199ba230f5239f 2024-04-02T14:40:56.414 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:40:56 smithi204 systemd[1]: Started Ceph mds.f for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:40:56.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:56 smithi204 ceph-mon[35948]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:56.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:56 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1746066949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:56.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:56 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:56.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:56 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:56.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:56 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:56.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:56 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:56.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:56 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:56.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:56 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:56.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:56 smithi062 ceph-mon[32116]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:56.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:56 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1746066949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:56.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:56 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:56.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:56 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:56.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:56 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:56.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:56 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:56.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:56 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:56.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:56 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:56.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:56 smithi113 ceph-mon[36135]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:56.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:56 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1746066949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:56.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:56 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:56.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:56 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:56.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:56 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:40:56.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:56 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:40:56.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:56 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:40:56.871 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:56 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:40:57.479 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:40:57.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.707+0000 7ff58be00640 1 -- 172.21.15.62:0/735642842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c074560 msgr2=0x7ff58c0749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:57.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.707+0000 7ff58be00640 1 --2- 172.21.15.62:0/735642842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c074560 0x7ff58c0749e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff578009860 tx=0x7ff57802bb70 comp rx=0 tx=0).stop 2024-04-02T14:40:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.707+0000 7ff58be00640 1 -- 172.21.15.62:0/735642842 shutdown_connections 2024-04-02T14:40:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.707+0000 7ff58be00640 1 --2- 172.21.15.62:0/735642842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58c10df10 0x7ff58c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.707+0000 7ff58be00640 1 --2- 172.21.15.62:0/735642842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c074560 0x7ff58c0749e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.707+0000 7ff58be00640 1 --2- 172.21.15.62:0/735642842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c075b40 0x7ff58c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.707+0000 7ff58be00640 1 -- 172.21.15.62:0/735642842 >> 172.21.15.62:0/735642842 conn(0x7ff58c0fce90 msgr2=0x7ff58c0ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.707+0000 7ff58be00640 1 -- 172.21.15.62:0/735642842 shutdown_connections 2024-04-02T14:40:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.707+0000 7ff58be00640 1 -- 172.21.15.62:0/735642842 wait complete. 2024-04-02T14:40:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.708+0000 7ff58be00640 1 Processor -- start 2024-04-02T14:40:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.708+0000 7ff58be00640 1 -- start start 2024-04-02T14:40:57.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.708+0000 7ff58be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c074560 0x7ff58c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.708+0000 7ff58be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c075b40 0x7ff58c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.708+0000 7ff58be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58c10df10 0x7ff58c1a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.708+0000 7ff58be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c113cd0 con 0x7ff58c10df10 2024-04-02T14:40:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.708+0000 7ff58be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c113b50 con 0x7ff58c075b40 2024-04-02T14:40:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.708+0000 7ff58be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c113e50 con 0x7ff58c074560 2024-04-02T14:40:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.709+0000 7ff58b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58c10df10 0x7ff58c1a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.709+0000 7ff58b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58c10df10 0x7ff58c1a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53640/0 (socket says 172.21.15.62:53640) 2024-04-02T14:40:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.709+0000 7ff58aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c074560 0x7ff58c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.709+0000 7ff58b400640 1 -- 172.21.15.62:0/201491304 learned_addr learned my addr 172.21.15.62:0/201491304 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:40:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.709+0000 7ff58a000640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c075b40 0x7ff58c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.709+0000 7ff58b400640 1 -- 172.21.15.62:0/201491304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c074560 msgr2=0x7ff58c19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.709+0000 7ff58b400640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c074560 0x7ff58c19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.709+0000 7ff58b400640 1 -- 172.21.15.62:0/201491304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c075b40 msgr2=0x7ff58c19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.709+0000 7ff58b400640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c075b40 0x7ff58c19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.709+0000 7ff58b400640 1 -- 172.21.15.62:0/201491304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c1a5f80 con 0x7ff58c10df10 2024-04-02T14:40:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.709+0000 7ff58b400640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58c10df10 0x7ff58c1a5910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff58400eb10 tx=0x7ff58400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.710+0000 7ff583400640 1 -- 172.21.15.62:0/201491304 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58400ce20 con 0x7ff58c10df10 2024-04-02T14:40:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.710+0000 7ff583400640 1 -- 172.21.15.62:0/201491304 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff584004b20 con 0x7ff58c10df10 2024-04-02T14:40:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.710+0000 7ff58be00640 1 -- 172.21.15.62:0/201491304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c1a6270 con 0x7ff58c10df10 2024-04-02T14:40:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.710+0000 7ff583400640 1 -- 172.21.15.62:0/201491304 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584010760 con 0x7ff58c10df10 2024-04-02T14:40:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.710+0000 7ff58be00640 1 -- 172.21.15.62:0/201491304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c1a6740 con 0x7ff58c10df10 2024-04-02T14:40:57.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.711+0000 7ff58be00640 1 -- 172.21.15.62:0/201491304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58c103fd0 con 0x7ff58c10df10 2024-04-02T14:40:57.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.713+0000 7ff583400640 1 -- 172.21.15.62:0/201491304 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff584005020 con 0x7ff58c10df10 2024-04-02T14:40:57.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.714+0000 7ff583400640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff5700a3bb0 0x7ff5700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:40:57.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.715+0000 7ff583400640 1 -- 172.21.15.62:0/201491304 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff5840d0570 con 0x7ff58c10df10 2024-04-02T14:40:57.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.715+0000 7ff58aa00640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff5700a3bb0 0x7ff5700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:40:57.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.715+0000 7ff58aa00640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff5700a3bb0 0x7ff5700a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005fd0 tx=0x7ff57c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:40:57.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.720+0000 7ff583400640 1 -- 172.21.15.62:0/201491304 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff58409d350 con 0x7ff58c10df10 2024-04-02T14:40:57.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:57 smithi204 ceph-mon[35948]: Deploying daemon mds.c on smithi204 2024-04-02T14:40:57.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:57 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.204:6834/3756932584,v1:172.21.15.204:6835/3756932584] up:boot 2024-04-02T14:40:57.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:57 smithi204 ceph-mon[35948]: fsmap 6 up:standby 2024-04-02T14:40:57.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:57 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:40:57.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:57 smithi062 ceph-mon[32116]: Deploying daemon mds.c on smithi204 2024-04-02T14:40:57.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:57 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.204:6834/3756932584,v1:172.21.15.204:6835/3756932584] up:boot 2024-04-02T14:40:57.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:57 smithi062 ceph-mon[32116]: fsmap 6 up:standby 2024-04-02T14:40:57.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:57 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:40:57.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:57 smithi113 ceph-mon[36135]: Deploying daemon mds.c on smithi204 2024-04-02T14:40:57.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:57 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.204:6834/3756932584,v1:172.21.15.204:6835/3756932584] up:boot 2024-04-02T14:40:57.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:57 smithi113 ceph-mon[36135]: fsmap 6 up:standby 2024-04-02T14:40:57.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:57 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:40:57.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.970+0000 7ff58be00640 1 -- 172.21.15.62:0/201491304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff58c02d370 con 0x7ff58c10df10 2024-04-02T14:40:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.971+0000 7ff583400640 1 -- 172.21.15.62:0/201491304 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7ff58409cc50 con 0x7ff58c10df10 2024-04-02T14:40:57.972 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:40:57.972 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-02T14:40:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-02T14:40:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 -- 172.21.15.62:0/201491304 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff5700a3bb0 msgr2=0x7ff5700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff5700a3bb0 0x7ff5700a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005fd0 tx=0x7ff57c009260 comp rx=0 tx=0).stop 2024-04-02T14:40:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 -- 172.21.15.62:0/201491304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58c10df10 msgr2=0x7ff58c1a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:40:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58c10df10 0x7ff58c1a5910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff58400eb10 tx=0x7ff58400ef10 comp rx=0 tx=0).stop 2024-04-02T14:40:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 -- 172.21.15.62:0/201491304 shutdown_connections 2024-04-02T14:40:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff5700a3bb0 0x7ff5700a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58c10df10 0x7ff58c1a5910 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c075b40 0x7ff58c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 --2- 172.21.15.62:0/201491304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c074560 0x7ff58c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:40:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 -- 172.21.15.62:0/201491304 >> 172.21.15.62:0/201491304 conn(0x7ff58c0fce90 msgr2=0x7ff58c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:40:57.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 -- 172.21.15.62:0/201491304 shutdown_connections 2024-04-02T14:40:57.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:40:57.974+0000 7ff58be00640 1 -- 172.21.15.62:0/201491304 wait complete. 2024-04-02T14:40:58.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:58 smithi204 ceph-mon[35948]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:58.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:40:58 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/201491304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:58.758 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:40:58 smithi204 systemd[1]: Starting Ceph mds.c for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:40:58.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:58 smithi062 ceph-mon[32116]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:58.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:40:58 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/201491304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:58.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:58 smithi113 ceph-mon[36135]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:40:58.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:40:58 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/201491304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:40:59.009 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:40:58 smithi204 podman[67295]: 2024-04-02 14:40:58.66093396 +0000 UTC m=+0.017808595 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:40:59.009 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:40:58 smithi204 podman[67295]: 2024-04-02 14:40:58.760701837 +0000 UTC m=+0.117576472 container create b1021335a446eea8ab19d6ff3a7e7a2a89ccfc634225a5973e4d209090e2eb71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=) 2024-04-02T14:40:59.509 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:40:59 smithi204 podman[67295]: 2024-04-02 14:40:59.013277731 +0000 UTC m=+0.370152365 container init b1021335a446eea8ab19d6ff3a7e7a2a89ccfc634225a5973e4d209090e2eb71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.build-date=20240326) 2024-04-02T14:40:59.509 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:40:59 smithi204 podman[67295]: 2024-04-02 14:40:59.01582816 +0000 UTC m=+0.372702795 container start b1021335a446eea8ab19d6ff3a7e7a2a89ccfc634225a5973e4d209090e2eb71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-04-02T14:40:59.509 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:40:59 smithi204 ceph-mds[67314]: -- 172.21.15.204:0/3650176207 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55758d789a40 con 0x55758d75d000 2024-04-02T14:40:59.509 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:40:59 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c[67310]: starting mds.c at 2024-04-02T14:40:59.509 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:40:59 smithi204 bash[67295]: b1021335a446eea8ab19d6ff3a7e7a2a89ccfc634225a5973e4d209090e2eb71 2024-04-02T14:40:59.509 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:40:59 smithi204 systemd[1]: Started Ceph mds.c for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:40:59.997 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:41:00.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.229+0000 7f3ecee00640 1 -- 172.21.15.62:0/2155002949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8108140 msgr2=0x7f3ec8108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:00.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.229+0000 7f3ecee00640 1 --2- 172.21.15.62:0/2155002949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8108140 0x7f3ec8108560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009ca0 tx=0x7f3eb802c100 comp rx=0 tx=0).stop 2024-04-02T14:41:00.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.229+0000 7f3ecee00640 1 -- 172.21.15.62:0/2155002949 shutdown_connections 2024-04-02T14:41:00.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.229+0000 7f3ecee00640 1 --2- 172.21.15.62:0/2155002949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ec8109cf0 0x7f3ec8110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:00.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.229+0000 7f3ecee00640 1 --2- 172.21.15.62:0/2155002949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec8109330 0x7f3ec81097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:00.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.229+0000 7f3ecee00640 1 --2- 172.21.15.62:0/2155002949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8108140 0x7f3ec8108560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:00.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.229+0000 7f3ecee00640 1 -- 172.21.15.62:0/2155002949 >> 172.21.15.62:0/2155002949 conn(0x7f3ec8077110 msgr2=0x7f3ec8079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:00.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.229+0000 7f3ecee00640 1 -- 172.21.15.62:0/2155002949 shutdown_connections 2024-04-02T14:41:00.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.230+0000 7f3ecee00640 1 -- 172.21.15.62:0/2155002949 wait complete. 2024-04-02T14:41:00.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.231+0000 7f3ecee00640 1 Processor -- start 2024-04-02T14:41:00.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.231+0000 7f3ecee00640 1 -- start start 2024-04-02T14:41:00.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.231+0000 7f3ecee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec8108140 0x7f3ec819e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:00.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.231+0000 7f3ecee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ec8109330 0x7f3ec819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:00.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.231+0000 7f3ecee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8109cf0 0x7f3ec81a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:00.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.231+0000 7f3ecee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8113ef0 con 0x7f3ec8109330 2024-04-02T14:41:00.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.231+0000 7f3ecee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8113d70 con 0x7f3ec8108140 2024-04-02T14:41:00.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.231+0000 7f3ecee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8114070 con 0x7f3ec8109cf0 2024-04-02T14:41:00.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.232+0000 7f3ebf400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ec8109330 0x7f3ec819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.232+0000 7f3ec7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec8108140 0x7f3ec819e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.232+0000 7f3ebf400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ec8109330 0x7f3ec819ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53662/0 (socket says 172.21.15.62:53662) 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.232+0000 7f3ebf400640 1 -- 172.21.15.62:0/689674044 learned_addr learned my addr 172.21.15.62:0/689674044 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.232+0000 7f3ecce00640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8109cf0 0x7f3ec81a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.232+0000 7f3ec7e00640 1 -- 172.21.15.62:0/689674044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8109cf0 msgr2=0x7f3ec81a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.232+0000 7f3ec7e00640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8109cf0 0x7f3ec81a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.232+0000 7f3ec7e00640 1 -- 172.21.15.62:0/689674044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ec8109330 msgr2=0x7f3ec819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.232+0000 7f3ec7e00640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ec8109330 0x7f3ec819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.232+0000 7f3ec7e00640 1 -- 172.21.15.62:0/689674044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec81a6120 con 0x7f3ec8108140 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.232+0000 7f3ebf400640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ec8109330 0x7f3ec819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.233+0000 7f3ec7e00640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec8108140 0x7f3ec819e500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8037040 tx=0x7f3eb8034650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:00.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.233+0000 7f3ec5600640 1 -- 172.21.15.62:0/689674044 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8039040 con 0x7f3ec8108140 2024-04-02T14:41:00.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.233+0000 7f3ec5600640 1 -- 172.21.15.62:0/689674044 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3eb8032340 con 0x7f3ec8108140 2024-04-02T14:41:00.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.233+0000 7f3ec5600640 1 -- 172.21.15.62:0/689674044 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb80045a0 con 0x7f3ec8108140 2024-04-02T14:41:00.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.233+0000 7f3ecee00640 1 -- 172.21.15.62:0/689674044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec81a63b0 con 0x7f3ec8108140 2024-04-02T14:41:00.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.233+0000 7f3ecee00640 1 -- 172.21.15.62:0/689674044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec81a6820 con 0x7f3ec8108140 2024-04-02T14:41:00.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.234+0000 7f3ecee00640 1 -- 172.21.15.62:0/689674044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec810f400 con 0x7f3ec8108140 2024-04-02T14:41:00.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.236+0000 7f3ec5600640 1 -- 172.21.15.62:0/689674044 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3eb8046030 con 0x7f3ec8108140 2024-04-02T14:41:00.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.237+0000 7f3ec5600640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3eac0a3c30 0x7f3eac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:00.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.237+0000 7f3ec5600640 1 -- 172.21.15.62:0/689674044 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f3eb80f1b00 con 0x7f3ec8108140 2024-04-02T14:41:00.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.237+0000 7f3ebf400640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3eac0a3c30 0x7f3eac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:00.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.238+0000 7f3ebf400640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3eac0a3c30 0x7f3eac0a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3ec819fa10 tx=0x7f3eb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.241+0000 7f3ec5600640 1 -- 172.21.15.62:0/689674044 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3eb804d060 con 0x7f3ec8108140 2024-04-02T14:41:00.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:00.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:00.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:00.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:00.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:00.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:00.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:00 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.204:6836/3537379638,v1:172.21.15.204:6837/3537379638] up:boot 2024-04-02T14:41:00.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:00 smithi062 ceph-mon[32116]: fsmap 7 up:standby 2024-04-02T14:41:00.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:00 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:41:00.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.494+0000 7f3ecee00640 1 -- 172.21.15.62:0/689674044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ec802d370 con 0x7f3ec8108140 2024-04-02T14:41:00.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.495+0000 7f3ec5600640 1 -- 172.21.15.62:0/689674044 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f3eb80be970 con 0x7f3ec8108140 2024-04-02T14:41:00.496 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:00.496 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-02T14:41:00.496 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-04-02T14:41:00.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.498+0000 7f3ecee00640 1 -- 172.21.15.62:0/689674044 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3eac0a3c30 msgr2=0x7f3eac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:00.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.498+0000 7f3ecee00640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3eac0a3c30 0x7f3eac0a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3ec819fa10 tx=0x7f3eb0005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:00.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.498+0000 7f3ecee00640 1 -- 172.21.15.62:0/689674044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec8108140 msgr2=0x7f3ec819e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:00.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.498+0000 7f3ecee00640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec8108140 0x7f3ec819e500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8037040 tx=0x7f3eb8034650 comp rx=0 tx=0).stop 2024-04-02T14:41:00.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.499+0000 7f3ecee00640 1 -- 172.21.15.62:0/689674044 shutdown_connections 2024-04-02T14:41:00.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.499+0000 7f3ecee00640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3eac0a3c30 0x7f3eac0a60f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:00.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.499+0000 7f3ecee00640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8109cf0 0x7f3ec81a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:00.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.499+0000 7f3ecee00640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ec8109330 0x7f3ec819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:00.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.499+0000 7f3ecee00640 1 --2- 172.21.15.62:0/689674044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec8108140 0x7f3ec819e500 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:00.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.499+0000 7f3ecee00640 1 -- 172.21.15.62:0/689674044 >> 172.21.15.62:0/689674044 conn(0x7f3ec8077110 msgr2=0x7f3ec810c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:00.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.499+0000 7f3ecee00640 1 -- 172.21.15.62:0/689674044 shutdown_connections 2024-04-02T14:41:00.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:00.499+0000 7f3ecee00640 1 -- 172.21.15.62:0/689674044 wait complete. 2024-04-02T14:41:00.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:00.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:00.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:00.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:00.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:00.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:00.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:00 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.204:6836/3537379638,v1:172.21.15.204:6837/3537379638] up:boot 2024-04-02T14:41:00.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:00 smithi204 ceph-mon[35948]: fsmap 7 up:standby 2024-04-02T14:41:00.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:00 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:41:00.519 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:00.519 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:00.519 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:00.519 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:00.520 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:00.520 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:00.520 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:00 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.204:6836/3537379638,v1:172.21.15.204:6837/3537379638] up:boot 2024-04-02T14:41:00.520 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:00 smithi113 ceph-mon[36135]: fsmap 7 up:standby 2024-04-02T14:41:00.520 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:00 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:41:01.370 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:41:01 smithi113 systemd[1]: Starting Ceph mds.h for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:41:01.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:01 smithi113 ceph-mon[36135]: Deploying daemon mds.h on smithi113 2024-04-02T14:41:01.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:01 smithi113 ceph-mon[36135]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:01.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:01 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/689674044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:01.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:01 smithi204 ceph-mon[35948]: Deploying daemon mds.h on smithi113 2024-04-02T14:41:01.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:01 smithi204 ceph-mon[35948]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:01.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:01 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/689674044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:01.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:01 smithi062 ceph-mon[32116]: Deploying daemon mds.h on smithi113 2024-04-02T14:41:01.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:01 smithi062 ceph-mon[32116]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:01.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:01 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/689674044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:01.791 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:41:01 smithi113 podman[66831]: 2024-04-02 14:41:01.498227328 +0000 UTC m=+0.102386829 container create 16a6adee0d7a7b21f3018c04cbb5339923c7d4a5d31e4d2cf9bb08fbf97c1db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-02T14:41:01.791 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:41:01 smithi113 podman[66831]: 2024-04-02 14:41:01.413885946 +0000 UTC m=+0.018045457 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:41:01.791 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:41:01 smithi113 podman[66831]: 2024-04-02 14:41:01.708834911 +0000 UTC m=+0.312994423 container init 16a6adee0d7a7b21f3018c04cbb5339923c7d4a5d31e4d2cf9bb08fbf97c1db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5) 2024-04-02T14:41:01.791 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:41:01 smithi113 podman[66831]: 2024-04-02 14:41:01.711552943 +0000 UTC m=+0.315712449 container start 16a6adee0d7a7b21f3018c04cbb5339923c7d4a5d31e4d2cf9bb08fbf97c1db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-02T14:41:01.791 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:41:01 smithi113 ceph-mds[66850]: -- 172.21.15.113:0/3296623915 <== mon.2 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56338291fa40 con 0x5633828f2800 2024-04-02T14:41:01.791 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:41:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: starting mds.h at 2024-04-02T14:41:01.792 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:41:01 smithi113 bash[66831]: 16a6adee0d7a7b21f3018c04cbb5339923c7d4a5d31e4d2cf9bb08fbf97c1db9 2024-04-02T14:41:02.120 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:41:01 smithi113 systemd[1]: Started Ceph mds.h for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:41:02.518 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:41:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.741+0000 7f689dc00640 1 -- 172.21.15.62:0/2813484319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68981025c0 msgr2=0x7f68981049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.741+0000 7f689dc00640 1 --2- 172.21.15.62:0/2813484319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68981025c0 0x7f68981049b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6888009900 tx=0x7f688802bc40 comp rx=0 tx=0).stop 2024-04-02T14:41:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.741+0000 7f689dc00640 1 -- 172.21.15.62:0/2813484319 shutdown_connections 2024-04-02T14:41:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.741+0000 7f689dc00640 1 --2- 172.21.15.62:0/2813484319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6898107820 0x7f6898109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.741+0000 7f689dc00640 1 --2- 172.21.15.62:0/2813484319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6898104ef0 0x7f68981072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.741+0000 7f689dc00640 1 --2- 172.21.15.62:0/2813484319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68981025c0 0x7f68981049b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.741+0000 7f689dc00640 1 -- 172.21.15.62:0/2813484319 >> 172.21.15.62:0/2813484319 conn(0x7f68980fbfb0 msgr2=0x7f68980fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.741+0000 7f689dc00640 1 -- 172.21.15.62:0/2813484319 shutdown_connections 2024-04-02T14:41:02.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.741+0000 7f689dc00640 1 -- 172.21.15.62:0/2813484319 wait complete. 2024-04-02T14:41:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.742+0000 7f689dc00640 1 Processor -- start 2024-04-02T14:41:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f689dc00640 1 -- start start 2024-04-02T14:41:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f689dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68981025c0 0x7f689819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f689dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6898104ef0 0x7f689819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f689dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6898107820 0x7f68981a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f689dc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f689810d910 con 0x7f6898104ef0 2024-04-02T14:41:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f689dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f689810d790 con 0x7f68981025c0 2024-04-02T14:41:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f689dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f689810da90 con 0x7f6898107820 2024-04-02T14:41:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f6896a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68981025c0 0x7f689819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f6896a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68981025c0 0x7f689819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:47154/0 (socket says 172.21.15.62:47154) 2024-04-02T14:41:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f6896a00640 1 -- 172.21.15.62:0/2178812883 learned_addr learned my addr 172.21.15.62:0/2178812883 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f688e000640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6898104ef0 0x7f689819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.743+0000 7f6897400640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6898107820 0x7f68981a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f6896a00640 1 -- 172.21.15.62:0/2178812883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6898107820 msgr2=0x7f68981a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f6896a00640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6898107820 0x7f68981a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f6896a00640 1 -- 172.21.15.62:0/2178812883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6898104ef0 msgr2=0x7f689819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f6896a00640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6898104ef0 0x7f689819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f6896a00640 1 -- 172.21.15.62:0/2178812883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68981a1d70 con 0x7f68981025c0 2024-04-02T14:41:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f6897400640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6898107820 0x7f68981a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f6896a00640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68981025c0 0x7f689819a1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f688802c250 tx=0x7f6888004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f688fe00640 1 -- 172.21.15.62:0/2178812883 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888043020 con 0x7f68981025c0 2024-04-02T14:41:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f688fe00640 1 -- 172.21.15.62:0/2178812883 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6888004250 con 0x7f68981025c0 2024-04-02T14:41:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f689dc00640 1 -- 172.21.15.62:0/2178812883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68981a2000 con 0x7f68981025c0 2024-04-02T14:41:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f688fe00640 1 -- 172.21.15.62:0/2178812883 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888003b50 con 0x7f68981025c0 2024-04-02T14:41:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.744+0000 7f689dc00640 1 -- 172.21.15.62:0/2178812883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68981a25a0 con 0x7f68981025c0 2024-04-02T14:41:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.745+0000 7f689dc00640 1 -- 172.21.15.62:0/2178812883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6898109370 con 0x7f68981025c0 2024-04-02T14:41:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.747+0000 7f688fe00640 1 -- 172.21.15.62:0/2178812883 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6888046020 con 0x7f68981025c0 2024-04-02T14:41:02.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.749+0000 7f688fe00640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f687c0a3c30 0x7f687c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:02.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.749+0000 7f688fe00640 1 -- 172.21.15.62:0/2178812883 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f68880f23d0 con 0x7f68981025c0 2024-04-02T14:41:02.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.749+0000 7f688e000640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f687c0a3c30 0x7f687c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:02.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.750+0000 7f688e000640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f687c0a3c30 0x7f687c0a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f689819b660 tx=0x7f6880005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:02.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:02.751+0000 7f688fe00640 1 -- 172.21.15.62:0/2178812883 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f688804d020 con 0x7f68981025c0 2024-04-02T14:41:03.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.001+0000 7f689dc00640 1 -- 172.21.15.62:0/2178812883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f689802d370 con 0x7f68981025c0 2024-04-02T14:41:03.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.002+0000 7f688fe00640 1 -- 172.21.15.62:0/2178812883 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7f68880bf1b0 con 0x7f68981025c0 2024-04-02T14:41:03.003 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:03.003 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-02T14:41:03.003 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 9 2024-04-02T14:41:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.005+0000 7f689dc00640 1 -- 172.21.15.62:0/2178812883 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f687c0a3c30 msgr2=0x7f687c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.005+0000 7f689dc00640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f687c0a3c30 0x7f687c0a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f689819b660 tx=0x7f6880005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:03.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.005+0000 7f689dc00640 1 -- 172.21.15.62:0/2178812883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68981025c0 msgr2=0x7f689819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:03.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.005+0000 7f689dc00640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68981025c0 0x7f689819a1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f688802c250 tx=0x7f6888004ab0 comp rx=0 tx=0).stop 2024-04-02T14:41:03.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.005+0000 7f689dc00640 1 -- 172.21.15.62:0/2178812883 shutdown_connections 2024-04-02T14:41:03.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.005+0000 7f689dc00640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f687c0a3c30 0x7f687c0a60f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:03.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.005+0000 7f689dc00640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6898107820 0x7f68981a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:03.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.005+0000 7f689dc00640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6898104ef0 0x7f689819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:03.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.005+0000 7f689dc00640 1 --2- 172.21.15.62:0/2178812883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68981025c0 0x7f689819a1c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:03.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.005+0000 7f689dc00640 1 -- 172.21.15.62:0/2178812883 >> 172.21.15.62:0/2178812883 conn(0x7f68980fbfb0 msgr2=0x7f6898101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:03.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.006+0000 7f689dc00640 1 -- 172.21.15.62:0/2178812883 shutdown_connections 2024-04-02T14:41:03.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:03.006+0000 7f689dc00640 1 -- 172.21.15.62:0/2178812883 wait complete. 2024-04-02T14:41:03.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:02 smithi062 ceph-mon[32116]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:03.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:03.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:03.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:03.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:03.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:03.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:03.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:02 smithi062 ceph-mon[32116]: Deploying daemon mds.i on smithi204 2024-04-02T14:41:03.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:02 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] up:boot 2024-04-02T14:41:03.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:02 smithi062 ceph-mon[32116]: fsmap 8 up:standby 2024-04-02T14:41:03.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:02 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:41:03.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:02 smithi113 ceph-mon[36135]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:03.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:03.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:03.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:03.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:03.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:03.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:03.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:02 smithi113 ceph-mon[36135]: Deploying daemon mds.i on smithi204 2024-04-02T14:41:03.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:02 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] up:boot 2024-04-02T14:41:03.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:02 smithi113 ceph-mon[36135]: fsmap 8 up:standby 2024-04-02T14:41:03.121 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:02 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:41:03.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:02 smithi204 ceph-mon[35948]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:03.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:03.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:03.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:03.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:03.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:03.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:03.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:02 smithi204 ceph-mon[35948]: Deploying daemon mds.i on smithi204 2024-04-02T14:41:03.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:02 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] up:boot 2024-04-02T14:41:03.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:02 smithi204 ceph-mon[35948]: fsmap 8 up:standby 2024-04-02T14:41:03.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:02 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:41:04.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:03 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2178812883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:04.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:03 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2178812883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:04.259 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:41:04 smithi204 systemd[1]: Starting Ceph mds.i for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:41:04.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:03 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2178812883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:04.575 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:41:04 smithi204 podman[67606]: 2024-04-02 14:41:04.226186441 +0000 UTC m=+0.018112962 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:41:04.575 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:41:04 smithi204 podman[67606]: 2024-04-02 14:41:04.348063084 +0000 UTC m=+0.139989603 container create 781f554886b89a4f3aadfae974e0221857f362f4e63050476e18e895421228ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:41:04.576 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:41:04 smithi204 podman[67606]: 2024-04-02 14:41:04.550794766 +0000 UTC m=+0.342721286 container init 781f554886b89a4f3aadfae974e0221857f362f4e63050476e18e895421228ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-02T14:41:04.576 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:41:04 smithi204 podman[67606]: 2024-04-02 14:41:04.553502563 +0000 UTC m=+0.345429083 container start 781f554886b89a4f3aadfae974e0221857f362f4e63050476e18e895421228ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-02T14:41:04.858 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:41:04 smithi204 ceph-mds[67633]: -- 172.21.15.204:0/1111736374 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dca3d57a40 con 0x55dca3d2b000 2024-04-02T14:41:04.858 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:41:04 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i[67624]: starting mds.i at 2024-04-02T14:41:04.859 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:41:04 smithi204 bash[67606]: 781f554886b89a4f3aadfae974e0221857f362f4e63050476e18e895421228ae 2024-04-02T14:41:04.859 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:41:04 smithi204 systemd[1]: Started Ceph mds.i for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:41:05.027 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:41:05.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:04 smithi062 ceph-mon[32116]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:05.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:05.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:05.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:05.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:05.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:05.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:04 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:05.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:04 smithi113 ceph-mon[36135]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:05.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:05.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:05.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:05.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:05.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:05.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:04 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:05.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.248+0000 7f2751400640 1 -- 172.21.15.62:0/3151592532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c104d00 msgr2=0x7f274c1070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:05.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.248+0000 7f2751400640 1 --2- 172.21.15.62:0/3151592532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c104d00 0x7f274c1070f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2734009900 tx=0x7f273402bc30 comp rx=0 tx=0).stop 2024-04-02T14:41:05.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.248+0000 7f2751400640 1 -- 172.21.15.62:0/3151592532 shutdown_connections 2024-04-02T14:41:05.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.248+0000 7f2751400640 1 --2- 172.21.15.62:0/3151592532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f274c107630 0x7f274c109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:05.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.248+0000 7f2751400640 1 --2- 172.21.15.62:0/3151592532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c104d00 0x7f274c1070f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:05.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.248+0000 7f2751400640 1 --2- 172.21.15.62:0/3151592532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c1023d0 0x7f274c1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:05.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.248+0000 7f2751400640 1 -- 172.21.15.62:0/3151592532 >> 172.21.15.62:0/3151592532 conn(0x7f274c0fbde0 msgr2=0x7f274c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:05.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.248+0000 7f2751400640 1 -- 172.21.15.62:0/3151592532 shutdown_connections 2024-04-02T14:41:05.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.248+0000 7f2751400640 1 -- 172.21.15.62:0/3151592532 wait complete. 2024-04-02T14:41:05.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.249+0000 7f2751400640 1 Processor -- start 2024-04-02T14:41:05.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.249+0000 7f2751400640 1 -- start start 2024-04-02T14:41:05.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.249+0000 7f2751400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c1023d0 0x7f274c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:05.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.249+0000 7f2751400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c104d00 0x7f274c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:05.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.249+0000 7f2751400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f274c107630 0x7f274c1a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:05.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.249+0000 7f2751400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f274c10d6b0 con 0x7f274c107630 2024-04-02T14:41:05.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.249+0000 7f2751400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f274c10d530 con 0x7f274c104d00 2024-04-02T14:41:05.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.249+0000 7f2751400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f274c10d830 con 0x7f274c1023d0 2024-04-02T14:41:05.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.249+0000 7f274be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c1023d0 0x7f274c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f274be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c1023d0 0x7f274c199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:51556/0 (socket says 172.21.15.62:51556) 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f2750a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f274c107630 0x7f274c1a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f274be00640 1 -- 172.21.15.62:0/124790116 learned_addr learned my addr 172.21.15.62:0/124790116 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f274b400640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c104d00 0x7f274c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f274be00640 1 -- 172.21.15.62:0/124790116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c104d00 msgr2=0x7f274c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f274be00640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c104d00 0x7f274c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f274be00640 1 -- 172.21.15.62:0/124790116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f274c107630 msgr2=0x7f274c1a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f274be00640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f274c107630 0x7f274c1a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f274be00640 1 -- 172.21.15.62:0/124790116 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c1a1b80 con 0x7f274c1023d0 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f274be00640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c1023d0 0x7f274c199f60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f273c00eb90 tx=0x7f273c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f2748c00640 1 -- 172.21.15.62:0/124790116 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c004d20 con 0x7f274c1023d0 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.250+0000 7f2748c00640 1 -- 172.21.15.62:0/124790116 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f273c00ee90 con 0x7f274c1023d0 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.251+0000 7f2751400640 1 -- 172.21.15.62:0/124790116 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c1a1e70 con 0x7f274c1023d0 2024-04-02T14:41:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.251+0000 7f2748c00640 1 -- 172.21.15.62:0/124790116 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c0108b0 con 0x7f274c1023d0 2024-04-02T14:41:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.251+0000 7f2751400640 1 -- 172.21.15.62:0/124790116 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c1a2280 con 0x7f274c1023d0 2024-04-02T14:41:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.252+0000 7f2751400640 1 -- 172.21.15.62:0/124790116 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c109180 con 0x7f274c1023d0 2024-04-02T14:41:05.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.254+0000 7f2748c00640 1 -- 172.21.15.62:0/124790116 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f273c010a50 con 0x7f274c1023d0 2024-04-02T14:41:05.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.255+0000 7f2748c00640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f27300a3bb0 0x7f27300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:05.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.255+0000 7f2748c00640 1 -- 172.21.15.62:0/124790116 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f273c0d05c0 con 0x7f274c1023d0 2024-04-02T14:41:05.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.255+0000 7f274b400640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f27300a3bb0 0x7f27300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:05.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.256+0000 7f274b400640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f27300a3bb0 0x7f27300a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2734000c00 tx=0x7f2734005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:05.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:04 smithi204 ceph-mon[35948]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:05.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:05.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:05.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:05.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:05.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:05.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:04 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.261+0000 7f2748c00640 1 -- 172.21.15.62:0/124790116 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f273c09d3a0 con 0x7f274c1023d0 2024-04-02T14:41:05.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.521+0000 7f2751400640 1 -- 172.21.15.62:0/124790116 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f274c02d5e0 con 0x7f274c1023d0 2024-04-02T14:41:05.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.522+0000 7f2748c00640 1 -- 172.21.15.62:0/124790116 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f273c09cca0 con 0x7f274c1023d0 2024-04-02T14:41:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:05.523 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-02T14:41:05.523 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 10 2024-04-02T14:41:05.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.524+0000 7f2751400640 1 -- 172.21.15.62:0/124790116 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f27300a3bb0 msgr2=0x7f27300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:05.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.524+0000 7f2751400640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f27300a3bb0 0x7f27300a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2734000c00 tx=0x7f2734005fd0 comp rx=0 tx=0).stop 2024-04-02T14:41:05.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.524+0000 7f2751400640 1 -- 172.21.15.62:0/124790116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c1023d0 msgr2=0x7f274c199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:05.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.524+0000 7f2751400640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c1023d0 0x7f274c199f60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f273c00eb90 tx=0x7f273c00c6a0 comp rx=0 tx=0).stop 2024-04-02T14:41:05.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.524+0000 7f2751400640 1 -- 172.21.15.62:0/124790116 shutdown_connections 2024-04-02T14:41:05.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.524+0000 7f2751400640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f27300a3bb0 0x7f27300a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:05.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.524+0000 7f2751400640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f274c107630 0x7f274c1a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:05.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.524+0000 7f2751400640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c104d00 0x7f274c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:05.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.524+0000 7f2751400640 1 --2- 172.21.15.62:0/124790116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c1023d0 0x7f274c199f60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:05.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.525+0000 7f2751400640 1 -- 172.21.15.62:0/124790116 >> 172.21.15.62:0/124790116 conn(0x7f274c0fbde0 msgr2=0x7f274c101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:05.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.525+0000 7f2751400640 1 -- 172.21.15.62:0/124790116 shutdown_connections 2024-04-02T14:41:05.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:05.525+0000 7f2751400640 1 -- 172.21.15.62:0/124790116 wait complete. 2024-04-02T14:41:06.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:05 smithi062 ceph-mon[32116]: Deploying daemon mds.g on smithi062 2024-04-02T14:41:06.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:05 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] up:boot 2024-04-02T14:41:06.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:05 smithi062 ceph-mon[32116]: fsmap 9 up:standby 2024-04-02T14:41:06.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:05 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:41:06.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:05 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/124790116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:06.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:05 smithi204 ceph-mon[35948]: Deploying daemon mds.g on smithi062 2024-04-02T14:41:06.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:05 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] up:boot 2024-04-02T14:41:06.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:05 smithi204 ceph-mon[35948]: fsmap 9 up:standby 2024-04-02T14:41:06.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:05 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:41:06.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:05 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/124790116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:06.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:05 smithi113 ceph-mon[36135]: Deploying daemon mds.g on smithi062 2024-04-02T14:41:06.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:05 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] up:boot 2024-04-02T14:41:06.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:05 smithi113 ceph-mon[36135]: fsmap 9 up:standby 2024-04-02T14:41:06.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:05 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:41:06.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:05 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/124790116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:07.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:06 smithi062 ceph-mon[32116]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:07.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:06 smithi204 ceph-mon[35948]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:07.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:06 smithi113 ceph-mon[36135]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:07.544 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:41:07.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.772+0000 7fad4b200640 1 -- 172.21.15.62:0/3476138247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad4c101a40 msgr2=0x7fad4c101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:07.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.772+0000 7fad4b200640 1 --2- 172.21.15.62:0/3476138247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad4c101a40 0x7fad4c101ec0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fad38009900 tx=0x7fad3802bc30 comp rx=0 tx=0).stop 2024-04-02T14:41:07.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.772+0000 7fad4b200640 1 -- 172.21.15.62:0/3476138247 shutdown_connections 2024-04-02T14:41:07.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.772+0000 7fad4b200640 1 --2- 172.21.15.62:0/3476138247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad4c102400 0x7fad4c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:07.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.772+0000 7fad4b200640 1 --2- 172.21.15.62:0/3476138247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad4c101a40 0x7fad4c101ec0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:07.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.772+0000 7fad4b200640 1 --2- 172.21.15.62:0/3476138247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad4c100850 0x7fad4c100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:07.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.772+0000 7fad4b200640 1 -- 172.21.15.62:0/3476138247 >> 172.21.15.62:0/3476138247 conn(0x7fad4c0fbde0 msgr2=0x7fad4c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:07.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.772+0000 7fad4b200640 1 -- 172.21.15.62:0/3476138247 shutdown_connections 2024-04-02T14:41:07.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.773+0000 7fad4b200640 1 -- 172.21.15.62:0/3476138247 wait complete. 2024-04-02T14:41:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.774+0000 7fad4b200640 1 Processor -- start 2024-04-02T14:41:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.774+0000 7fad4b200640 1 -- start start 2024-04-02T14:41:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.774+0000 7fad4b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad4c100850 0x7fad4c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.774+0000 7fad4b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad4c101a40 0x7fad4c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.774+0000 7fad4b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad4c102400 0x7fad4c1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.774+0000 7fad4b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10c5b0 con 0x7fad4c102400 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.774+0000 7fad4b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10c430 con 0x7fad4c100850 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.774+0000 7fad4b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10c730 con 0x7fad4c101a40 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad49e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad4c100850 0x7fad4c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad4a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad4c102400 0x7fad4c1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad49e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad4c100850 0x7fad4c199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:47188/0 (socket says 172.21.15.62:47188) 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad49e00640 1 -- 172.21.15.62:0/3986870223 learned_addr learned my addr 172.21.15.62:0/3986870223 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad4a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad4c102400 0x7fad4c1a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37728/0 (socket says 172.21.15.62:37728) 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad49400640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad4c101a40 0x7fad4c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad49e00640 1 -- 172.21.15.62:0/3986870223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad4c101a40 msgr2=0x7fad4c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad49e00640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad4c101a40 0x7fad4c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad49e00640 1 -- 172.21.15.62:0/3986870223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad4c102400 msgr2=0x7fad4c1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad49e00640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad4c102400 0x7fad4c1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:07.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad49e00640 1 -- 172.21.15.62:0/3986870223 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c1a1bd0 con 0x7fad4c100850 2024-04-02T14:41:07.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad49400640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad4c101a40 0x7fad4c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:41:07.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad4a800640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad4c102400 0x7fad4c1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:07.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.775+0000 7fad49e00640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad4c100850 0x7fad4c199fb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00e9c0 tx=0x7fad3c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:07.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.776+0000 7fad42a00640 1 -- 172.21.15.62:0/3986870223 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c00cde0 con 0x7fad4c100850 2024-04-02T14:41:07.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.776+0000 7fad42a00640 1 -- 172.21.15.62:0/3986870223 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad3c010930 con 0x7fad4c100850 2024-04-02T14:41:07.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.776+0000 7fad4b200640 1 -- 172.21.15.62:0/3986870223 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c1a1ec0 con 0x7fad4c100850 2024-04-02T14:41:07.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.776+0000 7fad42a00640 1 -- 172.21.15.62:0/3986870223 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c018a60 con 0x7fad4c100850 2024-04-02T14:41:07.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.776+0000 7fad4b200640 1 -- 172.21.15.62:0/3986870223 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c1a2320 con 0x7fad4c100850 2024-04-02T14:41:07.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.777+0000 7fad4b200640 1 -- 172.21.15.62:0/3986870223 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c1a2550 con 0x7fad4c100850 2024-04-02T14:41:07.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.779+0000 7fad42a00640 1 -- 172.21.15.62:0/3986870223 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fad3c01f020 con 0x7fad4c100850 2024-04-02T14:41:07.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.780+0000 7fad42a00640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fad300a3bb0 0x7fad300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:07.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.780+0000 7fad42a00640 1 -- 172.21.15.62:0/3986870223 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fad3c0d0450 con 0x7fad4c100850 2024-04-02T14:41:07.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.780+0000 7fad49400640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fad300a3bb0 0x7fad300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:07.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.781+0000 7fad49400640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fad300a3bb0 0x7fad300a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad4c19b4c0 tx=0x7fad3802c9d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:07.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:07.783+0000 7fad42a00640 1 -- 172.21.15.62:0/3986870223 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fad3c09d230 con 0x7fad4c100850 2024-04-02T14:41:08.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.035+0000 7fad4b200640 1 -- 172.21.15.62:0/3986870223 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad4c1a2720 con 0x7fad4c100850 2024-04-02T14:41:08.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.036+0000 7fad42a00640 1 -- 172.21.15.62:0/3986870223 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7fad3c09cb30 con 0x7fad4c100850 2024-04-02T14:41:08.036 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:08.036 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-02T14:41:08.037 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 10 2024-04-02T14:41:08.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.038+0000 7fad4b200640 1 -- 172.21.15.62:0/3986870223 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fad300a3bb0 msgr2=0x7fad300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:08.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.038+0000 7fad4b200640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fad300a3bb0 0x7fad300a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad4c19b4c0 tx=0x7fad3802c9d0 comp rx=0 tx=0).stop 2024-04-02T14:41:08.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.038+0000 7fad4b200640 1 -- 172.21.15.62:0/3986870223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad4c100850 msgr2=0x7fad4c199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:08.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.038+0000 7fad4b200640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad4c100850 0x7fad4c199fb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00e9c0 tx=0x7fad3c00ee80 comp rx=0 tx=0).stop 2024-04-02T14:41:08.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.039+0000 7fad4b200640 1 -- 172.21.15.62:0/3986870223 shutdown_connections 2024-04-02T14:41:08.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.039+0000 7fad4b200640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fad300a3bb0 0x7fad300a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:08.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.039+0000 7fad4b200640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad4c102400 0x7fad4c1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:08.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.039+0000 7fad4b200640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad4c101a40 0x7fad4c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:08.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.039+0000 7fad4b200640 1 --2- 172.21.15.62:0/3986870223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad4c100850 0x7fad4c199fb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:08.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.039+0000 7fad4b200640 1 -- 172.21.15.62:0/3986870223 >> 172.21.15.62:0/3986870223 conn(0x7fad4c0fbde0 msgr2=0x7fad4c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:08.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.039+0000 7fad4b200640 1 -- 172.21.15.62:0/3986870223 shutdown_connections 2024-04-02T14:41:08.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:08.039+0000 7fad4b200640 1 -- 172.21.15.62:0/3986870223 wait complete. 2024-04-02T14:41:08.897 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:41:08 smithi062 systemd[1]: Started Ceph mds.g for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:41:09.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:08 smithi204 ceph-mon[35948]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:09.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:08 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3986870223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:09.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:08 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:09.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:08 smithi062 ceph-mon[32116]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:09.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:08 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3986870223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:09.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:08 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:09.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:08 smithi113 ceph-mon[36135]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:09.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:08 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3986870223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:09.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:08 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:10.058 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:41:10.155 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:10.155 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:09 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] up:boot 2024-04-02T14:41:10.155 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:09 smithi062 ceph-mon[32116]: fsmap 10 up:standby 2024-04-02T14:41:10.155 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:41:10.155 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:10.156 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:10.156 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:10.156 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:09 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:10.156 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:09 smithi062 ceph-mon[32116]: Deploying daemon mds.a on smithi062 2024-04-02T14:41:10.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:10.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:09 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] up:boot 2024-04-02T14:41:10.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:09 smithi204 ceph-mon[35948]: fsmap 10 up:standby 2024-04-02T14:41:10.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:41:10.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:10.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:10.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:10.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:09 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:10.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:09 smithi204 ceph-mon[35948]: Deploying daemon mds.a on smithi062 2024-04-02T14:41:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.290+0000 7f922fe00640 1 -- 172.21.15.62:0/2002060037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9230107f10 msgr2=0x7f9230108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.290+0000 7f922fe00640 1 --2- 172.21.15.62:0/2002060037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9230107f10 0x7f9230108350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9220009900 tx=0x7f922002bc50 comp rx=0 tx=0).stop 2024-04-02T14:41:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.291+0000 7f922fe00640 1 -- 172.21.15.62:0/2002060037 shutdown_connections 2024-04-02T14:41:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.291+0000 7f922fe00640 1 --2- 172.21.15.62:0/2002060037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230109aa0 0x7f92301103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:10.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.291+0000 7f922fe00640 1 --2- 172.21.15.62:0/2002060037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92301090e0 0x7f9230109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:10.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.291+0000 7f922fe00640 1 --2- 172.21.15.62:0/2002060037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9230107f10 0x7f9230108350 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:10.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.291+0000 7f922fe00640 1 -- 172.21.15.62:0/2002060037 >> 172.21.15.62:0/2002060037 conn(0x7f9230076f10 msgr2=0x7f9230079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:10.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.291+0000 7f922fe00640 1 -- 172.21.15.62:0/2002060037 shutdown_connections 2024-04-02T14:41:10.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.291+0000 7f922fe00640 1 -- 172.21.15.62:0/2002060037 wait complete. 2024-04-02T14:41:10.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.292+0000 7f922fe00640 1 Processor -- start 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.292+0000 7f922fe00640 1 -- start start 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.292+0000 7f922fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230107f10 0x7f923019e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92301090e0 0x7f923019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230109aa0 0x7f92301a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9230113d00 con 0x7f92301090e0 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9230113b80 con 0x7f9230109aa0 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9230113e80 con 0x7f9230107f10 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f9227e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92301090e0 0x7f923019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f9227e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92301090e0 0x7f923019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37762/0 (socket says 172.21.15.62:37762) 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f9227e00640 1 -- 172.21.15.62:0/1573484105 learned_addr learned my addr 172.21.15.62:0/1573484105 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922ea00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230107f10 0x7f923019e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922f400640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230109aa0 0x7f92301a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922ea00640 1 -- 172.21.15.62:0/1573484105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230109aa0 msgr2=0x7f92301a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:10.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922ea00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230109aa0 0x7f92301a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:10.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922ea00640 1 -- 172.21.15.62:0/1573484105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92301090e0 msgr2=0x7f923019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:10.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922ea00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92301090e0 0x7f923019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:10.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f922ea00640 1 -- 172.21.15.62:0/1573484105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92301a5f60 con 0x7f9230107f10 2024-04-02T14:41:10.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.293+0000 7f9227e00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92301090e0 0x7f923019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.294+0000 7f922ea00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230107f10 0x7f923019e340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f922002c190 tx=0x7f9220004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.294+0000 7f9234a00640 1 -- 172.21.15.62:0/1573484105 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220043020 con 0x7f9230107f10 2024-04-02T14:41:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.294+0000 7f9234a00640 1 -- 172.21.15.62:0/1573484105 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9220004250 con 0x7f9230107f10 2024-04-02T14:41:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.294+0000 7f922fe00640 1 -- 172.21.15.62:0/1573484105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92301a61f0 con 0x7f9230107f10 2024-04-02T14:41:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.294+0000 7f9234a00640 1 -- 172.21.15.62:0/1573484105 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220003c30 con 0x7f9230107f10 2024-04-02T14:41:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.294+0000 7f922fe00640 1 -- 172.21.15.62:0/1573484105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92301a6760 con 0x7f9230107f10 2024-04-02T14:41:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.295+0000 7f922fe00640 1 -- 172.21.15.62:0/1573484105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923010f320 con 0x7f9230107f10 2024-04-02T14:41:10.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.299+0000 7f9234a00640 1 -- 172.21.15.62:0/1573484105 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9220046020 con 0x7f9230107f10 2024-04-02T14:41:10.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.300+0000 7f9234a00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 0x7f92140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:10.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.300+0000 7f9234a00640 1 -- 172.21.15.62:0/1573484105 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f92200f1ff0 con 0x7f9230107f10 2024-04-02T14:41:10.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.301+0000 7f9227e00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 0x7f92140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:10.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.301+0000 7f9227e00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 0x7f92140a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f923019f850 tx=0x7f921800b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.305+0000 7f9234a00640 1 -- 172.21.15.62:0/1573484105 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f922004d020 con 0x7f9230107f10 2024-04-02T14:41:10.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:10.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:09 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] up:boot 2024-04-02T14:41:10.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:09 smithi113 ceph-mon[36135]: fsmap 10 up:standby 2024-04-02T14:41:10.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:41:10.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:10.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:10.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:10.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:09 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:10.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:09 smithi113 ceph-mon[36135]: Deploying daemon mds.a on smithi062 2024-04-02T14:41:10.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.575+0000 7f922fe00640 1 -- 172.21.15.62:0/1573484105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f923002d370 con 0x7f9230107f10 2024-04-02T14:41:10.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.576+0000 7f9234a00640 1 -- 172.21.15.62:0/1573484105 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f92200bedf0 con 0x7f9230107f10 2024-04-02T14:41:10.576 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:10.576 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-02T14:41:10.577 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 11 2024-04-02T14:41:10.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 -- 172.21.15.62:0/1573484105 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 msgr2=0x7f92140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:10.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 0x7f92140a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f923019f850 tx=0x7f921800b4a0 comp rx=0 tx=0).stop 2024-04-02T14:41:10.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 -- 172.21.15.62:0/1573484105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230107f10 msgr2=0x7f923019e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:10.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230107f10 0x7f923019e340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f922002c190 tx=0x7f9220004ab0 comp rx=0 tx=0).stop 2024-04-02T14:41:10.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 -- 172.21.15.62:0/1573484105 shutdown_connections 2024-04-02T14:41:10.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 0x7f92140a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:10.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230109aa0 0x7f92301a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:10.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92301090e0 0x7f923019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:10.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 --2- 172.21.15.62:0/1573484105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230107f10 0x7f923019e340 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:10.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 -- 172.21.15.62:0/1573484105 >> 172.21.15.62:0/1573484105 conn(0x7f9230076f10 msgr2=0x7f923010c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:10.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 -- 172.21.15.62:0/1573484105 shutdown_connections 2024-04-02T14:41:10.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:10.579+0000 7f922fe00640 1 -- 172.21.15.62:0/1573484105 wait complete. 2024-04-02T14:41:11.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:10 smithi062 ceph-mon[32116]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:11.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:10 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1573484105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:11.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:10 smithi204 ceph-mon[35948]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:11.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:10 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1573484105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:11.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:10 smithi113 ceph-mon[36135]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:11.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:10 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1573484105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:12.343 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:41:12 smithi062 systemd[1]: Starting Ceph mds.a for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:41:12.599 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:41:12.684 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:41:12 smithi062 podman[72281]: 2024-04-02 14:41:12.336442714 +0000 UTC m=+0.036922320 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:41:12.684 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:41:12 smithi062 podman[72281]: 2024-04-02 14:41:12.441283541 +0000 UTC m=+0.141763170 container create e5aee3247ea0b3a05590c9d5b6147383f343464fe8ba7e531779bcbe427fd953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:41:12.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.828+0000 7f58a7a00640 1 -- 172.21.15.62:0/2597125960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a01036d0 msgr2=0x7f58a0109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.828+0000 7f58a7a00640 1 --2- 172.21.15.62:0/2597125960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a01036d0 0x7f58a0109f60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5898009be0 tx=0x7f589802bef0 comp rx=0 tx=0).stop 2024-04-02T14:41:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.828+0000 7f58a7a00640 1 -- 172.21.15.62:0/2597125960 shutdown_connections 2024-04-02T14:41:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.828+0000 7f58a7a00640 1 --2- 172.21.15.62:0/2597125960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a01036d0 0x7f58a0109f60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.828+0000 7f58a7a00640 1 --2- 172.21.15.62:0/2597125960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58a0102d10 0x7f58a0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.828+0000 7f58a7a00640 1 --2- 172.21.15.62:0/2597125960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a0101b20 0x7f58a0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.828+0000 7f58a7a00640 1 -- 172.21.15.62:0/2597125960 >> 172.21.15.62:0/2597125960 conn(0x7f58a00fd0b0 msgr2=0x7f58a00ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.828+0000 7f58a7a00640 1 -- 172.21.15.62:0/2597125960 shutdown_connections 2024-04-02T14:41:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.828+0000 7f58a7a00640 1 -- 172.21.15.62:0/2597125960 wait complete. 2024-04-02T14:41:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.829+0000 7f58a7a00640 1 Processor -- start 2024-04-02T14:41:12.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.829+0000 7f58a7a00640 1 -- start start 2024-04-02T14:41:12.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.829+0000 7f58a7a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a0101b20 0x7f58a00758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:12.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.829+0000 7f58a7a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a0102d10 0x7f58a0073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:12.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.829+0000 7f58a7a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58a01036d0 0x7f58a0074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.829+0000 7f58a7a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f58a010d910 con 0x7f58a0101b20 2024-04-02T14:41:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.829+0000 7f58a7a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f58a010d790 con 0x7f58a01036d0 2024-04-02T14:41:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.829+0000 7f58a7a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f58a010da90 con 0x7f58a0102d10 2024-04-02T14:41:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.830+0000 7f58a5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a0101b20 0x7f58a00758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.830+0000 7f58a5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a0101b20 0x7f58a00758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52432/0 (socket says 172.21.15.62:52432) 2024-04-02T14:41:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.830+0000 7f58a5000640 1 -- 172.21.15.62:0/972391653 learned_addr learned my addr 172.21.15.62:0/972391653 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.830+0000 7f58a5a00640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58a01036d0 0x7f58a0074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.830+0000 7f589fe00640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a0102d10 0x7f58a0073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.830+0000 7f58a5000640 1 -- 172.21.15.62:0/972391653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a0102d10 msgr2=0x7f58a0073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.830+0000 7f58a5000640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a0102d10 0x7f58a0073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.830+0000 7f58a5000640 1 -- 172.21.15.62:0/972391653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58a01036d0 msgr2=0x7f58a0074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.830+0000 7f58a5000640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58a01036d0 0x7f58a0074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.830+0000 7f58a5000640 1 -- 172.21.15.62:0/972391653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a0074cc0 con 0x7f58a0101b20 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.830+0000 7f58a5a00640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58a01036d0 0x7f58a0074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.831+0000 7f58a5000640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a0101b20 0x7f58a00758b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f589400e9a0 tx=0x7f589400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.831+0000 7f589d600640 1 -- 172.21.15.62:0/972391653 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589400cd80 con 0x7f58a0101b20 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.831+0000 7f589d600640 1 -- 172.21.15.62:0/972391653 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5894010930 con 0x7f58a0101b20 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.831+0000 7f589d600640 1 -- 172.21.15.62:0/972391653 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894018a70 con 0x7f58a0101b20 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.831+0000 7f58a7a00640 1 -- 172.21.15.62:0/972391653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a01a1fb0 con 0x7f58a0101b20 2024-04-02T14:41:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.831+0000 7f58a7a00640 1 -- 172.21.15.62:0/972391653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a01a23c0 con 0x7f58a0101b20 2024-04-02T14:41:12.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.832+0000 7f58a7a00640 1 -- 172.21.15.62:0/972391653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a0108d70 con 0x7f58a0101b20 2024-04-02T14:41:12.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.835+0000 7f589d600640 1 -- 172.21.15.62:0/972391653 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5894018c10 con 0x7f58a0101b20 2024-04-02T14:41:12.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.837+0000 7f589d600640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58880a3c30 0x7f58880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:12.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.837+0000 7f589d600640 1 -- 172.21.15.62:0/972391653 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f58940d0180 con 0x7f58a0101b20 2024-04-02T14:41:12.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.837+0000 7f589fe00640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58880a3c30 0x7f58880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:12.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.838+0000 7f589fe00640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58880a3c30 0x7f58880a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f588c0095f0 tx=0x7f588c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:12.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:12.841+0000 7f589d600640 1 -- 172.21.15.62:0/972391653 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f589409c910 con 0x7f58a0101b20 2024-04-02T14:41:12.974 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:41:12 smithi062 podman[72281]: 2024-04-02 14:41:12.703405047 +0000 UTC m=+0.403884692 container init e5aee3247ea0b3a05590c9d5b6147383f343464fe8ba7e531779bcbe427fd953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326) 2024-04-02T14:41:12.974 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:41:12 smithi062 podman[72281]: 2024-04-02 14:41:12.712821962 +0000 UTC m=+0.413301560 container start e5aee3247ea0b3a05590c9d5b6147383f343464fe8ba7e531779bcbe427fd953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:41:12.974 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:41:12 smithi062 ceph-mds[72320]: -- 172.21.15.62:0/3687757471 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561dfc7b7a40 con 0x561dfc78a800 2024-04-02T14:41:12.974 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:41:12 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: starting mds.a at 2024-04-02T14:41:12.974 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:41:12 smithi062 bash[72281]: e5aee3247ea0b3a05590c9d5b6147383f343464fe8ba7e531779bcbe427fd953 2024-04-02T14:41:12.974 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:41:12 smithi062 systemd[1]: Started Ceph mds.a for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:41:13.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.108+0000 7f58a7a00640 1 -- 172.21.15.62:0/972391653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58a0061f40 con 0x7f58a0101b20 2024-04-02T14:41:13.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.110+0000 7f589d600640 1 -- 172.21.15.62:0/972391653 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f58940a0c10 con 0x7f58a0101b20 2024-04-02T14:41:13.112 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:13.112 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24565,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-04-02T14:41:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 12 2024-04-02T14:41:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 -- 172.21.15.62:0/972391653 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58880a3c30 msgr2=0x7f58880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58880a3c30 0x7f58880a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f588c0095f0 tx=0x7f588c005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 -- 172.21.15.62:0/972391653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a0101b20 msgr2=0x7f58a00758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a0101b20 0x7f58a00758b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f589400e9a0 tx=0x7f589400ee60 comp rx=0 tx=0).stop 2024-04-02T14:41:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 -- 172.21.15.62:0/972391653 shutdown_connections 2024-04-02T14:41:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58880a3c30 0x7f58880a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58a01036d0 0x7f58a0074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a0102d10 0x7f58a0073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 --2- 172.21.15.62:0/972391653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a0101b20 0x7f58a00758b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 -- 172.21.15.62:0/972391653 >> 172.21.15.62:0/972391653 conn(0x7f58a00fd0b0 msgr2=0x7f58a0105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 -- 172.21.15.62:0/972391653 shutdown_connections 2024-04-02T14:41:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:13.113+0000 7f58a7a00640 1 -- 172.21.15.62:0/972391653 wait complete. 2024-04-02T14:41:13.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:12 smithi204 ceph-mon[35948]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:13.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:13.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:13.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:13.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:13.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:13.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:12 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:13.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:12 smithi062 ceph-mon[32116]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:13.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:13.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:13.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:13.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:13.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:13.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:12 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:13.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:12 smithi113 ceph-mon[36135]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:13.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:13.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:13.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:13.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T14:41:13.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T14:41:13.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:12 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:14.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:13 smithi113 ceph-mon[36135]: Deploying daemon mds.b on smithi113 2024-04-02T14:41:14.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:13 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] up:boot 2024-04-02T14:41:14.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:13 smithi113 ceph-mon[36135]: fsmap 11 up:standby 2024-04-02T14:41:14.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:13 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:41:14.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:13 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/972391653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:14.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:13 smithi204 ceph-mon[35948]: Deploying daemon mds.b on smithi113 2024-04-02T14:41:14.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:13 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] up:boot 2024-04-02T14:41:14.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:13 smithi204 ceph-mon[35948]: fsmap 11 up:standby 2024-04-02T14:41:14.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:13 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:41:14.259 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:13 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/972391653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:14.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:13 smithi062 ceph-mon[32116]: Deploying daemon mds.b on smithi113 2024-04-02T14:41:14.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:13 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] up:boot 2024-04-02T14:41:14.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:13 smithi062 ceph-mon[32116]: fsmap 11 up:standby 2024-04-02T14:41:14.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:13 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:41:14.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:13 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/972391653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:15.120 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:14 smithi113 ceph-mon[36135]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:15.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:41:15.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:14 smithi062 ceph-mon[32116]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:15.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:14 smithi204 ceph-mon[35948]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:15.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.384+0000 7f9240e00640 1 -- 172.21.15.62:0/1157816518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f923c109aa0 msgr2=0x7f923c1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:15.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.384+0000 7f9240e00640 1 --2- 172.21.15.62:0/1157816518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f923c109aa0 0x7f923c1103c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9234009ca0 tx=0x7f923402c100 comp rx=0 tx=0).stop 2024-04-02T14:41:15.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.384+0000 7f9240e00640 1 -- 172.21.15.62:0/1157816518 shutdown_connections 2024-04-02T14:41:15.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.384+0000 7f9240e00640 1 --2- 172.21.15.62:0/1157816518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f923c109aa0 0x7f923c1103c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:15.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.384+0000 7f9240e00640 1 --2- 172.21.15.62:0/1157816518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f923c1090e0 0x7f923c109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:15.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.384+0000 7f9240e00640 1 --2- 172.21.15.62:0/1157816518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f923c107f10 0x7f923c108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:15.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.384+0000 7f9240e00640 1 -- 172.21.15.62:0/1157816518 >> 172.21.15.62:0/1157816518 conn(0x7f923c076f10 msgr2=0x7f923c079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:15.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.384+0000 7f9240e00640 1 -- 172.21.15.62:0/1157816518 shutdown_connections 2024-04-02T14:41:15.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.384+0000 7f9240e00640 1 -- 172.21.15.62:0/1157816518 wait complete. 2024-04-02T14:41:15.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.385+0000 7f9240e00640 1 Processor -- start 2024-04-02T14:41:15.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.385+0000 7f9240e00640 1 -- start start 2024-04-02T14:41:15.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.385+0000 7f9240e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f923c107f10 0x7f923c19e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:15.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.385+0000 7f9240e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f923c1090e0 0x7f923c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.385+0000 7f9240e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f923c109aa0 0x7f923c1a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.385+0000 7f9240e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f923c113d20 con 0x7f923c109aa0 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.385+0000 7f9240e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f923c113ba0 con 0x7f923c1090e0 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.385+0000 7f9240e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f923c113ea0 con 0x7f923c107f10 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f923c109aa0 0x7f923c1a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f923c107f10 0x7f923c19e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f923c1090e0 0x7f923c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f923c107f10 0x7f923c19e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:43918/0 (socket says 172.21.15.62:43918) 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f923c1090e0 0x7f923c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41568/0 (socket says 172.21.15.62:41568) 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923b400640 1 -- 172.21.15.62:0/1320510987 learned_addr learned my addr 172.21.15.62:0/1320510987 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923be00640 1 -- 172.21.15.62:0/1320510987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f923c107f10 msgr2=0x7f923c19e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923be00640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f923c107f10 0x7f923c19e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923be00640 1 -- 172.21.15.62:0/1320510987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f923c1090e0 msgr2=0x7f923c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:15.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923be00640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f923c1090e0 0x7f923c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:15.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923be00640 1 -- 172.21.15.62:0/1320510987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923c1a5ef0 con 0x7f923c109aa0 2024-04-02T14:41:15.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.386+0000 7f923aa00640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f923c1090e0 0x7f923c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:15.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.387+0000 7f923be00640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f923c109aa0 0x7f923c1a5880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f923402c640 tx=0x7f92340346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:15.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.387+0000 7f9233e00640 1 -- 172.21.15.62:0/1320510987 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234039040 con 0x7f923c109aa0 2024-04-02T14:41:15.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.387+0000 7f9233e00640 1 -- 172.21.15.62:0/1320510987 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9234044070 con 0x7f923c109aa0 2024-04-02T14:41:15.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.387+0000 7f9233e00640 1 -- 172.21.15.62:0/1320510987 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92340044d0 con 0x7f923c109aa0 2024-04-02T14:41:15.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.387+0000 7f9240e00640 1 -- 172.21.15.62:0/1320510987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923c1a6180 con 0x7f923c109aa0 2024-04-02T14:41:15.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.387+0000 7f9240e00640 1 -- 172.21.15.62:0/1320510987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f923c1a65f0 con 0x7f923c109aa0 2024-04-02T14:41:15.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.388+0000 7f9240e00640 1 -- 172.21.15.62:0/1320510987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923c10f2a0 con 0x7f923c109aa0 2024-04-02T14:41:15.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.390+0000 7f9233e00640 1 -- 172.21.15.62:0/1320510987 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9234004670 con 0x7f923c109aa0 2024-04-02T14:41:15.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.391+0000 7f9233e00640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92200a3bb0 0x7f92200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:15.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.392+0000 7f9233e00640 1 -- 172.21.15.62:0/1320510987 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f92340f2550 con 0x7f923c109aa0 2024-04-02T14:41:15.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.392+0000 7f923b400640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92200a3bb0 0x7f92200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:15.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.392+0000 7f923b400640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92200a3bb0 0x7f92200a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f922c0095f0 tx=0x7f922c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:15.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.395+0000 7f9233e00640 1 -- 172.21.15.62:0/1320510987 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f92340bf330 con 0x7f923c109aa0 2024-04-02T14:41:15.514 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:41:15 smithi113 systemd[1]: Started Ceph mds.b for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4. 2024-04-02T14:41:15.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.649+0000 7f9240e00640 1 -- 172.21.15.62:0/1320510987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f923c02d370 con 0x7f923c109aa0 2024-04-02T14:41:15.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.650+0000 7f9233e00640 1 -- 172.21.15.62:0/1320510987 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f92340bec30 con 0x7f923c109aa0 2024-04-02T14:41:15.651 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:15.651 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24565,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-04-02T14:41:15.651 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 12 2024-04-02T14:41:15.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 -- 172.21.15.62:0/1320510987 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92200a3bb0 msgr2=0x7f92200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:15.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92200a3bb0 0x7f92200a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f922c0095f0 tx=0x7f922c005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:15.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 -- 172.21.15.62:0/1320510987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f923c109aa0 msgr2=0x7f923c1a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:15.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f923c109aa0 0x7f923c1a5880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f923402c640 tx=0x7f92340346f0 comp rx=0 tx=0).stop 2024-04-02T14:41:15.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 -- 172.21.15.62:0/1320510987 shutdown_connections 2024-04-02T14:41:15.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92200a3bb0 0x7f92200a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:15.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f923c109aa0 0x7f923c1a5880 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:15.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f923c1090e0 0x7f923c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:15.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 --2- 172.21.15.62:0/1320510987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f923c107f10 0x7f923c19e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:15.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 -- 172.21.15.62:0/1320510987 >> 172.21.15.62:0/1320510987 conn(0x7f923c076f10 msgr2=0x7f923c10c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:15.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 -- 172.21.15.62:0/1320510987 shutdown_connections 2024-04-02T14:41:15.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:15.653+0000 7f9240e00640 1 -- 172.21.15.62:0/1320510987 wait complete. 2024-04-02T14:41:16.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:16 smithi204 ceph-mon[35948]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:16.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:16 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:16 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:16 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:16 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:16 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:16 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1320510987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:16.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:16 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:41:16.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:16 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] up:boot 2024-04-02T14:41:16.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:16 smithi204 ceph-mon[35948]: fsmap 12 up:standby 2024-04-02T14:41:16.760 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:16 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:41:16.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:16 smithi062 ceph-mon[32116]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:16.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:16 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:16 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:16 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:16 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:16 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:16 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1320510987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:16.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:16 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:41:16.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:16 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] up:boot 2024-04-02T14:41:16.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:16 smithi062 ceph-mon[32116]: fsmap 12 up:standby 2024-04-02T14:41:16.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:16 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:41:16.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:16 smithi113 ceph-mon[36135]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:16.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:16 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:16 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:16 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:16 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:16 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:16.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:16 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1320510987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:16.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:16 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:41:16.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:16 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] up:boot 2024-04-02T14:41:16.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:16 smithi113 ceph-mon[36135]: fsmap 12 up:standby 2024-04-02T14:41:16.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:16 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:41:17.673 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:41:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.900+0000 7fcc7c400640 1 -- 172.21.15.62:0/207985373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc74075ae0 msgr2=0x7fcc74073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.900+0000 7fcc7c400640 1 --2- 172.21.15.62:0/207985373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc74075ae0 0x7fcc74073f30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009860 tx=0x7fcc6802bc20 comp rx=0 tx=0).stop 2024-04-02T14:41:17.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.900+0000 7fcc7c400640 1 -- 172.21.15.62:0/207985373 shutdown_connections 2024-04-02T14:41:17.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.900+0000 7fcc7c400640 1 --2- 172.21.15.62:0/207985373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc741099b0 0x7fcc7410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:17.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.900+0000 7fcc7c400640 1 --2- 172.21.15.62:0/207985373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc74074500 0x7fcc74074960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:17.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.900+0000 7fcc7c400640 1 --2- 172.21.15.62:0/207985373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc74075ae0 0x7fcc74073f30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:17.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.900+0000 7fcc7c400640 1 -- 172.21.15.62:0/207985373 >> 172.21.15.62:0/207985373 conn(0x7fcc740fd0b0 msgr2=0x7fcc740ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:17.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.900+0000 7fcc7c400640 1 -- 172.21.15.62:0/207985373 shutdown_connections 2024-04-02T14:41:17.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.900+0000 7fcc7c400640 1 -- 172.21.15.62:0/207985373 wait complete. 2024-04-02T14:41:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.901+0000 7fcc7c400640 1 Processor -- start 2024-04-02T14:41:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.901+0000 7fcc7c400640 1 -- start start 2024-04-02T14:41:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.902+0000 7fcc7c400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc74074500 0x7fcc7419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.902+0000 7fcc7c400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc74075ae0 0x7fcc7419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.902+0000 7fcc7c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc741099b0 0x7fcc741a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.902+0000 7fcc7c400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7410fad0 con 0x7fcc74074500 2024-04-02T14:41:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.902+0000 7fcc7c400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7410f950 con 0x7fcc74075ae0 2024-04-02T14:41:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.902+0000 7fcc7c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7410fc50 con 0x7fcc741099b0 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.902+0000 7fcc79a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc74074500 0x7fcc7419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.902+0000 7fcc79000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc74075ae0 0x7fcc7419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.902+0000 7fcc79000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc74075ae0 0x7fcc7419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41594/0 (socket says 172.21.15.62:41594) 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.902+0000 7fcc79000640 1 -- 172.21.15.62:0/427445153 learned_addr learned my addr 172.21.15.62:0/427445153 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.902+0000 7fcc7a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc741099b0 0x7fcc741a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.903+0000 7fcc79000640 1 -- 172.21.15.62:0/427445153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc741099b0 msgr2=0x7fcc741a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.903+0000 7fcc79000640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc741099b0 0x7fcc741a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.903+0000 7fcc79000640 1 -- 172.21.15.62:0/427445153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc74074500 msgr2=0x7fcc7419a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.903+0000 7fcc79000640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc74074500 0x7fcc7419a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.903+0000 7fcc79000640 1 -- 172.21.15.62:0/427445153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc741a1d10 con 0x7fcc74075ae0 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.903+0000 7fcc7a400640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc741099b0 0x7fcc741a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.903+0000 7fcc79000640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc74075ae0 0x7fcc7419a630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000e9c0 tx=0x7fcc6000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.903+0000 7fcc72000640 1 -- 172.21.15.62:0/427445153 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000cd80 con 0x7fcc74075ae0 2024-04-02T14:41:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.903+0000 7fcc7c400640 1 -- 172.21.15.62:0/427445153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc741a1fa0 con 0x7fcc74075ae0 2024-04-02T14:41:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.903+0000 7fcc72000640 1 -- 172.21.15.62:0/427445153 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc600035f0 con 0x7fcc74075ae0 2024-04-02T14:41:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.904+0000 7fcc72000640 1 -- 172.21.15.62:0/427445153 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60010690 con 0x7fcc74075ae0 2024-04-02T14:41:17.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.904+0000 7fcc7c400640 1 -- 172.21.15.62:0/427445153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc741a24c0 con 0x7fcc74075ae0 2024-04-02T14:41:17.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.905+0000 7fcc7c400640 1 -- 172.21.15.62:0/427445153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc74108770 con 0x7fcc74075ae0 2024-04-02T14:41:17.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.906+0000 7fcc72000640 1 -- 172.21.15.62:0/427445153 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcc600037a0 con 0x7fcc74075ae0 2024-04-02T14:41:17.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.908+0000 7fcc72000640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcc5c0a3bb0 0x7fcc5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:17.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.908+0000 7fcc72000640 1 -- 172.21.15.62:0/427445153 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fcc600d0260 con 0x7fcc74075ae0 2024-04-02T14:41:17.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.908+0000 7fcc79a00640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcc5c0a3bb0 0x7fcc5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:17.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.909+0000 7fcc79a00640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcc5c0a3bb0 0x7fcc5c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009830 tx=0x7fcc68034bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:17.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:17.912+0000 7fcc72000640 1 -- 172.21.15.62:0/427445153 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcc6009d040 con 0x7fcc74075ae0 2024-04-02T14:41:18.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.187+0000 7fcc7c400640 1 -- 172.21.15.62:0/427445153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc74062700 con 0x7fcc74075ae0 2024-04-02T14:41:18.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.188+0000 7fcc72000640 1 -- 172.21.15.62:0/427445153 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7fcc6009c940 con 0x7fcc74075ae0 2024-04-02T14:41:18.187 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:18.187 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24565,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-02T14:41:18.188 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 13 2024-04-02T14:41:18.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.190+0000 7fcc7c400640 1 -- 172.21.15.62:0/427445153 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcc5c0a3bb0 msgr2=0x7fcc5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:18.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.190+0000 7fcc7c400640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcc5c0a3bb0 0x7fcc5c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009830 tx=0x7fcc68034bb0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.190+0000 7fcc7c400640 1 -- 172.21.15.62:0/427445153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc74075ae0 msgr2=0x7fcc7419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:18.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.190+0000 7fcc7c400640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc74075ae0 0x7fcc7419a630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000e9c0 tx=0x7fcc6000edc0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.191+0000 7fcc7c400640 1 -- 172.21.15.62:0/427445153 shutdown_connections 2024-04-02T14:41:18.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.191+0000 7fcc7c400640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcc5c0a3bb0 0x7fcc5c0a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.191+0000 7fcc7c400640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc741099b0 0x7fcc741a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.191+0000 7fcc7c400640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc74075ae0 0x7fcc7419a630 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.191+0000 7fcc7c400640 1 --2- 172.21.15.62:0/427445153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc74074500 0x7fcc7419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.191+0000 7fcc7c400640 1 -- 172.21.15.62:0/427445153 >> 172.21.15.62:0/427445153 conn(0x7fcc740fd0b0 msgr2=0x7fcc74105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:18.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.191+0000 7fcc7c400640 1 -- 172.21.15.62:0/427445153 shutdown_connections 2024-04-02T14:41:18.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.191+0000 7fcc7c400640 1 -- 172.21.15.62:0/427445153 wait complete. 2024-04-02T14:41:18.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:41:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.404+0000 7f1fc3a00640 1 -- 172.21.15.62:0/4126906800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fbc1099d0 msgr2=0x7f1fbc10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.404+0000 7f1fc3a00640 1 --2- 172.21.15.62:0/4126906800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fbc1099d0 0x7f1fbc10be60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009ca0 tx=0x7f1fb402c180 comp rx=0 tx=0).stop 2024-04-02T14:41:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.404+0000 7f1fc3a00640 1 -- 172.21.15.62:0/4126906800 shutdown_connections 2024-04-02T14:41:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.404+0000 7f1fc3a00640 1 --2- 172.21.15.62:0/4126906800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fbc1099d0 0x7f1fbc10be60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.404+0000 7f1fc3a00640 1 --2- 172.21.15.62:0/4126906800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fbc0744e0 0x7f1fbc074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.404+0000 7f1fc3a00640 1 --2- 172.21.15.62:0/4126906800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc075ac0 0x7f1fbc073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.404+0000 7f1fc3a00640 1 -- 172.21.15.62:0/4126906800 >> 172.21.15.62:0/4126906800 conn(0x7f1fbc0fd0b0 msgr2=0x7f1fbc0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.404+0000 7f1fc3a00640 1 -- 172.21.15.62:0/4126906800 shutdown_connections 2024-04-02T14:41:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.404+0000 7f1fc3a00640 1 -- 172.21.15.62:0/4126906800 wait complete. 2024-04-02T14:41:18.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.405+0000 7f1fc3a00640 1 Processor -- start 2024-04-02T14:41:18.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fc3a00640 1 -- start start 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fc3a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fbc0744e0 0x7f1fbc19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fc3a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fbc075ac0 0x7f1fbc19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fc3a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc1099d0 0x7f1fbc1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fc3a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc10fae0 con 0x7f1fbc075ac0 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fc3a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc10f960 con 0x7f1fbc0744e0 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fc3a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc10fc60 con 0x7f1fbc1099d0 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fbbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fbc075ac0 0x7f1fbc19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fbbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fbc075ac0 0x7f1fbc19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52482/0 (socket says 172.21.15.62:52482) 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fbbe00640 1 -- 172.21.15.62:0/70987108 learned_addr learned my addr 172.21.15.62:0/70987108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fc1000640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fbc0744e0 0x7f1fbc19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.406+0000 7f1fc1a00640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc1099d0 0x7f1fbc1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.407+0000 7f1fbbe00640 1 -- 172.21.15.62:0/70987108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc1099d0 msgr2=0x7f1fbc1a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.407+0000 7f1fbbe00640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc1099d0 0x7f1fbc1a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.407+0000 7f1fbbe00640 1 -- 172.21.15.62:0/70987108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fbc0744e0 msgr2=0x7f1fbc19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.407+0000 7f1fbbe00640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fbc0744e0 0x7f1fbc19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.407+0000 7f1fbbe00640 1 -- 172.21.15.62:0/70987108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc1a1cb0 con 0x7f1fbc075ac0 2024-04-02T14:41:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.407+0000 7f1fbbe00640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fbc075ac0 0x7f1fbc19a5d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800e9f0 tx=0x7f1fa800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.407+0000 7f1fb9600640 1 -- 172.21.15.62:0/70987108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa800cd80 con 0x7f1fbc075ac0 2024-04-02T14:41:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.407+0000 7f1fb9600640 1 -- 172.21.15.62:0/70987108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1fa8010930 con 0x7f1fbc075ac0 2024-04-02T14:41:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.407+0000 7f1fc3a00640 1 -- 172.21.15.62:0/70987108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc1a1fa0 con 0x7f1fbc075ac0 2024-04-02T14:41:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.407+0000 7f1fb9600640 1 -- 172.21.15.62:0/70987108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8018b00 con 0x7f1fbc075ac0 2024-04-02T14:41:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.407+0000 7f1fc3a00640 1 -- 172.21.15.62:0/70987108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc1a24b0 con 0x7f1fbc075ac0 2024-04-02T14:41:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.409+0000 7f1fc3a00640 1 -- 172.21.15.62:0/70987108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fbc108790 con 0x7f1fbc075ac0 2024-04-02T14:41:18.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.411+0000 7f1fb9600640 1 -- 172.21.15.62:0/70987108 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1fa8018ca0 con 0x7f1fbc075ac0 2024-04-02T14:41:18.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.413+0000 7f1fb9600640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1fa40a3c30 0x7f1fa40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:18.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.413+0000 7f1fb9600640 1 -- 172.21.15.62:0/70987108 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f1fa80d02c0 con 0x7f1fbc075ac0 2024-04-02T14:41:18.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.413+0000 7f1fc1000640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1fa40a3c30 0x7f1fa40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:18.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.414+0000 7f1fc1000640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1fa40a3c30 0x7f1fa40a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009bd0 tx=0x7f1fb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:18.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.420+0000 7f1fb9600640 1 -- 172.21.15.62:0/70987108 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1fa809ca50 con 0x7f1fbc075ac0 2024-04-02T14:41:18.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:18 smithi062 ceph-mon[32116]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:18.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:18 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:18 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:18 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:18 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:18 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/427445153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:18.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.652+0000 7f1fc3a00640 1 -- 172.21.15.62:0/70987108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fbc0627a0 con 0x7f1fbc075ac0 2024-04-02T14:41:18.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.653+0000 7f1fb9600640 1 -- 172.21.15.62:0/70987108 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18434 (secure 0 0 0) 0x7f1fa80a0d50 con 0x7f1fbc075ac0 2024-04-02T14:41:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:18.653 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":84,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:40:25.221043+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:41:18.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 -- 172.21.15.62:0/70987108 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1fa40a3c30 msgr2=0x7f1fa40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:18.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1fa40a3c30 0x7f1fa40a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009bd0 tx=0x7f1fb0005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:18.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 -- 172.21.15.62:0/70987108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fbc075ac0 msgr2=0x7f1fbc19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:18.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fbc075ac0 0x7f1fbc19a5d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800e9f0 tx=0x7f1fa800eeb0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 -- 172.21.15.62:0/70987108 shutdown_connections 2024-04-02T14:41:18.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1fa40a3c30 0x7f1fa40a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc1099d0 0x7f1fbc1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fbc075ac0 0x7f1fbc19a5d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 --2- 172.21.15.62:0/70987108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fbc0744e0 0x7f1fbc19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 -- 172.21.15.62:0/70987108 >> 172.21.15.62:0/70987108 conn(0x7f1fbc0fd0b0 msgr2=0x7f1fbc105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:18.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 -- 172.21.15.62:0/70987108 shutdown_connections 2024-04-02T14:41:18.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.656+0000 7f1fc3a00640 1 -- 172.21.15.62:0/70987108 wait complete. 2024-04-02T14:41:18.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:41:18.676 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:41:18.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:18 smithi204 ceph-mon[35948]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:18.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:18 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:18 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:18 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:18 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:18 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/427445153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:18.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:18 smithi113 ceph-mon[36135]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:18.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:18 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:18 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:18 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:18 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:18.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:18 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/427445153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:18.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.895+0000 7f5bba400640 1 -- 172.21.15.62:0/3530331472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb4109330 msgr2=0x7f5bb41097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:18.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.895+0000 7f5bba400640 1 --2- 172.21.15.62:0/3530331472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb4109330 0x7f5bb41097b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009900 tx=0x7f5b9c02bc40 comp rx=0 tx=0).stop 2024-04-02T14:41:18.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.895+0000 7f5bba400640 1 -- 172.21.15.62:0/3530331472 shutdown_connections 2024-04-02T14:41:18.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.895+0000 7f5bba400640 1 --2- 172.21.15.62:0/3530331472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4109cf0 0x7f5bb4110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.895+0000 7f5bba400640 1 --2- 172.21.15.62:0/3530331472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb4109330 0x7f5bb41097b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.895+0000 7f5bba400640 1 --2- 172.21.15.62:0/3530331472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bb4108140 0x7f5bb4108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.895+0000 7f5bba400640 1 -- 172.21.15.62:0/3530331472 >> 172.21.15.62:0/3530331472 conn(0x7f5bb40770f0 msgr2=0x7f5bb4079510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:18.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.895+0000 7f5bba400640 1 -- 172.21.15.62:0/3530331472 shutdown_connections 2024-04-02T14:41:18.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.895+0000 7f5bba400640 1 -- 172.21.15.62:0/3530331472 wait complete. 2024-04-02T14:41:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.896+0000 7f5bba400640 1 Processor -- start 2024-04-02T14:41:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.896+0000 7f5bba400640 1 -- start start 2024-04-02T14:41:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.896+0000 7f5bba400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4108140 0x7f5bb419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.896+0000 7f5bba400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb4109330 0x7f5bb419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.896+0000 7f5bba400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bb4109cf0 0x7f5bb41a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.896+0000 7f5bba400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4113ef0 con 0x7f5bb4109330 2024-04-02T14:41:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.896+0000 7f5bba400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4113d70 con 0x7f5bb4109cf0 2024-04-02T14:41:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.896+0000 7f5bba400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4114070 con 0x7f5bb4108140 2024-04-02T14:41:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.897+0000 7f5babe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb4109330 0x7f5bb419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.897+0000 7f5babe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb4109330 0x7f5bb419eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52500/0 (socket says 172.21.15.62:52500) 2024-04-02T14:41:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.897+0000 7f5babe00640 1 -- 172.21.15.62:0/1063365776 learned_addr learned my addr 172.21.15.62:0/1063365776 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.897+0000 7f5bb3e00640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bb4109cf0 0x7f5bb41a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.897+0000 7f5bb3400640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4108140 0x7f5bb419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.897+0000 7f5babe00640 1 -- 172.21.15.62:0/1063365776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4108140 msgr2=0x7f5bb419e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.897+0000 7f5babe00640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4108140 0x7f5bb419e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.897+0000 7f5babe00640 1 -- 172.21.15.62:0/1063365776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bb4109cf0 msgr2=0x7f5bb41a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.897+0000 7f5babe00640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bb4109cf0 0x7f5bb41a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.897+0000 7f5babe00640 1 -- 172.21.15.62:0/1063365776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb41a6180 con 0x7f5bb4109330 2024-04-02T14:41:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.897+0000 7f5babe00640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb4109330 0x7f5bb419eaa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009230 tx=0x7f5b9c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.898+0000 7f5bb0c00640 1 -- 172.21.15.62:0/1063365776 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c043020 con 0x7f5bb4109330 2024-04-02T14:41:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.898+0000 7f5bb0c00640 1 -- 172.21.15.62:0/1063365776 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b9c004250 con 0x7f5bb4109330 2024-04-02T14:41:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.898+0000 7f5bb0c00640 1 -- 172.21.15.62:0/1063365776 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c003b50 con 0x7f5bb4109330 2024-04-02T14:41:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.898+0000 7f5bba400640 1 -- 172.21.15.62:0/1063365776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb41a6410 con 0x7f5bb4109330 2024-04-02T14:41:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.898+0000 7f5bba400640 1 -- 172.21.15.62:0/1063365776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb41a6960 con 0x7f5bb4109330 2024-04-02T14:41:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.899+0000 7f5bba400640 1 -- 172.21.15.62:0/1063365776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb410f390 con 0x7f5bb4109330 2024-04-02T14:41:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.901+0000 7f5bb0c00640 1 -- 172.21.15.62:0/1063365776 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b9c046020 con 0x7f5bb4109330 2024-04-02T14:41:18.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.902+0000 7f5bb0c00640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5b980a3bb0 0x7f5b980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:18.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.902+0000 7f5bb0c00640 1 -- 172.21.15.62:0/1063365776 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f5b9c0f20d0 con 0x7f5bb4109330 2024-04-02T14:41:18.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.903+0000 7f5bb3400640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5b980a3bb0 0x7f5b980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:18.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.903+0000 7f5bb3400640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5b980a3bb0 0x7f5b980a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4109190 tx=0x7f5ba4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:18.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:18.907+0000 7f5bb0c00640 1 -- 172.21.15.62:0/1063365776 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5b9c04d020 con 0x7f5bb4109330 2024-04-02T14:41:19.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.117+0000 7f5bba400640 1 -- 172.21.15.62:0/1063365776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5bb4062700 con 0x7f5bb4109330 2024-04-02T14:41:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.117+0000 7f5bb0c00640 1 -- 172.21.15.62:0/1063365776 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f5b9c0beeb0 con 0x7f5bb4109330 2024-04-02T14:41:19.116 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:41:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 -- 172.21.15.62:0/1063365776 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5b980a3bb0 msgr2=0x7f5b980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5b980a3bb0 0x7f5b980a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4109190 tx=0x7f5ba4005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 -- 172.21.15.62:0/1063365776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb4109330 msgr2=0x7f5bb419eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb4109330 0x7f5bb419eaa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009230 tx=0x7f5b9c004ab0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 -- 172.21.15.62:0/1063365776 shutdown_connections 2024-04-02T14:41:19.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5b980a3bb0 0x7f5b980a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bb4109cf0 0x7f5bb41a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb4109330 0x7f5bb419eaa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 --2- 172.21.15.62:0/1063365776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4108140 0x7f5bb419e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 -- 172.21.15.62:0/1063365776 >> 172.21.15.62:0/1063365776 conn(0x7f5bb40770f0 msgr2=0x7f5bb410c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:19.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 -- 172.21.15.62:0/1063365776 shutdown_connections 2024-04-02T14:41:19.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.120+0000 7f5bba400640 1 -- 172.21.15.62:0/1063365776 wait complete. 2024-04-02T14:41:19.136 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-02T14:41:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.328+0000 7fb94dc00640 1 -- 172.21.15.62:0/3657548651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9481036d0 msgr2=0x7fb948109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.328+0000 7fb94dc00640 1 --2- 172.21.15.62:0/3657548651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9481036d0 0x7fb948109f60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb940009ca0 tx=0x7fb94002c100 comp rx=0 tx=0).stop 2024-04-02T14:41:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.328+0000 7fb94dc00640 1 -- 172.21.15.62:0/3657548651 shutdown_connections 2024-04-02T14:41:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.328+0000 7fb94dc00640 1 --2- 172.21.15.62:0/3657548651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9481036d0 0x7fb948109f60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.328+0000 7fb94dc00640 1 --2- 172.21.15.62:0/3657548651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb948102d10 0x7fb948103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.328+0000 7fb94dc00640 1 --2- 172.21.15.62:0/3657548651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb948101b20 0x7fb948101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.328+0000 7fb94dc00640 1 -- 172.21.15.62:0/3657548651 >> 172.21.15.62:0/3657548651 conn(0x7fb9480fd0b0 msgr2=0x7fb9480ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.328+0000 7fb94dc00640 1 -- 172.21.15.62:0/3657548651 shutdown_connections 2024-04-02T14:41:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.328+0000 7fb94dc00640 1 -- 172.21.15.62:0/3657548651 wait complete. 2024-04-02T14:41:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.329+0000 7fb94dc00640 1 Processor -- start 2024-04-02T14:41:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.329+0000 7fb94dc00640 1 -- start start 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb94dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb948101b20 0x7fb9480758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb94dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb948102d10 0x7fb948073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb94dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9481036d0 0x7fb948074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb94dc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810d8d0 con 0x7fb9481036d0 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb94dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810d750 con 0x7fb948102d10 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb94dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810da50 con 0x7fb948101b20 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb946000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb948102d10 0x7fb948073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb946a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb948101b20 0x7fb9480758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb946000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb948102d10 0x7fb948073ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41654/0 (socket says 172.21.15.62:41654) 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb946000640 1 -- 172.21.15.62:0/1847786952 learned_addr learned my addr 172.21.15.62:0/1847786952 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb947400640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9481036d0 0x7fb948074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb946000640 1 -- 172.21.15.62:0/1847786952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb948101b20 msgr2=0x7fb9480758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb946000640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb948101b20 0x7fb9480758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb946000640 1 -- 172.21.15.62:0/1847786952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9481036d0 msgr2=0x7fb948074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:19.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb946000640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9481036d0 0x7fb948074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.330+0000 7fb946000640 1 -- 172.21.15.62:0/1847786952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb948074cc0 con 0x7fb948102d10 2024-04-02T14:41:19.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.331+0000 7fb946000640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb948102d10 0x7fb948073ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb93400e9f0 tx=0x7fb93400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:19.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.331+0000 7fb93f400640 1 -- 172.21.15.62:0/1847786952 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93400cd80 con 0x7fb948102d10 2024-04-02T14:41:19.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.331+0000 7fb93f400640 1 -- 172.21.15.62:0/1847786952 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb934005520 con 0x7fb948102d10 2024-04-02T14:41:19.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.331+0000 7fb94dc00640 1 -- 172.21.15.62:0/1847786952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9481a1fa0 con 0x7fb948102d10 2024-04-02T14:41:19.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.331+0000 7fb93f400640 1 -- 172.21.15.62:0/1847786952 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934010690 con 0x7fb948102d10 2024-04-02T14:41:19.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.331+0000 7fb94dc00640 1 -- 172.21.15.62:0/1847786952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9481a2460 con 0x7fb948102d10 2024-04-02T14:41:19.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.333+0000 7fb94dc00640 1 -- 172.21.15.62:0/1847786952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb948108de0 con 0x7fb948102d10 2024-04-02T14:41:19.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.335+0000 7fb93f400640 1 -- 172.21.15.62:0/1847786952 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb9340046c0 con 0x7fb948102d10 2024-04-02T14:41:19.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.336+0000 7fb93f400640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb92c0a3c30 0x7fb92c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:19.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.336+0000 7fb93f400640 1 -- 172.21.15.62:0/1847786952 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb9340d03b0 con 0x7fb948102d10 2024-04-02T14:41:19.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.337+0000 7fb946a00640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb92c0a3c30 0x7fb92c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:19.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.337+0000 7fb946a00640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb92c0a3c30 0x7fb92c0a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb948102b70 tx=0x7fb93800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:19.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.340+0000 7fb93f400640 1 -- 172.21.15.62:0/1847786952 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb93409d190 con 0x7fb948102d10 2024-04-02T14:41:19.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.553+0000 7fb94dc00640 1 -- 172.21.15.62:0/1847786952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb948062700 con 0x7fb948102d10 2024-04-02T14:41:19.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.554+0000 7fb93f400640 1 -- 172.21.15.62:0/1847786952 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7fb9340bd0d0 con 0x7fb948102d10 2024-04-02T14:41:19.553 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:19.553 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-02T14:41:19.553 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:41:19.553 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-02T14:41:19.553 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-02T14:41:19.553 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:41:19.553 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-02T14:41:19.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.556+0000 7fb94dc00640 1 -- 172.21.15.62:0/1847786952 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb92c0a3c30 msgr2=0x7fb92c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:19.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.556+0000 7fb94dc00640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb92c0a3c30 0x7fb92c0a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb948102b70 tx=0x7fb93800b390 comp rx=0 tx=0).stop 2024-04-02T14:41:19.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.557+0000 7fb94dc00640 1 -- 172.21.15.62:0/1847786952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb948102d10 msgr2=0x7fb948073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:19.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.557+0000 7fb94dc00640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb948102d10 0x7fb948073ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb93400e9f0 tx=0x7fb93400edf0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.557+0000 7fb94dc00640 1 -- 172.21.15.62:0/1847786952 shutdown_connections 2024-04-02T14:41:19.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.557+0000 7fb94dc00640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb92c0a3c30 0x7fb92c0a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.557+0000 7fb94dc00640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9481036d0 0x7fb948074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.557+0000 7fb94dc00640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb948102d10 0x7fb948073ef0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.557+0000 7fb94dc00640 1 --2- 172.21.15.62:0/1847786952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb948101b20 0x7fb9480758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.557+0000 7fb94dc00640 1 -- 172.21.15.62:0/1847786952 >> 172.21.15.62:0/1847786952 conn(0x7fb9480fd0b0 msgr2=0x7fb948105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:19.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.557+0000 7fb94dc00640 1 -- 172.21.15.62:0/1847786952 shutdown_connections 2024-04-02T14:41:19.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.557+0000 7fb94dc00640 1 -- 172.21.15.62:0/1847786952 wait complete. 2024-04-02T14:41:19.575 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-02T14:41:19.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-02T14:41:19.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:19 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/70987108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:19.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:19.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:19.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:19.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:41:19.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:19.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:41:19.756 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:19 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1063365776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:41:19.756 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:19 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:19.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:19 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/70987108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:19.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:19.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:19.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:19.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:41:19.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:19.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:41:19.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:19 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1063365776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:41:19.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:19 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.782+0000 7f2709200640 1 -- 172.21.15.62:0/3411010623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704074560 msgr2=0x7f27040749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.782+0000 7f2709200640 1 --2- 172.21.15.62:0/3411010623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704074560 0x7f27040749e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009890 tx=0x7f26f002bb70 comp rx=0 tx=0).stop 2024-04-02T14:41:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.782+0000 7f2709200640 1 -- 172.21.15.62:0/3411010623 shutdown_connections 2024-04-02T14:41:19.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.782+0000 7f2709200640 1 --2- 172.21.15.62:0/3411010623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f270410e0e0 0x7f27041104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.782+0000 7f2709200640 1 --2- 172.21.15.62:0/3411010623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704074560 0x7f27040749e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.782+0000 7f2709200640 1 --2- 172.21.15.62:0/3411010623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2704075b40 0x7f2704073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.782+0000 7f2709200640 1 -- 172.21.15.62:0/3411010623 >> 172.21.15.62:0/3411010623 conn(0x7f27040fd0a0 msgr2=0x7f27040ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:19.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.782+0000 7f2709200640 1 -- 172.21.15.62:0/3411010623 shutdown_connections 2024-04-02T14:41:19.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.782+0000 7f2709200640 1 -- 172.21.15.62:0/3411010623 wait complete. 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.783+0000 7f2709200640 1 Processor -- start 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.783+0000 7f2709200640 1 -- start start 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.783+0000 7f2709200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2704074560 0x7f270419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.783+0000 7f2709200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704075b40 0x7f270419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.783+0000 7f2709200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f270410e0e0 0x7f27041a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.783+0000 7f2709200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2704113f20 con 0x7f270410e0e0 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.783+0000 7f2709200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2704113da0 con 0x7f2704075b40 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.783+0000 7f2709200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27041140a0 con 0x7f2704074560 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.784+0000 7f2701400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704075b40 0x7f270419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.784+0000 7f2702800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f270410e0e0 0x7f27041a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.784+0000 7f2701400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704075b40 0x7f270419eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41660/0 (socket says 172.21.15.62:41660) 2024-04-02T14:41:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.784+0000 7f2701400640 1 -- 172.21.15.62:0/3298062552 learned_addr learned my addr 172.21.15.62:0/3298062552 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.784+0000 7f2701e00640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2704074560 0x7f270419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.784+0000 7f2701400640 1 -- 172.21.15.62:0/3298062552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2704074560 msgr2=0x7f270419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.784+0000 7f2701400640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2704074560 0x7f270419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.784+0000 7f2701400640 1 -- 172.21.15.62:0/3298062552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f270410e0e0 msgr2=0x7f27041a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.784+0000 7f2701400640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f270410e0e0 0x7f27041a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.784+0000 7f2701400640 1 -- 172.21.15.62:0/3298062552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27041a61a0 con 0x7f2704075b40 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.785+0000 7f2701400640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704075b40 0x7f270419eac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f26f002c180 tx=0x7f26f0035f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.785+0000 7f26faa00640 1 -- 172.21.15.62:0/3298062552 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0041020 con 0x7f2704075b40 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.785+0000 7f26faa00640 1 -- 172.21.15.62:0/3298062552 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f26f002ca10 con 0x7f2704075b40 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.785+0000 7f2709200640 1 -- 172.21.15.62:0/3298062552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27041a6430 con 0x7f2704075b40 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.785+0000 7f26faa00640 1 -- 172.21.15.62:0/3298062552 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0039040 con 0x7f2704075b40 2024-04-02T14:41:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.785+0000 7f2709200640 1 -- 172.21.15.62:0/3298062552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27041a69a0 con 0x7f2704075b40 2024-04-02T14:41:19.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.786+0000 7f2709200640 1 -- 172.21.15.62:0/3298062552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27041041a0 con 0x7f2704075b40 2024-04-02T14:41:19.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.788+0000 7f26faa00640 1 -- 172.21.15.62:0/3298062552 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f26f0044020 con 0x7f2704075b40 2024-04-02T14:41:19.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.789+0000 7f26faa00640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26e80a3bb0 0x7f26e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:19.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.789+0000 7f26faa00640 1 -- 172.21.15.62:0/3298062552 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f26f00f1960 con 0x7f2704075b40 2024-04-02T14:41:19.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.789+0000 7f2701e00640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26e80a3bb0 0x7f26e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:19.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.790+0000 7f2701e00640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26e80a3bb0 0x7f26e80a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005fd0 tx=0x7f26f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:19.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:19.793+0000 7f26faa00640 1 -- 172.21.15.62:0/3298062552 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f26f004b020 con 0x7f2704075b40 2024-04-02T14:41:19.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:19 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/70987108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:19.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:19 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:19.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:19 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:19.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:19 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:19.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:19 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:41:19.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:19 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:19.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:19 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:41:19.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:19 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1063365776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:41:19.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:19 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:20.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.020+0000 7f2709200640 1 -- 172.21.15.62:0/3298062552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f270402d370 con 0x7f2704075b40 2024-04-02T14:41:20.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.453+0000 7f26faa00640 1 -- 172.21.15.62:0/3298062552 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7f26f00be740 con 0x7f2704075b40 2024-04-02T14:41:20.453 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-02T14:41:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.460+0000 7f2709200640 1 -- 172.21.15.62:0/3298062552 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26e80a3bb0 msgr2=0x7f26e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.460+0000 7f2709200640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26e80a3bb0 0x7f26e80a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005fd0 tx=0x7f26f4009260 comp rx=0 tx=0).stop 2024-04-02T14:41:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.460+0000 7f2709200640 1 -- 172.21.15.62:0/3298062552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704075b40 msgr2=0x7f270419eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.460+0000 7f2709200640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704075b40 0x7f270419eac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f26f002c180 tx=0x7f26f0035f10 comp rx=0 tx=0).stop 2024-04-02T14:41:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.460+0000 7f2709200640 1 -- 172.21.15.62:0/3298062552 shutdown_connections 2024-04-02T14:41:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.460+0000 7f2709200640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26e80a3bb0 0x7f26e80a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.460+0000 7f2709200640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f270410e0e0 0x7f27041a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.460+0000 7f2709200640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704075b40 0x7f270419eac0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f26f002c180 tx=0x7f26f0035f10 comp rx=0 tx=0).stop 2024-04-02T14:41:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.460+0000 7f2709200640 1 --2- 172.21.15.62:0/3298062552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2704074560 0x7f270419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.460+0000 7f2709200640 1 -- 172.21.15.62:0/3298062552 >> 172.21.15.62:0/3298062552 conn(0x7f27040fd0a0 msgr2=0x7f2704101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.461+0000 7f2709200640 1 -- 172.21.15.62:0/3298062552 shutdown_connections 2024-04-02T14:41:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.462+0000 7f2709200640 1 -- 172.21.15.62:0/3298062552 wait complete. 2024-04-02T14:41:20.481 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-02T14:41:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.680+0000 7fb723e00640 1 -- 172.21.15.62:0/2554244113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71c104eb0 msgr2=0x7fb71c1072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.680+0000 7fb723e00640 1 --2- 172.21.15.62:0/2554244113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71c104eb0 0x7fb71c1072a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb708009900 tx=0x7fb70802bc30 comp rx=0 tx=0).stop 2024-04-02T14:41:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.680+0000 7fb723e00640 1 -- 172.21.15.62:0/2554244113 shutdown_connections 2024-04-02T14:41:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.680+0000 7fb723e00640 1 --2- 172.21.15.62:0/2554244113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb71c1077e0 0x7fb71c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.680+0000 7fb723e00640 1 --2- 172.21.15.62:0/2554244113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71c104eb0 0x7fb71c1072a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.680+0000 7fb723e00640 1 --2- 172.21.15.62:0/2554244113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c102580 0x7fb71c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:20.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.680+0000 7fb723e00640 1 -- 172.21.15.62:0/2554244113 >> 172.21.15.62:0/2554244113 conn(0x7fb71c0fbf90 msgr2=0x7fb71c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:20.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.681+0000 7fb723e00640 1 -- 172.21.15.62:0/2554244113 shutdown_connections 2024-04-02T14:41:20.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.681+0000 7fb723e00640 1 -- 172.21.15.62:0/2554244113 wait complete. 2024-04-02T14:41:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.681+0000 7fb723e00640 1 Processor -- start 2024-04-02T14:41:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb723e00640 1 -- start start 2024-04-02T14:41:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb723e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb71c102580 0x7fb71c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb723e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71c104eb0 0x7fb71c196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb723e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c1077e0 0x7fb71c193420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb723e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c10d860 con 0x7fb71c102580 2024-04-02T14:41:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb723e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c10d6e0 con 0x7fb71c1077e0 2024-04-02T14:41:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb723e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c10d9e0 con 0x7fb71c104eb0 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb721400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb71c102580 0x7fb71c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb720a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71c104eb0 0x7fb71c196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb720a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71c104eb0 0x7fb71c196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:44032/0 (socket says 172.21.15.62:44032) 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb720a00640 1 -- 172.21.15.62:0/3653548063 learned_addr learned my addr 172.21.15.62:0/3653548063 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.682+0000 7fb721e00640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c1077e0 0x7fb71c193420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.683+0000 7fb720a00640 1 -- 172.21.15.62:0/3653548063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c1077e0 msgr2=0x7fb71c193420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.683+0000 7fb720a00640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c1077e0 0x7fb71c193420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.683+0000 7fb720a00640 1 -- 172.21.15.62:0/3653548063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb71c102580 msgr2=0x7fb71c1968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.683+0000 7fb720a00640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb71c102580 0x7fb71c1968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.683+0000 7fb720a00640 1 -- 172.21.15.62:0/3653548063 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c193ce0 con 0x7fb71c104eb0 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.683+0000 7fb721e00640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c1077e0 0x7fb71c193420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.683+0000 7fb720a00640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71c104eb0 0x7fb71c196df0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb708000c00 tx=0x7fb70802c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.683+0000 7fb71a000640 1 -- 172.21.15.62:0/3653548063 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708043020 con 0x7fb71c104eb0 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.683+0000 7fb723e00640 1 -- 172.21.15.62:0/3653548063 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c1a65f0 con 0x7fb71c104eb0 2024-04-02T14:41:20.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.683+0000 7fb71a000640 1 -- 172.21.15.62:0/3653548063 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb7080042d0 con 0x7fb71c104eb0 2024-04-02T14:41:20.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.683+0000 7fb71a000640 1 -- 172.21.15.62:0/3653548063 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708003cc0 con 0x7fb71c104eb0 2024-04-02T14:41:20.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.684+0000 7fb723e00640 1 -- 172.21.15.62:0/3653548063 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c1a69f0 con 0x7fb71c104eb0 2024-04-02T14:41:20.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.685+0000 7fb723e00640 1 -- 172.21.15.62:0/3653548063 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c0688f0 con 0x7fb71c104eb0 2024-04-02T14:41:20.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.686+0000 7fb71a000640 1 -- 172.21.15.62:0/3653548063 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb708046020 con 0x7fb71c104eb0 2024-04-02T14:41:20.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.688+0000 7fb71a000640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb7040a3bb0 0x7fb7040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:20.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.689+0000 7fb71a000640 1 -- 172.21.15.62:0/3653548063 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7fb7080f2990 con 0x7fb71c104eb0 2024-04-02T14:41:20.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.689+0000 7fb721400640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb7040a3bb0 0x7fb7040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:20.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.689+0000 7fb721400640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb7040a3bb0 0x7fb7040a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb710009bd0 tx=0x7fb710005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:20.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.690+0000 7fb71a000640 1 -- 172.21.15.62:0/3653548063 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb70804d020 con 0x7fb71c104eb0 2024-04-02T14:41:20.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:20 smithi204 ceph-mon[35948]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:20.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:20 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1847786952' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T14:41:20.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:20 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3298062552' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T14:41:20.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:20 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T14:41:20.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:20 smithi062 ceph-mon[32116]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:20.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:20 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1847786952' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T14:41:20.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:20 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3298062552' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T14:41:20.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:20 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T14:41:20.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:20 smithi113 ceph-mon[36135]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:20.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:20 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1847786952' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T14:41:20.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:20 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3298062552' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T14:41:20.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:20 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T14:41:20.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:20.902+0000 7fb723e00640 1 -- 172.21.15.62:0/3653548063 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb71c109330 con 0x7fb71c104eb0 2024-04-02T14:41:21.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.455+0000 7fb71a000640 1 -- 172.21.15.62:0/3653548063 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7fb71c109330 con 0x7fb71c104eb0 2024-04-02T14:41:21.455 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-02T14:41:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.462+0000 7fb723e00640 1 -- 172.21.15.62:0/3653548063 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb7040a3bb0 msgr2=0x7fb7040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.462+0000 7fb723e00640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb7040a3bb0 0x7fb7040a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb710009bd0 tx=0x7fb710005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.462+0000 7fb723e00640 1 -- 172.21.15.62:0/3653548063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71c104eb0 msgr2=0x7fb71c196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.463+0000 7fb723e00640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71c104eb0 0x7fb71c196df0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb708000c00 tx=0x7fb70802c710 comp rx=0 tx=0).stop 2024-04-02T14:41:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.463+0000 7fb723e00640 1 -- 172.21.15.62:0/3653548063 shutdown_connections 2024-04-02T14:41:21.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.463+0000 7fb723e00640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb7040a3bb0 0x7fb7040a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:21.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.463+0000 7fb723e00640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c1077e0 0x7fb71c193420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:21.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.463+0000 7fb723e00640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71c104eb0 0x7fb71c196df0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:21.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.463+0000 7fb723e00640 1 --2- 172.21.15.62:0/3653548063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb71c102580 0x7fb71c1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:21.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.463+0000 7fb723e00640 1 -- 172.21.15.62:0/3653548063 >> 172.21.15.62:0/3653548063 conn(0x7fb71c0fbf90 msgr2=0x7fb71c101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:21.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.464+0000 7fb723e00640 1 -- 172.21.15.62:0/3653548063 shutdown_connections 2024-04-02T14:41:21.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.464+0000 7fb723e00640 1 -- 172.21.15.62:0/3653548063 wait complete. 2024-04-02T14:41:21.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-02T14:41:21.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.703+0000 7eff5fe00640 1 -- 172.21.15.62:0/333488802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff60107710 msgr2=0x7eff60109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:21.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.703+0000 7eff5fe00640 1 --2- 172.21.15.62:0/333488802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff60107710 0x7eff60109b60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff58009be0 tx=0x7eff5802bef0 comp rx=0 tx=0).stop 2024-04-02T14:41:21.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.703+0000 7eff5fe00640 1 -- 172.21.15.62:0/333488802 shutdown_connections 2024-04-02T14:41:21.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.703+0000 7eff5fe00640 1 --2- 172.21.15.62:0/333488802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff60107710 0x7eff60109b60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:21.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.703+0000 7eff5fe00640 1 --2- 172.21.15.62:0/333488802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff60104de0 0x7eff601071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:21.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.703+0000 7eff5fe00640 1 --2- 172.21.15.62:0/333488802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff601024b0 0x7eff601048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:21.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.703+0000 7eff5fe00640 1 -- 172.21.15.62:0/333488802 >> 172.21.15.62:0/333488802 conn(0x7eff600fbea0 msgr2=0x7eff600fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:21.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.703+0000 7eff5fe00640 1 -- 172.21.15.62:0/333488802 shutdown_connections 2024-04-02T14:41:21.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.703+0000 7eff5fe00640 1 -- 172.21.15.62:0/333488802 wait complete. 2024-04-02T14:41:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.704+0000 7eff5fe00640 1 Processor -- start 2024-04-02T14:41:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5fe00640 1 -- start start 2024-04-02T14:41:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff601024b0 0x7eff6019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff60104de0 0x7eff6019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff60107710 0x7eff601a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010d800 con 0x7eff60107710 2024-04-02T14:41:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010d680 con 0x7eff60104de0 2024-04-02T14:41:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010d980 con 0x7eff601024b0 2024-04-02T14:41:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff601024b0 0x7eff6019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff60107710 0x7eff601a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff60104de0 0x7eff6019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff601024b0 0x7eff6019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:44050/0 (socket says 172.21.15.62:44050) 2024-04-02T14:41:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.705+0000 7eff5ea00640 1 -- 172.21.15.62:0/2469635643 learned_addr learned my addr 172.21.15.62:0/2469635643 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.706+0000 7eff5ea00640 1 -- 172.21.15.62:0/2469635643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff60104de0 msgr2=0x7eff6019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.706+0000 7eff5ea00640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff60104de0 0x7eff6019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.706+0000 7eff5ea00640 1 -- 172.21.15.62:0/2469635643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff60107710 msgr2=0x7eff601a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.706+0000 7eff5ea00640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff60107710 0x7eff601a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.706+0000 7eff5ea00640 1 -- 172.21.15.62:0/2469635643 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff601a1c50 con 0x7eff601024b0 2024-04-02T14:41:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.706+0000 7eff5e000640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff60104de0 0x7eff6019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.706+0000 7eff5ea00640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff601024b0 0x7eff6019a030 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff5000e9c0 tx=0x7eff5000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.706+0000 7eff64a00640 1 -- 172.21.15.62:0/2469635643 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5000cde0 con 0x7eff601024b0 2024-04-02T14:41:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.706+0000 7eff64a00640 1 -- 172.21.15.62:0/2469635643 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff500035f0 con 0x7eff601024b0 2024-04-02T14:41:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.706+0000 7eff64a00640 1 -- 172.21.15.62:0/2469635643 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50010670 con 0x7eff601024b0 2024-04-02T14:41:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.707+0000 7eff5fe00640 1 -- 172.21.15.62:0/2469635643 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff601a1f40 con 0x7eff601024b0 2024-04-02T14:41:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.707+0000 7eff5fe00640 1 -- 172.21.15.62:0/2469635643 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff601a22f0 con 0x7eff601024b0 2024-04-02T14:41:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.708+0000 7eff5fe00640 1 -- 172.21.15.62:0/2469635643 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff60109200 con 0x7eff601024b0 2024-04-02T14:41:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.710+0000 7eff64a00640 1 -- 172.21.15.62:0/2469635643 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7eff500037a0 con 0x7eff601024b0 2024-04-02T14:41:21.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.711+0000 7eff64a00640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff440a3c30 0x7eff440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:21.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.711+0000 7eff64a00640 1 -- 172.21.15.62:0/2469635643 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7879+0+0 (secure 0 0 0) 0x7eff500d0570 con 0x7eff601024b0 2024-04-02T14:41:21.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.711+0000 7eff5e000640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff440a3c30 0x7eff440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.712+0000 7eff5e000640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff440a3c30 0x7eff440a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7eff6019b540 tx=0x7eff4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:21.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:21.715+0000 7eff64a00640 1 -- 172.21.15.62:0/2469635643 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7eff5009d060 con 0x7eff601024b0 2024-04-02T14:41:21.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:21 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T14:41:21.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:21 smithi204 ceph-mon[35948]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T14:41:21.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:21 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3653548063' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T14:41:21.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:21 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T14:41:21.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:21 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T14:41:21.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:21 smithi204 ceph-mon[35948]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T14:41:21.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:21 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T14:41:21.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:21 smithi062 ceph-mon[32116]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T14:41:21.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:21 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3653548063' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T14:41:21.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:21 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T14:41:21.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:21 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T14:41:21.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:21 smithi062 ceph-mon[32116]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T14:41:21.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:21 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T14:41:21.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:21 smithi113 ceph-mon[36135]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T14:41:21.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:21.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:21 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3653548063' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T14:41:21.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:21 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T14:41:21.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:21 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T14:41:21.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:21 smithi113 ceph-mon[36135]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T14:41:22.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:22.004+0000 7eff5fe00640 1 -- 172.21.15.62:0/2469635643 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7eff6002d370 con 0x7eff601024b0 2024-04-02T14:41:22.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:22 smithi204 ceph-mon[35948]: pgmap v316: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:22.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:22 smithi204 ceph-mon[35948]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T14:41:22.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:22.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:22.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:22.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:41:22.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:22 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2469635643' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T14:41:22.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:22 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:22.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:22 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T14:41:22.759 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:22 smithi204 ceph-mon[35948]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T14:41:22.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:22 smithi062 ceph-mon[32116]: pgmap v316: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:22.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:22 smithi062 ceph-mon[32116]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T14:41:22.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:22.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:22.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:22.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:41:22.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:22 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2469635643' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T14:41:22.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:22 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:22.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:22 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T14:41:22.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:22 smithi062 ceph-mon[32116]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T14:41:22.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:22 smithi113 ceph-mon[36135]: pgmap v316: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:22.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:22 smithi113 ceph-mon[36135]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T14:41:22.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:22.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:22.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:41:22.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:41:22.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:22 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2469635643' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T14:41:22.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:22 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:41:22.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:22 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T14:41:22.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:22 smithi113 ceph-mon[36135]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T14:41:23.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.008+0000 7eff64a00640 1 -- 172.21.15.62:0/2469635643 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7eff5009c960 con 0x7eff601024b0 2024-04-02T14:41:23.007 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 2 and data pool 3 2024-04-02T14:41:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.015+0000 7eff5fe00640 1 -- 172.21.15.62:0/2469635643 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff440a3c30 msgr2=0x7eff440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.015+0000 7eff5fe00640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff440a3c30 0x7eff440a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7eff6019b540 tx=0x7eff4c005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.015+0000 7eff5fe00640 1 -- 172.21.15.62:0/2469635643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff601024b0 msgr2=0x7eff6019a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.015+0000 7eff5fe00640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff601024b0 0x7eff6019a030 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff5000e9c0 tx=0x7eff5000ee80 comp rx=0 tx=0).stop 2024-04-02T14:41:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.015+0000 7eff5fe00640 1 -- 172.21.15.62:0/2469635643 shutdown_connections 2024-04-02T14:41:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.015+0000 7eff5fe00640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7eff440a3c30 0x7eff440a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.015+0000 7eff5fe00640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff60107710 0x7eff601a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.015+0000 7eff5fe00640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff60104de0 0x7eff6019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.015+0000 7eff5fe00640 1 --2- 172.21.15.62:0/2469635643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff601024b0 0x7eff6019a030 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.015+0000 7eff5fe00640 1 -- 172.21.15.62:0/2469635643 >> 172.21.15.62:0/2469635643 conn(0x7eff600fbea0 msgr2=0x7eff60101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.017+0000 7eff5fe00640 1 -- 172.21.15.62:0/2469635643 shutdown_connections 2024-04-02T14:41:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.017+0000 7eff5fe00640 1 -- 172.21.15.62:0/2469635643 wait complete. 2024-04-02T14:41:23.037 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:41:23.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.252+0000 7f98ffe00640 1 -- 172.21.15.62:0/1973297595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9900107f10 msgr2=0x7f9900108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.252+0000 7f98ffe00640 1 --2- 172.21.15.62:0/1973297595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9900107f10 0x7f9900108350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009900 tx=0x7f98f002bc30 comp rx=0 tx=0).stop 2024-04-02T14:41:23.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.252+0000 7f98ffe00640 1 -- 172.21.15.62:0/1973297595 shutdown_connections 2024-04-02T14:41:23.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.252+0000 7f98ffe00640 1 --2- 172.21.15.62:0/1973297595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9900109aa0 0x7f99001103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.252+0000 7f98ffe00640 1 --2- 172.21.15.62:0/1973297595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99001090e0 0x7f9900109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.252+0000 7f98ffe00640 1 --2- 172.21.15.62:0/1973297595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9900107f10 0x7f9900108350 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.252+0000 7f98ffe00640 1 -- 172.21.15.62:0/1973297595 >> 172.21.15.62:0/1973297595 conn(0x7f9900076f10 msgr2=0x7f9900079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:23.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.252+0000 7f98ffe00640 1 -- 172.21.15.62:0/1973297595 shutdown_connections 2024-04-02T14:41:23.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.252+0000 7f98ffe00640 1 -- 172.21.15.62:0/1973297595 wait complete. 2024-04-02T14:41:23.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.253+0000 7f98ffe00640 1 Processor -- start 2024-04-02T14:41:23.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.253+0000 7f98ffe00640 1 -- start start 2024-04-02T14:41:23.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.253+0000 7f98ffe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9900107f10 0x7f990019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:23.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.253+0000 7f98ffe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99001090e0 0x7f990019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:23.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.253+0000 7f98ffe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9900109aa0 0x7f99001a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:23.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.253+0000 7f98ffe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9900113cd0 con 0x7f99001090e0 2024-04-02T14:41:23.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.253+0000 7f98ffe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9900113b50 con 0x7f9900107f10 2024-04-02T14:41:23.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.253+0000 7f98ffe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9900113e50 con 0x7f9900109aa0 2024-04-02T14:41:23.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98fea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9900107f10 0x7f990019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:23.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98fea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9900107f10 0x7f990019e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:36784/0 (socket says 172.21.15.62:36784) 2024-04-02T14:41:23.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98fea00640 1 -- 172.21.15.62:0/3048971380 learned_addr learned my addr 172.21.15.62:0/3048971380 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:23.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98fe000640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99001090e0 0x7f990019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:23.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98ff400640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9900109aa0 0x7f99001a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:23.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98fea00640 1 -- 172.21.15.62:0/3048971380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9900109aa0 msgr2=0x7f99001a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98fea00640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9900109aa0 0x7f99001a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98fea00640 1 -- 172.21.15.62:0/3048971380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99001090e0 msgr2=0x7f990019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98fea00640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99001090e0 0x7f990019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98fea00640 1 -- 172.21.15.62:0/3048971380 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99001a5f50 con 0x7f9900107f10 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98ff400640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9900109aa0 0x7f99001a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.254+0000 7f98fea00640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9900107f10 0x7f990019e330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000c00 tx=0x7f98f0004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.255+0000 7f9904a00640 1 -- 172.21.15.62:0/3048971380 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0043020 con 0x7f9900107f10 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.255+0000 7f98ffe00640 1 -- 172.21.15.62:0/3048971380 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99001a61e0 con 0x7f9900107f10 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.255+0000 7f9904a00640 1 -- 172.21.15.62:0/3048971380 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f98f0004250 con 0x7f9900107f10 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.255+0000 7f9904a00640 1 -- 172.21.15.62:0/3048971380 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0003b90 con 0x7f9900107f10 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.255+0000 7f98ffe00640 1 -- 172.21.15.62:0/3048971380 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99001a6780 con 0x7f9900107f10 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.256+0000 7f98ffe00640 1 -- 172.21.15.62:0/3048971380 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990010f2c0 con 0x7f9900107f10 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.258+0000 7f9904a00640 1 -- 172.21.15.62:0/3048971380 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f98f0046020 con 0x7f9900107f10 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.259+0000 7f9904a00640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98e40a3bb0 0x7f98e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.260+0000 7f9904a00640 1 -- 172.21.15.62:0/3048971380 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f98f00f2ad0 con 0x7f9900107f10 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.260+0000 7f98fe000640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98e40a3bb0 0x7f98e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.260+0000 7f98fe000640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98e40a3bb0 0x7f98e40a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f990019f840 tx=0x7f98ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:23.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.262+0000 7f9904a00640 1 -- 172.21.15.62:0/3048971380 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f98f004d020 con 0x7f9900107f10 2024-04-02T14:41:23.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.471+0000 7f98ffe00640 1 -- 172.21.15.62:0/3048971380 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9900062700 con 0x7f9900107f10 2024-04-02T14:41:23.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.472+0000 7f9904a00640 1 -- 172.21.15.62:0/3048971380 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22210 (secure 0 0 0) 0x7f98f00bf560 con 0x7f9900107f10 2024-04-02T14:41:23.472 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:23.472 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":87,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:41:22.008327+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:41:23.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.475+0000 7f98ffe00640 1 -- 172.21.15.62:0/3048971380 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98e40a3bb0 msgr2=0x7f98e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.475+0000 7f98ffe00640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98e40a3bb0 0x7f98e40a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f990019f840 tx=0x7f98ec005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:23.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.475+0000 7f98ffe00640 1 -- 172.21.15.62:0/3048971380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9900107f10 msgr2=0x7f990019e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.475+0000 7f98ffe00640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9900107f10 0x7f990019e330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000c00 tx=0x7f98f0004a30 comp rx=0 tx=0).stop 2024-04-02T14:41:23.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.475+0000 7f98ffe00640 1 -- 172.21.15.62:0/3048971380 shutdown_connections 2024-04-02T14:41:23.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.475+0000 7f98ffe00640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f98e40a3bb0 0x7f98e40a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.475+0000 7f98ffe00640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9900109aa0 0x7f99001a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.475+0000 7f98ffe00640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99001090e0 0x7f990019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.475+0000 7f98ffe00640 1 --2- 172.21.15.62:0/3048971380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9900107f10 0x7f990019e330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.475+0000 7f98ffe00640 1 -- 172.21.15.62:0/3048971380 >> 172.21.15.62:0/3048971380 conn(0x7f9900076f10 msgr2=0x7f990010c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:23.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.476+0000 7f98ffe00640 1 -- 172.21.15.62:0/3048971380 shutdown_connections 2024-04-02T14:41:23.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.476+0000 7f98ffe00640 1 -- 172.21.15.62:0/3048971380 wait complete. 2024-04-02T14:41:23.496 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:41:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.717+0000 7fbdada00640 1 -- 172.21.15.62:0/1229474686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda8107940 msgr2=0x7fbda8109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.717+0000 7fbdada00640 1 --2- 172.21.15.62:0/1229474686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda8107940 0x7fbda8109d30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009ca0 tx=0x7fbda002c100 comp rx=0 tx=0).stop 2024-04-02T14:41:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.717+0000 7fbdada00640 1 -- 172.21.15.62:0/1229474686 shutdown_connections 2024-04-02T14:41:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.717+0000 7fbdada00640 1 --2- 172.21.15.62:0/1229474686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda8107940 0x7fbda8109d30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.717+0000 7fbdada00640 1 --2- 172.21.15.62:0/1229474686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda8105010 0x7fbda8107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.717+0000 7fbdada00640 1 --2- 172.21.15.62:0/1229474686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbda81026e0 0x7fbda8104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.717+0000 7fbdada00640 1 -- 172.21.15.62:0/1229474686 >> 172.21.15.62:0/1229474686 conn(0x7fbda80fc030 msgr2=0x7fbda80fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.717+0000 7fbdada00640 1 -- 172.21.15.62:0/1229474686 shutdown_connections 2024-04-02T14:41:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.717+0000 7fbdada00640 1 -- 172.21.15.62:0/1229474686 wait complete. 2024-04-02T14:41:23.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.718+0000 7fbdada00640 1 Processor -- start 2024-04-02T14:41:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbdada00640 1 -- start start 2024-04-02T14:41:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbdada00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda81026e0 0x7fbda819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbdada00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbda8105010 0x7fbda819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbdada00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda8107940 0x7fbda81a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbdada00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810d910 con 0x7fbda8107940 2024-04-02T14:41:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbdada00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810d790 con 0x7fbda8105010 2024-04-02T14:41:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbdada00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810da90 con 0x7fbda81026e0 2024-04-02T14:41:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbda6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda81026e0 0x7fbda819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbda6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbda8105010 0x7fbda819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbda7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda8107940 0x7fbda81a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbda6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbda8105010 0x7fbda819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:36804/0 (socket says 172.21.15.62:36804) 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbda6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda81026e0 0x7fbda819a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:34868/0 (socket says 172.21.15.62:34868) 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.719+0000 7fbda6000640 1 -- 172.21.15.62:0/907952056 learned_addr learned my addr 172.21.15.62:0/907952056 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.720+0000 7fbda6000640 1 -- 172.21.15.62:0/907952056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda81026e0 msgr2=0x7fbda819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.720+0000 7fbda6000640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda81026e0 0x7fbda819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.720+0000 7fbda6000640 1 -- 172.21.15.62:0/907952056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda8107940 msgr2=0x7fbda81a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.720+0000 7fbda6000640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda8107940 0x7fbda81a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.720+0000 7fbda6000640 1 -- 172.21.15.62:0/907952056 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda81a1ca0 con 0x7fbda8105010 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.720+0000 7fbda7400640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda8107940 0x7fbda81a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.720+0000 7fbda6000640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbda8105010 0x7fbda819a5c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400e9f0 tx=0x7fbd9400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.721+0000 7fbd9f400640 1 -- 172.21.15.62:0/907952056 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400ccc0 con 0x7fbda8105010 2024-04-02T14:41:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.721+0000 7fbdada00640 1 -- 172.21.15.62:0/907952056 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda81a1f90 con 0x7fbda8105010 2024-04-02T14:41:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.721+0000 7fbd9f400640 1 -- 172.21.15.62:0/907952056 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd94005520 con 0x7fbda8105010 2024-04-02T14:41:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.721+0000 7fbd9f400640 1 -- 172.21.15.62:0/907952056 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94010690 con 0x7fbda8105010 2024-04-02T14:41:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.721+0000 7fbdada00640 1 -- 172.21.15.62:0/907952056 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda81a2450 con 0x7fbda8105010 2024-04-02T14:41:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.722+0000 7fbdada00640 1 -- 172.21.15.62:0/907952056 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda8109430 con 0x7fbda8105010 2024-04-02T14:41:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.724+0000 7fbd9f400640 1 -- 172.21.15.62:0/907952056 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbd94004f70 con 0x7fbda8105010 2024-04-02T14:41:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.725+0000 7fbd9f400640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbd8c0a3c30 0x7fbd8c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.725+0000 7fbd9f400640 1 -- 172.21.15.62:0/907952056 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fbd940d05f0 con 0x7fbda8105010 2024-04-02T14:41:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.725+0000 7fbda6a00640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbd8c0a3c30 0x7fbd8c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.726+0000 7fbda6a00640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbd8c0a3c30 0x7fbd8c0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009bd0 tx=0x7fbd98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:23.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.729+0000 7fbd9f400640 1 -- 172.21.15.62:0/907952056 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbd9409d080 con 0x7fbda8105010 2024-04-02T14:41:23.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:23 smithi062 ceph-mon[32116]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T14:41:23.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:23 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T14:41:23.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:23 smithi062 ceph-mon[32116]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T14:41:23.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:23 smithi062 ceph-mon[32116]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T14:41:23.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:23 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3048971380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:23.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:23 smithi113 ceph-mon[36135]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T14:41:23.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:23 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T14:41:23.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:23 smithi113 ceph-mon[36135]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T14:41:23.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:23 smithi113 ceph-mon[36135]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T14:41:23.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:23 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3048971380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.942+0000 7fbdada00640 1 -- 172.21.15.62:0/907952056 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbda8062700 con 0x7fbda8105010 2024-04-02T14:41:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.944+0000 7fbd9f400640 1 -- 172.21.15.62:0/907952056 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22197 (secure 0 0 0) 0x7fbd9409c980 con 0x7fbda8105010 2024-04-02T14:41:23.944 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:23.944 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:41:23.500989+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:41:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 -- 172.21.15.62:0/907952056 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbd8c0a3c30 msgr2=0x7fbd8c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbd8c0a3c30 0x7fbd8c0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009bd0 tx=0x7fbd98005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 -- 172.21.15.62:0/907952056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbda8105010 msgr2=0x7fbda819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbda8105010 0x7fbda819a5c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400e9f0 tx=0x7fbd9400eeb0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 -- 172.21.15.62:0/907952056 shutdown_connections 2024-04-02T14:41:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbd8c0a3c30 0x7fbd8c0a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda8107940 0x7fbda81a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbda8105010 0x7fbda819a5c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 --2- 172.21.15.62:0/907952056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda81026e0 0x7fbda819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 -- 172.21.15.62:0/907952056 >> 172.21.15.62:0/907952056 conn(0x7fbda80fc030 msgr2=0x7fbda8101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 -- 172.21.15.62:0/907952056 shutdown_connections 2024-04-02T14:41:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:23.947+0000 7fbdada00640 1 -- 172.21.15.62:0/907952056 wait complete. 2024-04-02T14:41:23.965 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-02T14:41:24.008 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:23 smithi204 ceph-mon[35948]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T14:41:24.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:23 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T14:41:24.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:23 smithi204 ceph-mon[35948]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T14:41:24.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:23 smithi204 ceph-mon[35948]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T14:41:24.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:23 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3048971380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:24.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.169+0000 7f5cb7e00640 1 -- 172.21.15.62:0/49890579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb0100a20 msgr2=0x7f5cb0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:24.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.169+0000 7f5cb7e00640 1 --2- 172.21.15.62:0/49890579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb0100a20 0x7f5cb0100e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009900 tx=0x7f5ca402bc30 comp rx=0 tx=0).stop 2024-04-02T14:41:24.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.169+0000 7f5cb7e00640 1 -- 172.21.15.62:0/49890579 shutdown_connections 2024-04-02T14:41:24.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.169+0000 7f5cb7e00640 1 --2- 172.21.15.62:0/49890579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb01025d0 0x7f5cb0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:24.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.169+0000 7f5cb7e00640 1 --2- 172.21.15.62:0/49890579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb0101c10 0x7f5cb0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:24.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.169+0000 7f5cb7e00640 1 --2- 172.21.15.62:0/49890579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb0100a20 0x7f5cb0100e40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:24.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.169+0000 7f5cb7e00640 1 -- 172.21.15.62:0/49890579 >> 172.21.15.62:0/49890579 conn(0x7f5cb00fbf90 msgr2=0x7f5cb00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:24.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.169+0000 7f5cb7e00640 1 -- 172.21.15.62:0/49890579 shutdown_connections 2024-04-02T14:41:24.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.169+0000 7f5cb7e00640 1 -- 172.21.15.62:0/49890579 wait complete. 2024-04-02T14:41:24.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.170+0000 7f5cb7e00640 1 Processor -- start 2024-04-02T14:41:24.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.171+0000 7f5cb7e00640 1 -- start start 2024-04-02T14:41:24.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.171+0000 7f5cb7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb0100a20 0x7f5cb010b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.171+0000 7f5cb7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb0101c10 0x7f5cb01099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.171+0000 7f5cb7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb01025d0 0x7f5cb0109f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.171+0000 7f5cb7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb010d890 con 0x7f5cb0101c10 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.171+0000 7f5cb7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb010d710 con 0x7f5cb0100a20 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.171+0000 7f5cb7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb010da10 con 0x7f5cb01025d0 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.171+0000 7f5cb4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb0101c10 0x7f5cb01099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.171+0000 7f5cb4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb0101c10 0x7f5cb01099e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54818/0 (socket says 172.21.15.62:54818) 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.171+0000 7f5cb4a00640 1 -- 172.21.15.62:0/2570371650 learned_addr learned my addr 172.21.15.62:0/2570371650 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.172+0000 7f5cb5400640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb0100a20 0x7f5cb010b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.172+0000 7f5cb5e00640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb01025d0 0x7f5cb0109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.172+0000 7f5cb4a00640 1 -- 172.21.15.62:0/2570371650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb01025d0 msgr2=0x7f5cb0109f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.172+0000 7f5cb4a00640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb01025d0 0x7f5cb0109f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:24.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.172+0000 7f5cb4a00640 1 -- 172.21.15.62:0/2570371650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb0100a20 msgr2=0x7f5cb010b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:24.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.172+0000 7f5cb4a00640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb0100a20 0x7f5cb010b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:24.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.172+0000 7f5cb4a00640 1 -- 172.21.15.62:0/2570371650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb010a7b0 con 0x7f5cb0101c10 2024-04-02T14:41:24.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.172+0000 7f5cb5e00640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb01025d0 0x7f5cb0109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:41:24.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.172+0000 7f5cb4a00640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb0101c10 0x7f5cb01099e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00e9f0 tx=0x7f5c9c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:24.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.172+0000 7f5cae000640 1 -- 172.21.15.62:0/2570371650 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00cd80 con 0x7f5cb0101c10 2024-04-02T14:41:24.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.173+0000 7f5cae000640 1 -- 172.21.15.62:0/2570371650 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5c9c005520 con 0x7f5cb0101c10 2024-04-02T14:41:24.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.173+0000 7f5cb7e00640 1 -- 172.21.15.62:0/2570371650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb01a2110 con 0x7f5cb0101c10 2024-04-02T14:41:24.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.173+0000 7f5cae000640 1 -- 172.21.15.62:0/2570371650 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c010690 con 0x7f5cb0101c10 2024-04-02T14:41:24.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.173+0000 7f5cb7e00640 1 -- 172.21.15.62:0/2570371650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb01a2660 con 0x7f5cb0101c10 2024-04-02T14:41:24.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.174+0000 7f5cb7e00640 1 -- 172.21.15.62:0/2570371650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb0107de0 con 0x7f5cb0101c10 2024-04-02T14:41:24.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.177+0000 7f5cae000640 1 -- 172.21.15.62:0/2570371650 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5c9c0046c0 con 0x7f5cb0101c10 2024-04-02T14:41:24.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.179+0000 7f5cae000640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5c980a3c30 0x7f5c980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:24.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.179+0000 7f5cae000640 1 -- 172.21.15.62:0/2570371650 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5c9c0d09e0 con 0x7f5cb0101c10 2024-04-02T14:41:24.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.180+0000 7f5cb5400640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5c980a3c30 0x7f5c980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:24.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.180+0000 7f5cb5400640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5c980a3c30 0x7f5c980a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4005ab0 tx=0x7f5ca4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:24.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.181+0000 7f5cae000640 1 -- 172.21.15.62:0/2570371650 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5c9c053f20 con 0x7f5cb0101c10 2024-04-02T14:41:24.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:24.431+0000 7f5cb7e00640 1 -- 172.21.15.62:0/2570371650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5cb002d370 con 0x7f5cb0101c10 2024-04-02T14:41:24.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:24 smithi062 ceph-mon[32116]: pgmap v319: 97 pgs: 3 creating+peering, 11 active+clean, 83 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:24.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:24 smithi062 ceph-mon[32116]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T14:41:24.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:24 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/907952056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:24.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:24 smithi062 ceph-mon[32116]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T14:41:24.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:24 smithi062 ceph-mon[32116]: Cluster is now healthy 2024-04-02T14:41:24.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:24 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] up:active 2024-04-02T14:41:24.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:24 smithi062 ceph-mon[32116]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:24.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:24 smithi062 ceph-mon[32116]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:24.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:24 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2570371650' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T14:41:24.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:24 smithi113 ceph-mon[36135]: pgmap v319: 97 pgs: 3 creating+peering, 11 active+clean, 83 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:24.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:24 smithi113 ceph-mon[36135]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T14:41:24.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:24 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/907952056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:24.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:24 smithi113 ceph-mon[36135]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T14:41:24.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:24 smithi113 ceph-mon[36135]: Cluster is now healthy 2024-04-02T14:41:24.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:24 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] up:active 2024-04-02T14:41:24.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:24 smithi113 ceph-mon[36135]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:24.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:24 smithi113 ceph-mon[36135]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:24.870 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:24 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2570371650' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T14:41:25.008 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:24 smithi204 ceph-mon[35948]: pgmap v319: 97 pgs: 3 creating+peering, 11 active+clean, 83 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:41:25.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:24 smithi204 ceph-mon[35948]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T14:41:25.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:24 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/907952056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:25.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:24 smithi204 ceph-mon[35948]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T14:41:25.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:24 smithi204 ceph-mon[35948]: Cluster is now healthy 2024-04-02T14:41:25.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:24 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] up:active 2024-04-02T14:41:25.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:24 smithi204 ceph-mon[35948]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:25.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:24 smithi204 ceph-mon[35948]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:25.009 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:24 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2570371650' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T14:41:25.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.014+0000 7f5cae000640 1 -- 172.21.15.62:0/2570371650 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f5c9c09d470 con 0x7f5cb0101c10 2024-04-02T14:41:25.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.017+0000 7f5cb7e00640 1 -- 172.21.15.62:0/2570371650 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5c980a3c30 msgr2=0x7f5c980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:25.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.017+0000 7f5cb7e00640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5c980a3c30 0x7f5c980a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4005ab0 tx=0x7f5ca4005fd0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.017+0000 7f5cb7e00640 1 -- 172.21.15.62:0/2570371650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb0101c10 msgr2=0x7f5cb01099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:25.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.017+0000 7f5cb7e00640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb0101c10 0x7f5cb01099e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00e9f0 tx=0x7f5c9c00edf0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.017+0000 7f5cb7e00640 1 -- 172.21.15.62:0/2570371650 shutdown_connections 2024-04-02T14:41:25.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.017+0000 7f5cb7e00640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f5c980a3c30 0x7f5c980a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.017+0000 7f5cb7e00640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb01025d0 0x7f5cb0109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.017+0000 7f5cb7e00640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb0101c10 0x7f5cb01099e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.017+0000 7f5cb7e00640 1 --2- 172.21.15.62:0/2570371650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb0100a20 0x7f5cb010b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.017+0000 7f5cb7e00640 1 -- 172.21.15.62:0/2570371650 >> 172.21.15.62:0/2570371650 conn(0x7f5cb00fbf90 msgr2=0x7f5cb0104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:25.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.018+0000 7f5cb7e00640 1 -- 172.21.15.62:0/2570371650 shutdown_connections 2024-04-02T14:41:25.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.018+0000 7f5cb7e00640 1 -- 172.21.15.62:0/2570371650 wait complete. 2024-04-02T14:41:25.039 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-02T14:41:25.040 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-02T14:41:25.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.257+0000 7f64c9000640 1 -- 172.21.15.62:0/3409137847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64c4074500 msgr2=0x7f64c4074960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:25.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.257+0000 7f64c9000640 1 --2- 172.21.15.62:0/3409137847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64c4074500 0x7f64c4074960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009900 tx=0x7f64ac02bcd0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.257+0000 7f64c9000640 1 -- 172.21.15.62:0/3409137847 shutdown_connections 2024-04-02T14:41:25.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.257+0000 7f64c9000640 1 --2- 172.21.15.62:0/3409137847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64c4109890 0x7f64c410bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.257+0000 7f64c9000640 1 --2- 172.21.15.62:0/3409137847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64c4074500 0x7f64c4074960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.257+0000 7f64c9000640 1 --2- 172.21.15.62:0/3409137847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64c4075ae0 0x7f64c4073f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.257+0000 7f64c9000640 1 -- 172.21.15.62:0/3409137847 >> 172.21.15.62:0/3409137847 conn(0x7f64c40fcf70 msgr2=0x7f64c40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:25.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.257+0000 7f64c9000640 1 -- 172.21.15.62:0/3409137847 shutdown_connections 2024-04-02T14:41:25.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.257+0000 7f64c9000640 1 -- 172.21.15.62:0/3409137847 wait complete. 2024-04-02T14:41:25.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.258+0000 7f64c9000640 1 Processor -- start 2024-04-02T14:41:25.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.258+0000 7f64c9000640 1 -- start start 2024-04-02T14:41:25.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64c4074500 0x7f64c4199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:25.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c9000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64c4075ae0 0x7f64c419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c9000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64c4109890 0x7f64c41a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c9000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f64c410fa00 con 0x7f64c4074500 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c9000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f64c410f880 con 0x7f64c4109890 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c9000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64c410fb80 con 0x7f64c4075ae0 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64c4074500 0x7f64c4199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64c4074500 0x7f64c4199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54832/0 (socket says 172.21.15.62:54832) 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c3400640 1 -- 172.21.15.62:0/1030013 learned_addr learned my addr 172.21.15.62:0/1030013 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c2a00640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64c4075ae0 0x7f64c419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c3e00640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64c4109890 0x7f64c41a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c3400640 1 -- 172.21.15.62:0/1030013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64c4075ae0 msgr2=0x7f64c419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c3400640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64c4075ae0 0x7f64c419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c3400640 1 -- 172.21.15.62:0/1030013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64c4109890 msgr2=0x7f64c41a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c3400640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64c4109890 0x7f64c41a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c3400640 1 -- 172.21.15.62:0/1030013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c41a1be0 con 0x7f64c4074500 2024-04-02T14:41:25.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c2a00640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64c4075ae0 0x7f64c419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:41:25.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.259+0000 7f64c3e00640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64c4109890 0x7f64c41a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:41:25.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.260+0000 7f64c3400640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64c4074500 0x7f64c4199fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f64b400e9f0 tx=0x7f64b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:25.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.260+0000 7f64bbe00640 1 -- 172.21.15.62:0/1030013 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b400cd80 con 0x7f64c4074500 2024-04-02T14:41:25.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.260+0000 7f64bbe00640 1 -- 172.21.15.62:0/1030013 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f64b4005520 con 0x7f64c4074500 2024-04-02T14:41:25.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.260+0000 7f64c9000640 1 -- 172.21.15.62:0/1030013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c41a1ed0 con 0x7f64c4074500 2024-04-02T14:41:25.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.260+0000 7f64bbe00640 1 -- 172.21.15.62:0/1030013 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4010690 con 0x7f64c4074500 2024-04-02T14:41:25.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.260+0000 7f64c9000640 1 -- 172.21.15.62:0/1030013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c41a2330 con 0x7f64c4074500 2024-04-02T14:41:25.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.261+0000 7f64c9000640 1 -- 172.21.15.62:0/1030013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c4108650 con 0x7f64c4074500 2024-04-02T14:41:25.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.263+0000 7f64bbe00640 1 -- 172.21.15.62:0/1030013 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f64b40046c0 con 0x7f64c4074500 2024-04-02T14:41:25.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.265+0000 7f64bbe00640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f64a80a3c30 0x7f64a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:25.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.266+0000 7f64c2a00640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f64a80a3c30 0x7f64a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:25.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.266+0000 7f64bbe00640 1 -- 172.21.15.62:0/1030013 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f64b40d0830 con 0x7f64c4074500 2024-04-02T14:41:25.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.266+0000 7f64c2a00640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f64a80a3c30 0x7f64a80a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f64ac02c2e0 tx=0x7f64ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:25.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.270+0000 7f64bbe00640 1 -- 172.21.15.62:0/1030013 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f64b409d2c0 con 0x7f64c4074500 2024-04-02T14:41:25.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:25.523+0000 7f64c9000640 1 -- 172.21.15.62:0/1030013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f64c40627d0 con 0x7f64c4074500 2024-04-02T14:41:26.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.025+0000 7f64bbe00640 1 -- 172.21.15.62:0/1030013 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f64b409cbc0 con 0x7f64c4074500 2024-04-02T14:41:26.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.029+0000 7f64c9000640 1 -- 172.21.15.62:0/1030013 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f64a80a3c30 msgr2=0x7f64a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:26.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.029+0000 7f64c9000640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f64a80a3c30 0x7f64a80a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f64ac02c2e0 tx=0x7f64ac005fd0 comp rx=0 tx=0).stop 2024-04-02T14:41:26.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.029+0000 7f64c9000640 1 -- 172.21.15.62:0/1030013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64c4074500 msgr2=0x7f64c4199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:26.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.029+0000 7f64c9000640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64c4074500 0x7f64c4199fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f64b400e9f0 tx=0x7f64b400edf0 comp rx=0 tx=0).stop 2024-04-02T14:41:26.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.029+0000 7f64c9000640 1 -- 172.21.15.62:0/1030013 shutdown_connections 2024-04-02T14:41:26.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.029+0000 7f64c9000640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f64a80a3c30 0x7f64a80a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:26.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.029+0000 7f64c9000640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64c4109890 0x7f64c41a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:26.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.029+0000 7f64c9000640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64c4075ae0 0x7f64c419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:26.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.029+0000 7f64c9000640 1 --2- 172.21.15.62:0/1030013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64c4074500 0x7f64c4199fc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:26.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.029+0000 7f64c9000640 1 -- 172.21.15.62:0/1030013 >> 172.21.15.62:0/1030013 conn(0x7f64c40fcf70 msgr2=0x7f64c4105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:26.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.030+0000 7f64c9000640 1 -- 172.21.15.62:0/1030013 shutdown_connections 2024-04-02T14:41:26.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.030+0000 7f64c9000640 1 -- 172.21.15.62:0/1030013 wait complete. 2024-04-02T14:41:26.054 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-02T14:41:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.255+0000 7fc996400640 1 -- 172.21.15.62:0/4175303931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc990109310 msgr2=0x7fc990109790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.255+0000 7fc996400640 1 --2- 172.21.15.62:0/4175303931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc990109310 0x7fc990109790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009900 tx=0x7fc97c02bc50 comp rx=0 tx=0).stop 2024-04-02T14:41:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.255+0000 7fc996400640 1 -- 172.21.15.62:0/4175303931 shutdown_connections 2024-04-02T14:41:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.255+0000 7fc996400640 1 --2- 172.21.15.62:0/4175303931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc990109cd0 0x7fc990110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.255+0000 7fc996400640 1 --2- 172.21.15.62:0/4175303931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc990109310 0x7fc990109790 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.255+0000 7fc996400640 1 --2- 172.21.15.62:0/4175303931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990108120 0x7fc990108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:26.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.255+0000 7fc996400640 1 -- 172.21.15.62:0/4175303931 >> 172.21.15.62:0/4175303931 conn(0x7fc9900770c0 msgr2=0x7fc990079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:26.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.255+0000 7fc996400640 1 -- 172.21.15.62:0/4175303931 shutdown_connections 2024-04-02T14:41:26.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.255+0000 7fc996400640 1 -- 172.21.15.62:0/4175303931 wait complete. 2024-04-02T14:41:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.256+0000 7fc996400640 1 Processor -- start 2024-04-02T14:41:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.256+0000 7fc996400640 1 -- start start 2024-04-02T14:41:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.256+0000 7fc996400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc990108120 0x7fc99019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.256+0000 7fc996400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc990109310 0x7fc99019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.256+0000 7fc996400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990109cd0 0x7fc9901a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.256+0000 7fc996400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc990113ed0 con 0x7fc990109310 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.256+0000 7fc996400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc990113d50 con 0x7fc990109cd0 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.256+0000 7fc996400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc990114050 con 0x7fc990108120 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.257+0000 7fc98ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc990109310 0x7fc99019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.257+0000 7fc98ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc990109310 0x7fc99019ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54848/0 (socket says 172.21.15.62:54848) 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.257+0000 7fc98ea00640 1 -- 172.21.15.62:0/2752716629 learned_addr learned my addr 172.21.15.62:0/2752716629 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.257+0000 7fc98f400640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc990108120 0x7fc99019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.257+0000 7fc98fe00640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990109cd0 0x7fc9901a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.257+0000 7fc98ea00640 1 -- 172.21.15.62:0/2752716629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc990108120 msgr2=0x7fc99019e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.257+0000 7fc98ea00640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc990108120 0x7fc99019e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.257+0000 7fc98ea00640 1 -- 172.21.15.62:0/2752716629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990109cd0 msgr2=0x7fc9901a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.257+0000 7fc98ea00640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990109cd0 0x7fc9901a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.257+0000 7fc98ea00640 1 -- 172.21.15.62:0/2752716629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9901a6110 con 0x7fc990109310 2024-04-02T14:41:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.257+0000 7fc98ea00640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc990109310 0x7fc99019ea30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc97c005ab0 tx=0x7fc97c004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:26.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.258+0000 7fc987e00640 1 -- 172.21.15.62:0/2752716629 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c043020 con 0x7fc990109310 2024-04-02T14:41:26.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.258+0000 7fc987e00640 1 -- 172.21.15.62:0/2752716629 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc97c004250 con 0x7fc990109310 2024-04-02T14:41:26.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.258+0000 7fc996400640 1 -- 172.21.15.62:0/2752716629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9901a63a0 con 0x7fc990109310 2024-04-02T14:41:26.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.258+0000 7fc987e00640 1 -- 172.21.15.62:0/2752716629 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c003b30 con 0x7fc990109310 2024-04-02T14:41:26.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.258+0000 7fc996400640 1 -- 172.21.15.62:0/2752716629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a68f0 con 0x7fc990109310 2024-04-02T14:41:26.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.259+0000 7fc996400640 1 -- 172.21.15.62:0/2752716629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99010f3e0 con 0x7fc990109310 2024-04-02T14:41:26.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.262+0000 7fc987e00640 1 -- 172.21.15.62:0/2752716629 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc97c046020 con 0x7fc990109310 2024-04-02T14:41:26.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.263+0000 7fc987e00640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc9740a3bb0 0x7fc9740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:26.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.264+0000 7fc987e00640 1 -- 172.21.15.62:0/2752716629 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc97c0f2850 con 0x7fc990109310 2024-04-02T14:41:26.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.264+0000 7fc98f400640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc9740a3bb0 0x7fc9740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:26.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.264+0000 7fc98f400640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc9740a3bb0 0x7fc9740a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc990109170 tx=0x7fc980005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:26.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.265+0000 7fc987e00640 1 -- 172.21.15.62:0/2752716629 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc97c04d020 con 0x7fc990109310 2024-04-02T14:41:26.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:26 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2570371650' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T14:41:26.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:26 smithi062 ceph-mon[32116]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:26.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:26 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1030013' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T14:41:26.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:26 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2570371650' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T14:41:26.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:26 smithi113 ceph-mon[36135]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:26.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:26 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1030013' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T14:41:26.370 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:41:26 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b[67157]: 2024-04-02T14:41:26.002+0000 7f8f507db700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T14:41:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:26 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2570371650' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T14:41:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:26 smithi204 ceph-mon[35948]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:26.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:26 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1030013' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T14:41:26.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:26.515+0000 7fc996400640 1 -- 172.21.15.62:0/2752716629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc99002d5c0 con 0x7fc990109310 2024-04-02T14:41:27.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.038+0000 7fc987e00640 1 -- 172.21.15.62:0/2752716629 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7fc97c0bf400 con 0x7fc990109310 2024-04-02T14:41:27.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.042+0000 7fc996400640 1 -- 172.21.15.62:0/2752716629 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc9740a3bb0 msgr2=0x7fc9740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:27.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.042+0000 7fc996400640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc9740a3bb0 0x7fc9740a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc990109170 tx=0x7fc980005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:27.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.043+0000 7fc996400640 1 -- 172.21.15.62:0/2752716629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc990109310 msgr2=0x7fc99019ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:27.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.043+0000 7fc996400640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc990109310 0x7fc99019ea30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc97c005ab0 tx=0x7fc97c004a30 comp rx=0 tx=0).stop 2024-04-02T14:41:27.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.043+0000 7fc996400640 1 -- 172.21.15.62:0/2752716629 shutdown_connections 2024-04-02T14:41:27.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.043+0000 7fc996400640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc9740a3bb0 0x7fc9740a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:27.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.043+0000 7fc996400640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990109cd0 0x7fc9901a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:27.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.043+0000 7fc996400640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc990109310 0x7fc99019ea30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:27.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.043+0000 7fc996400640 1 --2- 172.21.15.62:0/2752716629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc990108120 0x7fc99019e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:27.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.043+0000 7fc996400640 1 -- 172.21.15.62:0/2752716629 >> 172.21.15.62:0/2752716629 conn(0x7fc9900770c0 msgr2=0x7fc99010c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:27.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.043+0000 7fc996400640 1 -- 172.21.15.62:0/2752716629 shutdown_connections 2024-04-02T14:41:27.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.043+0000 7fc996400640 1 -- 172.21.15.62:0/2752716629 wait complete. 2024-04-02T14:41:27.068 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-02T14:41:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.262+0000 7fb9c1400640 1 -- 172.21.15.62:0/2323833092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9bc109210 msgr2=0x7fb9bc109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.262+0000 7fb9c1400640 1 --2- 172.21.15.62:0/2323833092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9bc109210 0x7fb9bc109690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009900 tx=0x7fb9a402bc30 comp rx=0 tx=0).stop 2024-04-02T14:41:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.262+0000 7fb9c1400640 1 -- 172.21.15.62:0/2323833092 shutdown_connections 2024-04-02T14:41:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.262+0000 7fb9c1400640 1 --2- 172.21.15.62:0/2323833092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9bc109bd0 0x7fb9bc110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.262+0000 7fb9c1400640 1 --2- 172.21.15.62:0/2323833092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9bc109210 0x7fb9bc109690 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.262+0000 7fb9c1400640 1 --2- 172.21.15.62:0/2323833092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc108020 0x7fb9bc108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.262+0000 7fb9c1400640 1 -- 172.21.15.62:0/2323833092 >> 172.21.15.62:0/2323833092 conn(0x7fb9bc076fd0 msgr2=0x7fb9bc0793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.262+0000 7fb9c1400640 1 -- 172.21.15.62:0/2323833092 shutdown_connections 2024-04-02T14:41:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.262+0000 7fb9c1400640 1 -- 172.21.15.62:0/2323833092 wait complete. 2024-04-02T14:41:27.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.263+0000 7fb9c1400640 1 Processor -- start 2024-04-02T14:41:27.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.263+0000 7fb9c1400640 1 -- start start 2024-04-02T14:41:27.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9c1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9bc108020 0x7fb9bc19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:27.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9c1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9bc109210 0x7fb9bc19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9c1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc109bd0 0x7fb9bc1a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9c1400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc113d80 con 0x7fb9bc108020 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9c1400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc113c00 con 0x7fb9bc109210 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9c1400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc113f00 con 0x7fb9bc109bd0 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9bc108020 0x7fb9bc19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9b3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9bc109210 0x7fb9bc19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9bc108020 0x7fb9bc19e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54874/0 (socket says 172.21.15.62:54874) 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9bbe00640 1 -- 172.21.15.62:0/4120812893 learned_addr learned my addr 172.21.15.62:0/4120812893 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9c0a00640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc109bd0 0x7fb9bc1a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9b3e00640 1 -- 172.21.15.62:0/4120812893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc109bd0 msgr2=0x7fb9bc1a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9b3e00640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc109bd0 0x7fb9bc1a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9b3e00640 1 -- 172.21.15.62:0/4120812893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9bc108020 msgr2=0x7fb9bc19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9b3e00640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9bc108020 0x7fb9bc19e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9b3e00640 1 -- 172.21.15.62:0/4120812893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc1a5fc0 con 0x7fb9bc109210 2024-04-02T14:41:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.264+0000 7fb9bbe00640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9bc108020 0x7fb9bc19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.265+0000 7fb9b3e00640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9bc109210 0x7fb9bc19e8e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009230 tx=0x7fb9a4004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.265+0000 7fb9b9600640 1 -- 172.21.15.62:0/4120812893 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4043020 con 0x7fb9bc109210 2024-04-02T14:41:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.265+0000 7fb9b9600640 1 -- 172.21.15.62:0/4120812893 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb9a4004250 con 0x7fb9bc109210 2024-04-02T14:41:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.265+0000 7fb9b9600640 1 -- 172.21.15.62:0/4120812893 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4003b50 con 0x7fb9bc109210 2024-04-02T14:41:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.265+0000 7fb9c1400640 1 -- 172.21.15.62:0/4120812893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc1a6250 con 0x7fb9bc109210 2024-04-02T14:41:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.265+0000 7fb9c1400640 1 -- 172.21.15.62:0/4120812893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc1a6750 con 0x7fb9bc109210 2024-04-02T14:41:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.266+0000 7fb9c1400640 1 -- 172.21.15.62:0/4120812893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9bc10f340 con 0x7fb9bc109210 2024-04-02T14:41:27.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.268+0000 7fb9b9600640 1 -- 172.21.15.62:0/4120812893 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb9a4046020 con 0x7fb9bc109210 2024-04-02T14:41:27.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.269+0000 7fb9b9600640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb9a00a3c30 0x7fb9a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:27.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.269+0000 7fb9b9600640 1 -- 172.21.15.62:0/4120812893 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb9a40f23f0 con 0x7fb9bc109210 2024-04-02T14:41:27.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.269+0000 7fb9bbe00640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb9a00a3c30 0x7fb9a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:27.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.270+0000 7fb9bbe00640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb9a00a3c30 0x7fb9a00a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0104e0 tx=0x7fb9ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:27.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.273+0000 7fb9b9600640 1 -- 172.21.15.62:0/4120812893 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb9a404d020 con 0x7fb9bc109210 2024-04-02T14:41:27.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:27 smithi062 ceph-mon[32116]: pgmap v321: 97 pgs: 3 creating+peering, 83 active+clean, 11 unknown; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-04-02T14:41:27.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:27 smithi062 ceph-mon[32116]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T14:41:27.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:27 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1030013' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T14:41:27.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:27 smithi062 ceph-mon[32116]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T14:41:27.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:27 smithi062 ceph-mon[32116]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:27.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:27 smithi062 ceph-mon[32116]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-02T14:41:27.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:27 smithi062 ceph-mon[32116]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-02T14:41:27.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:27 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2752716629' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T14:41:27.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:27 smithi113 ceph-mon[36135]: pgmap v321: 97 pgs: 3 creating+peering, 83 active+clean, 11 unknown; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-04-02T14:41:27.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:27 smithi113 ceph-mon[36135]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T14:41:27.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:27 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1030013' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T14:41:27.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:27 smithi113 ceph-mon[36135]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T14:41:27.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:27 smithi113 ceph-mon[36135]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:27.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:27 smithi113 ceph-mon[36135]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-02T14:41:27.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:27 smithi113 ceph-mon[36135]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-02T14:41:27.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:27 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2752716629' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T14:41:27.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:27 smithi204 ceph-mon[35948]: pgmap v321: 97 pgs: 3 creating+peering, 83 active+clean, 11 unknown; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-04-02T14:41:27.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:27 smithi204 ceph-mon[35948]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T14:41:27.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:27 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1030013' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T14:41:27.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:27 smithi204 ceph-mon[35948]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T14:41:27.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:27 smithi204 ceph-mon[35948]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T14:41:27.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:27 smithi204 ceph-mon[35948]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-02T14:41:27.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:27 smithi204 ceph-mon[35948]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-02T14:41:27.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:27 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2752716629' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T14:41:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:27.520+0000 7fb9c1400640 1 -- 172.21.15.62:0/4120812893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb9bc02d370 con 0x7fb9bc109210 2024-04-02T14:41:28.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.055+0000 7fb9b9600640 1 -- 172.21.15.62:0/4120812893 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7fb9a40bee80 con 0x7fb9bc109210 2024-04-02T14:41:28.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.058+0000 7fb9c1400640 1 -- 172.21.15.62:0/4120812893 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb9a00a3c30 msgr2=0x7fb9a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:28.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.058+0000 7fb9c1400640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb9a00a3c30 0x7fb9a00a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0104e0 tx=0x7fb9ac00b410 comp rx=0 tx=0).stop 2024-04-02T14:41:28.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.058+0000 7fb9c1400640 1 -- 172.21.15.62:0/4120812893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9bc109210 msgr2=0x7fb9bc19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:28.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.058+0000 7fb9c1400640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9bc109210 0x7fb9bc19e8e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009230 tx=0x7fb9a4004a90 comp rx=0 tx=0).stop 2024-04-02T14:41:28.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.059+0000 7fb9c1400640 1 -- 172.21.15.62:0/4120812893 shutdown_connections 2024-04-02T14:41:28.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.059+0000 7fb9c1400640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb9a00a3c30 0x7fb9a00a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.059+0000 7fb9c1400640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc109bd0 0x7fb9bc1a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.059+0000 7fb9c1400640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9bc109210 0x7fb9bc19e8e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.059+0000 7fb9c1400640 1 --2- 172.21.15.62:0/4120812893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9bc108020 0x7fb9bc19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.059+0000 7fb9c1400640 1 -- 172.21.15.62:0/4120812893 >> 172.21.15.62:0/4120812893 conn(0x7fb9bc076fd0 msgr2=0x7fb9bc10c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:28.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.059+0000 7fb9c1400640 1 -- 172.21.15.62:0/4120812893 shutdown_connections 2024-04-02T14:41:28.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.059+0000 7fb9c1400640 1 -- 172.21.15.62:0/4120812893 wait complete. 2024-04-02T14:41:28.083 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-02T14:41:28.083 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-02T14:41:28.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.274+0000 7f7b5e600640 1 -- 172.21.15.62:0/1702934594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58104df0 msgr2=0x7f7b58105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:28.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.274+0000 7f7b5e600640 1 --2- 172.21.15.62:0/1702934594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58104df0 0x7f7b58105270 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009900 tx=0x7f7b4402bc30 comp rx=0 tx=0).stop 2024-04-02T14:41:28.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.275+0000 7f7b5e600640 1 -- 172.21.15.62:0/1702934594 shutdown_connections 2024-04-02T14:41:28.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.275+0000 7f7b5e600640 1 --2- 172.21.15.62:0/1702934594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b581057b0 0x7f7b5810c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.275+0000 7f7b5e600640 1 --2- 172.21.15.62:0/1702934594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58104df0 0x7f7b58105270 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.275+0000 7f7b5e600640 1 --2- 172.21.15.62:0/1702934594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58103c00 0x7f7b58104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.275+0000 7f7b5e600640 1 -- 172.21.15.62:0/1702934594 >> 172.21.15.62:0/1702934594 conn(0x7f7b580ff170 msgr2=0x7f7b581015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:28.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.275+0000 7f7b5e600640 1 -- 172.21.15.62:0/1702934594 shutdown_connections 2024-04-02T14:41:28.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.275+0000 7f7b5e600640 1 -- 172.21.15.62:0/1702934594 wait complete. 2024-04-02T14:41:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b5e600640 1 Processor -- start 2024-04-02T14:41:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b5e600640 1 -- start start 2024-04-02T14:41:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b5e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58103c00 0x7f7b58199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b5e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58104df0 0x7f7b5819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b5e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b581057b0 0x7f7b581a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b5e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5810f960 con 0x7f7b58104df0 2024-04-02T14:41:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b5e600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5810f7e0 con 0x7f7b581057b0 2024-04-02T14:41:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b5e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5810fae0 con 0x7f7b58103c00 2024-04-02T14:41:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b57e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58104df0 0x7f7b5819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b57e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58104df0 0x7f7b5819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54898/0 (socket says 172.21.15.62:54898) 2024-04-02T14:41:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b57e00640 1 -- 172.21.15.62:0/2287558539 learned_addr learned my addr 172.21.15.62:0/2287558539 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.276+0000 7f7b5dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b581057b0 0x7f7b581a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.277+0000 7f7b5d200640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58103c00 0x7f7b58199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.277+0000 7f7b5dc00640 1 -- 172.21.15.62:0/2287558539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58103c00 msgr2=0x7f7b58199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.277+0000 7f7b5dc00640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58103c00 0x7f7b58199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.277+0000 7f7b5dc00640 1 -- 172.21.15.62:0/2287558539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58104df0 msgr2=0x7f7b5819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.277+0000 7f7b5dc00640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58104df0 0x7f7b5819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.277+0000 7f7b5dc00640 1 -- 172.21.15.62:0/2287558539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b581a1bf0 con 0x7f7b581057b0 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.277+0000 7f7b57e00640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58104df0 0x7f7b5819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.277+0000 7f7b5dc00640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b581057b0 0x7f7b581a1580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009fd0 tx=0x7f7b5000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.277+0000 7f7b55600640 1 -- 172.21.15.62:0/2287558539 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5000ce20 con 0x7f7b581057b0 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.277+0000 7f7b55600640 1 -- 172.21.15.62:0/2287558539 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7b50004b20 con 0x7f7b581057b0 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.278+0000 7f7b5e600640 1 -- 172.21.15.62:0/2287558539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b581a1ee0 con 0x7f7b581057b0 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.278+0000 7f7b55600640 1 -- 172.21.15.62:0/2287558539 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50010790 con 0x7f7b581057b0 2024-04-02T14:41:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.278+0000 7f7b5e600640 1 -- 172.21.15.62:0/2287558539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b581a23a0 con 0x7f7b581057b0 2024-04-02T14:41:28.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.279+0000 7f7b5e600640 1 -- 172.21.15.62:0/2287558539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b5810af20 con 0x7f7b581057b0 2024-04-02T14:41:28.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.281+0000 7f7b55600640 1 -- 172.21.15.62:0/2287558539 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7b50005020 con 0x7f7b581057b0 2024-04-02T14:41:28.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.282+0000 7f7b55600640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7b400a3c30 0x7f7b400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:28.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.282+0000 7f7b55600640 1 -- 172.21.15.62:0/2287558539 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f7b500d0e60 con 0x7f7b581057b0 2024-04-02T14:41:28.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.282+0000 7f7b5d200640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7b400a3c30 0x7f7b400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:28.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.283+0000 7f7b5d200640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7b400a3c30 0x7f7b400a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005fd0 tx=0x7f7b4c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:28.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.284+0000 7f7b55600640 1 -- 172.21.15.62:0/2287558539 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7b5009d8f0 con 0x7f7b581057b0 2024-04-02T14:41:28.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2752716629' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T14:41:28.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] up:active 2024-04-02T14:41:28.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-mon[32116]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T14:41:28.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-mon[32116]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-02T14:41:28.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-mon[32116]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-04-02T14:41:28.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-mon[32116]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-02T14:41:28.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/4120812893' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T14:41:28.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T14:41:28.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:28 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2752716629' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T14:41:28.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:28 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] up:active 2024-04-02T14:41:28.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:28 smithi113 ceph-mon[36135]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T14:41:28.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:28 smithi113 ceph-mon[36135]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-02T14:41:28.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:28 smithi113 ceph-mon[36135]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-04-02T14:41:28.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:28 smithi113 ceph-mon[36135]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-02T14:41:28.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:28 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/4120812893' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T14:41:28.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:28 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T14:41:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.509+0000 7f7b5e600640 1 -- 172.21.15.62:0/2287558539 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7b58061f40 con 0x7f7b400a3c30 2024-04-02T14:41:28.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:28 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2752716629' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T14:41:28.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:28 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] up:active 2024-04-02T14:41:28.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:28 smithi204 ceph-mon[35948]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T14:41:28.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:28 smithi204 ceph-mon[35948]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-02T14:41:28.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:28 smithi204 ceph-mon[35948]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-04-02T14:41:28.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:28 smithi204 ceph-mon[35948]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-02T14:41:28.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:28 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/4120812893' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T14:41:28.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:28 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T14:41:28.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.547+0000 7f7b55600640 1 -- 172.21.15.62:0/2287558539 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7b58061f40 con 0x7f7b400a3c30 2024-04-02T14:41:28.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 -- 172.21.15.62:0/2287558539 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7b400a3c30 msgr2=0x7f7b400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:28.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7b400a3c30 0x7f7b400a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005fd0 tx=0x7f7b4c00b390 comp rx=0 tx=0).stop 2024-04-02T14:41:28.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 -- 172.21.15.62:0/2287558539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b581057b0 msgr2=0x7f7b581a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:28.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b581057b0 0x7f7b581a1580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009fd0 tx=0x7f7b5000ef10 comp rx=0 tx=0).stop 2024-04-02T14:41:28.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 -- 172.21.15.62:0/2287558539 shutdown_connections 2024-04-02T14:41:28.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7b400a3c30 0x7f7b400a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b581057b0 0x7f7b581a1580 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58104df0 0x7f7b5819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 --2- 172.21.15.62:0/2287558539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58103c00 0x7f7b58199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 -- 172.21.15.62:0/2287558539 >> 172.21.15.62:0/2287558539 conn(0x7f7b580ff170 msgr2=0x7f7b581080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:28.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 -- 172.21.15.62:0/2287558539 shutdown_connections 2024-04-02T14:41:28.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.550+0000 7f7b5e600640 1 -- 172.21.15.62:0/2287558539 wait complete. 2024-04-02T14:41:28.567 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-02T14:41:28.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.781+0000 7fb57e400640 1 -- 172.21.15.62:0/220645371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb578104e10 msgr2=0x7fb578105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:28.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.781+0000 7fb57e400640 1 --2- 172.21.15.62:0/220645371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb578104e10 0x7fb578105290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb564009900 tx=0x7fb56402bc30 comp rx=0 tx=0).stop 2024-04-02T14:41:28.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.781+0000 7fb57e400640 1 -- 172.21.15.62:0/220645371 shutdown_connections 2024-04-02T14:41:28.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.781+0000 7fb57e400640 1 --2- 172.21.15.62:0/220645371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5781057d0 0x7fb57810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.781+0000 7fb57e400640 1 --2- 172.21.15.62:0/220645371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb578104e10 0x7fb578105290 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.781+0000 7fb57e400640 1 --2- 172.21.15.62:0/220645371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb578103c20 0x7fb578104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.781+0000 7fb57e400640 1 -- 172.21.15.62:0/220645371 >> 172.21.15.62:0/220645371 conn(0x7fb5780ff1b0 msgr2=0x7fb5781015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:28.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.782+0000 7fb57e400640 1 -- 172.21.15.62:0/220645371 shutdown_connections 2024-04-02T14:41:28.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.782+0000 7fb57e400640 1 -- 172.21.15.62:0/220645371 wait complete. 2024-04-02T14:41:28.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.782+0000 7fb57e400640 1 Processor -- start 2024-04-02T14:41:28.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb57e400640 1 -- start start 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb57e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb578103c20 0x7fb57819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb57e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb578104e10 0x7fb57819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb57e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5781057d0 0x7fb5781a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb57e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb57810f980 con 0x7fb5781057d0 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb57e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb57810f800 con 0x7fb578103c20 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb57e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb57810fb00 con 0x7fb578104e10 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb57d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb578103c20 0x7fb57819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb57d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb578103c20 0x7fb57819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:36934/0 (socket says 172.21.15.62:36934) 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb57d000640 1 -- 172.21.15.62:0/2484963809 learned_addr learned my addr 172.21.15.62:0/2484963809 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb577e00640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb578104e10 0x7fb57819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.783+0000 7fb57da00640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5781057d0 0x7fb5781a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb57d000640 1 -- 172.21.15.62:0/2484963809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb578104e10 msgr2=0x7fb57819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:28.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb57d000640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb578104e10 0x7fb57819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb57d000640 1 -- 172.21.15.62:0/2484963809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5781057d0 msgr2=0x7fb5781a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:28.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb57d000640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5781057d0 0x7fb5781a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:28.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb57d000640 1 -- 172.21.15.62:0/2484963809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5781a1c30 con 0x7fb578103c20 2024-04-02T14:41:28.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb57da00640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5781057d0 0x7fb5781a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:41:28.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb577e00640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb578104e10 0x7fb57819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:28.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb57d000640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb578103c20 0x7fb57819a010 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00e9c0 tx=0x7fb56c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:28.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb575600640 1 -- 172.21.15.62:0/2484963809 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c00cd80 con 0x7fb578103c20 2024-04-02T14:41:28.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb575600640 1 -- 172.21.15.62:0/2484963809 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb56c0035f0 con 0x7fb578103c20 2024-04-02T14:41:28.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb57e400640 1 -- 172.21.15.62:0/2484963809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5781a1f20 con 0x7fb578103c20 2024-04-02T14:41:28.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb575600640 1 -- 172.21.15.62:0/2484963809 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c010690 con 0x7fb578103c20 2024-04-02T14:41:28.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.784+0000 7fb57e400640 1 -- 172.21.15.62:0/2484963809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5781a2380 con 0x7fb578103c20 2024-04-02T14:41:28.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.785+0000 7fb57e400640 1 -- 172.21.15.62:0/2484963809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb57810af40 con 0x7fb578103c20 2024-04-02T14:41:28.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.787+0000 7fb575600640 1 -- 172.21.15.62:0/2484963809 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb56c0037a0 con 0x7fb578103c20 2024-04-02T14:41:28.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.789+0000 7fb575600640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5600a3bb0 0x7fb5600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:28.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.789+0000 7fb575600640 1 -- 172.21.15.62:0/2484963809 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb56c0d08e0 con 0x7fb578103c20 2024-04-02T14:41:28.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.789+0000 7fb577e00640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5600a3bb0 0x7fb5600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:28.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.789+0000 7fb577e00640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5600a3bb0 0x7fb5600a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb564000c00 tx=0x7fb564005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:28.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:28.791+0000 7fb575600640 1 -- 172.21.15.62:0/2484963809 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb56c09d400 con 0x7fb578103c20 2024-04-02T14:41:28.842 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:41:28.521+0000 7fc53da00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:41:28.842 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:41:28.521+0000 7fc53da00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:41:28.842 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:41:28.521+0000 7fc53da00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:41:28.842 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:41:28.521+0000 7fc53da00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:41:28.843 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:41:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:41:28.521+0000 7fc53da00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:41:29.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.030+0000 7fb57e400640 1 -- 172.21.15.62:0/2484963809 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb578061f40 con 0x7fb5600a3bb0 2024-04-02T14:41:29.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.071+0000 7fb575600640 1 -- 172.21.15.62:0/2484963809 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb578061f40 con 0x7fb5600a3bb0 2024-04-02T14:41:29.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 -- 172.21.15.62:0/2484963809 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5600a3bb0 msgr2=0x7fb5600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:29.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5600a3bb0 0x7fb5600a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb564000c00 tx=0x7fb564005fd0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 -- 172.21.15.62:0/2484963809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb578103c20 msgr2=0x7fb57819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:29.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb578103c20 0x7fb57819a010 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00e9c0 tx=0x7fb56c00ee80 comp rx=0 tx=0).stop 2024-04-02T14:41:29.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 -- 172.21.15.62:0/2484963809 shutdown_connections 2024-04-02T14:41:29.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5600a3bb0 0x7fb5600a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5781057d0 0x7fb5781a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb578104e10 0x7fb57819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 --2- 172.21.15.62:0/2484963809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb578103c20 0x7fb57819a010 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 -- 172.21.15.62:0/2484963809 >> 172.21.15.62:0/2484963809 conn(0x7fb5780ff1b0 msgr2=0x7fb578108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:29.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 -- 172.21.15.62:0/2484963809 shutdown_connections 2024-04-02T14:41:29.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.075+0000 7fb57e400640 1 -- 172.21.15.62:0/2484963809 wait complete. 2024-04-02T14:41:29.097 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-02T14:41:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.300+0000 7fdce8c00640 1 -- 172.21.15.62:0/3208856418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce41058e0 msgr2=0x7fdce410c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.300+0000 7fdce8c00640 1 --2- 172.21.15.62:0/3208856418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce41058e0 0x7fdce410c170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009ca0 tx=0x7fdcdc02c100 comp rx=0 tx=0).stop 2024-04-02T14:41:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.300+0000 7fdce8c00640 1 -- 172.21.15.62:0/3208856418 shutdown_connections 2024-04-02T14:41:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.300+0000 7fdce8c00640 1 --2- 172.21.15.62:0/3208856418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce41058e0 0x7fdce410c170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.300+0000 7fdce8c00640 1 --2- 172.21.15.62:0/3208856418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4104f20 0x7fdce41053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.300+0000 7fdce8c00640 1 --2- 172.21.15.62:0/3208856418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce4103d30 0x7fdce4104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.300+0000 7fdce8c00640 1 -- 172.21.15.62:0/3208856418 >> 172.21.15.62:0/3208856418 conn(0x7fdce40ff2c0 msgr2=0x7fdce41016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.300+0000 7fdce8c00640 1 -- 172.21.15.62:0/3208856418 shutdown_connections 2024-04-02T14:41:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.301+0000 7fdce8c00640 1 -- 172.21.15.62:0/3208856418 wait complete. 2024-04-02T14:41:29.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.301+0000 7fdce8c00640 1 Processor -- start 2024-04-02T14:41:29.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce8c00640 1 -- start start 2024-04-02T14:41:29.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4103d30 0x7fdce419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:29.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce4104f20 0x7fdce419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:29.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce41058e0 0x7fdce41a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:29.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce8c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdce410fae0 con 0x7fdce41058e0 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce8c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdce410f960 con 0x7fdce4104f20 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce8c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdce410fc60 con 0x7fdce4103d30 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4103d30 0x7fdce419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce4104f20 0x7fdce419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4103d30 0x7fdce419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:34972/0 (socket says 172.21.15.62:34972) 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce1e00640 1 -- 172.21.15.62:0/2242911386 learned_addr learned my addr 172.21.15.62:0/2242911386 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.302+0000 7fdce2800640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce41058e0 0x7fdce41a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.303+0000 7fdce1400640 1 -- 172.21.15.62:0/2242911386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4103d30 msgr2=0x7fdce419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.303+0000 7fdce1400640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4103d30 0x7fdce419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.303+0000 7fdce1400640 1 -- 172.21.15.62:0/2242911386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce41058e0 msgr2=0x7fdce41a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.303+0000 7fdce1400640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce41058e0 0x7fdce41a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.303+0000 7fdce1400640 1 -- 172.21.15.62:0/2242911386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce41a1cb0 con 0x7fdce4104f20 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.303+0000 7fdce1400640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce4104f20 0x7fdce419a5d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000e9f0 tx=0x7fdcd000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.303+0000 7fdcdaa00640 1 -- 172.21.15.62:0/2242911386 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd000ccc0 con 0x7fdce4104f20 2024-04-02T14:41:29.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.303+0000 7fdcdaa00640 1 -- 172.21.15.62:0/2242911386 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdcd0005020 con 0x7fdce4104f20 2024-04-02T14:41:29.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.303+0000 7fdcdaa00640 1 -- 172.21.15.62:0/2242911386 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0004b40 con 0x7fdce4104f20 2024-04-02T14:41:29.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.303+0000 7fdce8c00640 1 -- 172.21.15.62:0/2242911386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce41a1fa0 con 0x7fdce4104f20 2024-04-02T14:41:29.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.303+0000 7fdce8c00640 1 -- 172.21.15.62:0/2242911386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce41a2410 con 0x7fdce4104f20 2024-04-02T14:41:29.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.304+0000 7fdce8c00640 1 -- 172.21.15.62:0/2242911386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce410af80 con 0x7fdce4104f20 2024-04-02T14:41:29.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.306+0000 7fdcdaa00640 1 -- 172.21.15.62:0/2242911386 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdcd0004ce0 con 0x7fdce4104f20 2024-04-02T14:41:29.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.308+0000 7fdcdaa00640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdcc80a3bb0 0x7fdcc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:29.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.308+0000 7fdcdaa00640 1 -- 172.21.15.62:0/2242911386 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fdcd00d02c0 con 0x7fdce4104f20 2024-04-02T14:41:29.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.308+0000 7fdce1e00640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdcc80a3bb0 0x7fdcc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:29.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.309+0000 7fdce1e00640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdcc80a3bb0 0x7fdcc80a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40095f0 tx=0x7fdcd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:29.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.312+0000 7fdcdaa00640 1 -- 172.21.15.62:0/2242911386 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdcd009cd50 con 0x7fdce4104f20 2024-04-02T14:41:29.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:29 smithi062 ceph-mon[32116]: pgmap v322: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-02T14:41:29.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:29 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T14:41:29.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:29 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] up:active 2024-04-02T14:41:29.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:29 smithi062 ceph-mon[32116]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T14:41:29.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:29 smithi062 ceph-mon[32116]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-02T14:41:29.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:29 smithi062 ceph-mon[32116]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:creating} 8 up:standby 2024-04-02T14:41:29.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:29 smithi062 ceph-mon[32116]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-04-02T14:41:29.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:29 smithi113 ceph-mon[36135]: pgmap v322: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-02T14:41:29.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:29 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T14:41:29.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:29 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] up:active 2024-04-02T14:41:29.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:29 smithi113 ceph-mon[36135]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T14:41:29.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:29 smithi113 ceph-mon[36135]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-02T14:41:29.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:29 smithi113 ceph-mon[36135]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:creating} 8 up:standby 2024-04-02T14:41:29.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:29 smithi113 ceph-mon[36135]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-04-02T14:41:29.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:29 smithi204 ceph-mon[35948]: pgmap v322: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-02T14:41:29.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:29 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T14:41:29.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:29 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] up:active 2024-04-02T14:41:29.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:29 smithi204 ceph-mon[35948]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T14:41:29.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:29 smithi204 ceph-mon[35948]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-02T14:41:29.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:29 smithi204 ceph-mon[35948]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:creating} 8 up:standby 2024-04-02T14:41:29.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:29 smithi204 ceph-mon[35948]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-04-02T14:41:29.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.556+0000 7fdce8c00640 1 -- 172.21.15.62:0/2242911386 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fdce4061f40 con 0x7fdcc80a3bb0 2024-04-02T14:41:29.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.565+0000 7fdcdaa00640 1 -- 172.21.15.62:0/2242911386 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fdce4061f40 con 0x7fdcc80a3bb0 2024-04-02T14:41:29.564 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/qa/sv_0/00f037b0-558c-4c57-a984-d74dcc5a4c79 2024-04-02T14:41:29.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.568+0000 7fdce8c00640 1 -- 172.21.15.62:0/2242911386 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdcc80a3bb0 msgr2=0x7fdcc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:29.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.568+0000 7fdce8c00640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdcc80a3bb0 0x7fdcc80a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40095f0 tx=0x7fdcd4005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:29.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.568+0000 7fdce8c00640 1 -- 172.21.15.62:0/2242911386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce4104f20 msgr2=0x7fdce419a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:29.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.568+0000 7fdce8c00640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce4104f20 0x7fdce419a5d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000e9f0 tx=0x7fdcd000eeb0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.568+0000 7fdce8c00640 1 -- 172.21.15.62:0/2242911386 shutdown_connections 2024-04-02T14:41:29.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.568+0000 7fdce8c00640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdcc80a3bb0 0x7fdcc80a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.568+0000 7fdce8c00640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce41058e0 0x7fdce41a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.568+0000 7fdce8c00640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce4104f20 0x7fdce419a5d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.568+0000 7fdce8c00640 1 --2- 172.21.15.62:0/2242911386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4103d30 0x7fdce419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.568+0000 7fdce8c00640 1 -- 172.21.15.62:0/2242911386 >> 172.21.15.62:0/2242911386 conn(0x7fdce40ff2c0 msgr2=0x7fdce4108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:29.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.569+0000 7fdce8c00640 1 -- 172.21.15.62:0/2242911386 shutdown_connections 2024-04-02T14:41:29.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.569+0000 7fdce8c00640 1 -- 172.21.15.62:0/2242911386 wait complete. 2024-04-02T14:41:29.587 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-02T14:41:29.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.812+0000 7f2d43000640 1 -- 172.21.15.62:0/2618630563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d3c109890 msgr2=0x7f2d3c10bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:29.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.812+0000 7f2d43000640 1 --2- 172.21.15.62:0/2618630563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d3c109890 0x7f2d3c10bd30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009ca0 tx=0x7f2d3402c180 comp rx=0 tx=0).stop 2024-04-02T14:41:29.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.812+0000 7f2d43000640 1 -- 172.21.15.62:0/2618630563 shutdown_connections 2024-04-02T14:41:29.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.812+0000 7f2d43000640 1 --2- 172.21.15.62:0/2618630563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d3c109890 0x7f2d3c10bd30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.812+0000 7f2d43000640 1 --2- 172.21.15.62:0/2618630563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d3c074500 0x7f2d3c074960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.812+0000 7f2d43000640 1 --2- 172.21.15.62:0/2618630563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c075ae0 0x7f2d3c073f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.812+0000 7f2d43000640 1 -- 172.21.15.62:0/2618630563 >> 172.21.15.62:0/2618630563 conn(0x7f2d3c0fcf70 msgr2=0x7f2d3c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:29.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.812+0000 7f2d43000640 1 -- 172.21.15.62:0/2618630563 shutdown_connections 2024-04-02T14:41:29.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.813+0000 7f2d43000640 1 -- 172.21.15.62:0/2618630563 wait complete. 2024-04-02T14:41:29.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.814+0000 7f2d43000640 1 Processor -- start 2024-04-02T14:41:29.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.814+0000 7f2d43000640 1 -- start start 2024-04-02T14:41:29.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.814+0000 7f2d43000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d3c074500 0x7f2d3c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:29.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.814+0000 7f2d43000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d3c075ae0 0x7f2d3c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:29.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d43000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c109890 0x7f2d3c1a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:29.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d43000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c10f9b0 con 0x7f2d3c109890 2024-04-02T14:41:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d43000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c10f830 con 0x7f2d3c075ae0 2024-04-02T14:41:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d43000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c10fb30 con 0x7f2d3c074500 2024-04-02T14:41:29.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d41200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d3c075ae0 0x7f2d3c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:29.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d41200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d3c075ae0 0x7f2d3c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:36968/0 (socket says 172.21.15.62:36968) 2024-04-02T14:41:29.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d41200640 1 -- 172.21.15.62:0/1475110274 learned_addr learned my addr 172.21.15.62:0/1475110274 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:29.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d41c00640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d3c074500 0x7f2d3c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:29.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d42600640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c109890 0x7f2d3c1a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:29.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d41200640 1 -- 172.21.15.62:0/1475110274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d3c074500 msgr2=0x7f2d3c199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:29.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d41200640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d3c074500 0x7f2d3c199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d41200640 1 -- 172.21.15.62:0/1475110274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c109890 msgr2=0x7f2d3c1a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:29.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d41200640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c109890 0x7f2d3c1a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:29.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.815+0000 7f2d41200640 1 -- 172.21.15.62:0/1475110274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c1a1b80 con 0x7f2d3c075ae0 2024-04-02T14:41:29.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.816+0000 7f2d41200640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d3c075ae0 0x7f2d3c19a4a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800e9f0 tx=0x7f2d2800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:29.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.816+0000 7f2d3a000640 1 -- 172.21.15.62:0/1475110274 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800cd80 con 0x7f2d3c075ae0 2024-04-02T14:41:29.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.816+0000 7f2d3a000640 1 -- 172.21.15.62:0/1475110274 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2d28005520 con 0x7f2d3c075ae0 2024-04-02T14:41:29.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.816+0000 7f2d3a000640 1 -- 172.21.15.62:0/1475110274 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28010690 con 0x7f2d3c075ae0 2024-04-02T14:41:29.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.816+0000 7f2d43000640 1 -- 172.21.15.62:0/1475110274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3c1a1e70 con 0x7f2d3c075ae0 2024-04-02T14:41:29.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.816+0000 7f2d43000640 1 -- 172.21.15.62:0/1475110274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3c1a22e0 con 0x7f2d3c075ae0 2024-04-02T14:41:29.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.817+0000 7f2d43000640 1 -- 172.21.15.62:0/1475110274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d3c108650 con 0x7f2d3c075ae0 2024-04-02T14:41:29.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.819+0000 7f2d3a000640 1 -- 172.21.15.62:0/1475110274 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2d280046c0 con 0x7f2d3c075ae0 2024-04-02T14:41:29.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.820+0000 7f2d3a000640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2d240a3c30 0x7f2d240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:29.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.821+0000 7f2d3a000640 1 -- 172.21.15.62:0/1475110274 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f2d280d06e0 con 0x7f2d3c075ae0 2024-04-02T14:41:29.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.821+0000 7f2d41c00640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2d240a3c30 0x7f2d240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.821+0000 7f2d41c00640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2d240a3c30 0x7f2d240a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009bd0 tx=0x7f2d30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:29.824+0000 7f2d3a000640 1 -- 172.21.15.62:0/1475110274 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2d2809d170 con 0x7f2d3c075ae0 2024-04-02T14:41:30.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.063+0000 7f2d43000640 1 -- 172.21.15.62:0/1475110274 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2d3c061f40 con 0x7f2d240a3c30 2024-04-02T14:41:30.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.096+0000 7f2d3a000640 1 -- 172.21.15.62:0/1475110274 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2d3c061f40 con 0x7f2d240a3c30 2024-04-02T14:41:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 -- 172.21.15.62:0/1475110274 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2d240a3c30 msgr2=0x7f2d240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2d240a3c30 0x7f2d240a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009bd0 tx=0x7f2d30005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 -- 172.21.15.62:0/1475110274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d3c075ae0 msgr2=0x7f2d3c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:30.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d3c075ae0 0x7f2d3c19a4a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800e9f0 tx=0x7f2d2800edf0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 -- 172.21.15.62:0/1475110274 shutdown_connections 2024-04-02T14:41:30.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2d240a3c30 0x7f2d240a60f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c109890 0x7f2d3c1a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d3c075ae0 0x7f2d3c19a4a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 --2- 172.21.15.62:0/1475110274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d3c074500 0x7f2d3c199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 -- 172.21.15.62:0/1475110274 >> 172.21.15.62:0/1475110274 conn(0x7f2d3c0fcf70 msgr2=0x7f2d3c105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:30.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 -- 172.21.15.62:0/1475110274 shutdown_connections 2024-04-02T14:41:30.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.099+0000 7f2d43000640 1 -- 172.21.15.62:0/1475110274 wait complete. 2024-04-02T14:41:30.124 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-02T14:41:30.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.319+0000 7fa32b200640 1 -- 172.21.15.62:0/1415202123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa324103d30 msgr2=0x7fa324104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:30.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.319+0000 7fa32b200640 1 --2- 172.21.15.62:0/1415202123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa324103d30 0x7fa324104150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa314009900 tx=0x7fa31402bc30 comp rx=0 tx=0).stop 2024-04-02T14:41:30.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.320+0000 7fa32b200640 1 -- 172.21.15.62:0/1415202123 shutdown_connections 2024-04-02T14:41:30.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.320+0000 7fa32b200640 1 --2- 172.21.15.62:0/1415202123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3241058e0 0x7fa32410c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.320+0000 7fa32b200640 1 --2- 172.21.15.62:0/1415202123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa324104f20 0x7fa3241053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.320+0000 7fa32b200640 1 --2- 172.21.15.62:0/1415202123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa324103d30 0x7fa324104150 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.320+0000 7fa32b200640 1 -- 172.21.15.62:0/1415202123 >> 172.21.15.62:0/1415202123 conn(0x7fa3240ff2c0 msgr2=0x7fa3241016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:30.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.320+0000 7fa32b200640 1 -- 172.21.15.62:0/1415202123 shutdown_connections 2024-04-02T14:41:30.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.320+0000 7fa32b200640 1 -- 172.21.15.62:0/1415202123 wait complete. 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa32b200640 1 Processor -- start 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa32b200640 1 -- start start 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa32b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa324103d30 0x7fa32419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa32b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa324104f20 0x7fa32419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa32b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3241058e0 0x7fa3241a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa32b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410fa90 con 0x7fa3241058e0 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa32b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410f910 con 0x7fa324103d30 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa32b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410fc10 con 0x7fa324104f20 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa323e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa324103d30 0x7fa32419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa323e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa324103d30 0x7fa32419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:36982/0 (socket says 172.21.15.62:36982) 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa323e00640 1 -- 172.21.15.62:0/2129886286 learned_addr learned my addr 172.21.15.62:0/2129886286 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:30.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa329200640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3241058e0 0x7fa3241a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.321+0000 7fa323400640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa324104f20 0x7fa32419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa323e00640 1 -- 172.21.15.62:0/2129886286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa324104f20 msgr2=0x7fa32419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa323e00640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa324104f20 0x7fa32419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa323e00640 1 -- 172.21.15.62:0/2129886286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3241058e0 msgr2=0x7fa3241a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa323e00640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3241058e0 0x7fa3241a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa323e00640 1 -- 172.21.15.62:0/2129886286 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3241a1d30 con 0x7fa324103d30 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa329200640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3241058e0 0x7fa3241a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa323e00640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa324103d30 0x7fa32419a110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa314000c00 tx=0x7fa314004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa320c00640 1 -- 172.21.15.62:0/2129886286 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314043020 con 0x7fa324103d30 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa320c00640 1 -- 172.21.15.62:0/2129886286 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa314004250 con 0x7fa324103d30 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa320c00640 1 -- 172.21.15.62:0/2129886286 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314003b90 con 0x7fa324103d30 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa32b200640 1 -- 172.21.15.62:0/2129886286 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3241a1fc0 con 0x7fa324103d30 2024-04-02T14:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.322+0000 7fa32b200640 1 -- 172.21.15.62:0/2129886286 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3241a2480 con 0x7fa324103d30 2024-04-02T14:41:30.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.323+0000 7fa32b200640 1 -- 172.21.15.62:0/2129886286 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa32410b120 con 0x7fa324103d30 2024-04-02T14:41:30.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.325+0000 7fa320c00640 1 -- 172.21.15.62:0/2129886286 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa314046020 con 0x7fa324103d30 2024-04-02T14:41:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.327+0000 7fa320c00640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa3080a3cc0 0x7fa3080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:30.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.327+0000 7fa320c00640 1 -- 172.21.15.62:0/2129886286 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa3140f2460 con 0x7fa324103d30 2024-04-02T14:41:30.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.327+0000 7fa323400640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa3080a3cc0 0x7fa3080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:30.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.328+0000 7fa323400640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa3080a3cc0 0x7fa3080a6180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa32419b620 tx=0x7fa310005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.331+0000 7fa320c00640 1 -- 172.21.15.62:0/2129886286 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa31404d020 con 0x7fa324103d30 2024-04-02T14:41:30.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:30 smithi062 ceph-mon[32116]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:30.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:30 smithi062 ceph-mon[32116]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:30.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:30 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] up:active 2024-04-02T14:41:30.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:30 smithi062 ceph-mon[32116]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T14:41:30.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:30 smithi062 ceph-mon[32116]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T14:41:30.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:30 smithi062 ceph-mon[32116]: Cluster is now healthy 2024-04-02T14:41:30.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:30 smithi062 ceph-mon[32116]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:active} 8 up:standby 2024-04-02T14:41:30.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:30 smithi062 ceph-mon[32116]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-02T14:41:30.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:30 smithi062 ceph-mon[32116]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-04-02T14:41:30.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:30 smithi113 ceph-mon[36135]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:30.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:30 smithi113 ceph-mon[36135]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:30.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:30 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] up:active 2024-04-02T14:41:30.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:30 smithi113 ceph-mon[36135]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T14:41:30.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:30 smithi113 ceph-mon[36135]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T14:41:30.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:30 smithi113 ceph-mon[36135]: Cluster is now healthy 2024-04-02T14:41:30.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:30 smithi113 ceph-mon[36135]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:active} 8 up:standby 2024-04-02T14:41:30.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:30 smithi113 ceph-mon[36135]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-02T14:41:30.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:30 smithi113 ceph-mon[36135]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-04-02T14:41:30.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:30 smithi204 ceph-mon[35948]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:30.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:30 smithi204 ceph-mon[35948]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:30.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:30 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] up:active 2024-04-02T14:41:30.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:30 smithi204 ceph-mon[35948]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T14:41:30.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:30 smithi204 ceph-mon[35948]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T14:41:30.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:30 smithi204 ceph-mon[35948]: Cluster is now healthy 2024-04-02T14:41:30.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:30 smithi204 ceph-mon[35948]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:active} 8 up:standby 2024-04-02T14:41:30.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:30 smithi204 ceph-mon[35948]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-02T14:41:30.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:30 smithi204 ceph-mon[35948]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-04-02T14:41:30.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.577+0000 7fa32b200640 1 -- 172.21.15.62:0/2129886286 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fa324061fd0 con 0x7fa3080a3cc0 2024-04-02T14:41:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.590+0000 7fa320c00640 1 -- 172.21.15.62:0/2129886286 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa324061fd0 con 0x7fa3080a3cc0 2024-04-02T14:41:30.589 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8 2024-04-02T14:41:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 -- 172.21.15.62:0/2129886286 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa3080a3cc0 msgr2=0x7fa3080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa3080a3cc0 0x7fa3080a6180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa32419b620 tx=0x7fa310005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 -- 172.21.15.62:0/2129886286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa324103d30 msgr2=0x7fa32419a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa324103d30 0x7fa32419a110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa314000c00 tx=0x7fa314004a30 comp rx=0 tx=0).stop 2024-04-02T14:41:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 -- 172.21.15.62:0/2129886286 shutdown_connections 2024-04-02T14:41:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa3080a3cc0 0x7fa3080a6180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3241058e0 0x7fa3241a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa324104f20 0x7fa32419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 --2- 172.21.15.62:0/2129886286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa324103d30 0x7fa32419a110 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 -- 172.21.15.62:0/2129886286 >> 172.21.15.62:0/2129886286 conn(0x7fa3240ff2c0 msgr2=0x7fa3241082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 -- 172.21.15.62:0/2129886286 shutdown_connections 2024-04-02T14:41:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.593+0000 7fa32b200640 1 -- 172.21.15.62:0/2129886286 wait complete. 2024-04-02T14:41:30.610 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:41:30.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.833+0000 7f9959400640 1 -- 172.21.15.62:0/3434637449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99541003b0 msgr2=0x7f99541027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:30.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.833+0000 7f9959400640 1 --2- 172.21.15.62:0/3434637449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99541003b0 0x7f99541027e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9944009900 tx=0x7f994402bc30 comp rx=0 tx=0).stop 2024-04-02T14:41:30.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.833+0000 7f9959400640 1 -- 172.21.15.62:0/3434637449 shutdown_connections 2024-04-02T14:41:30.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.833+0000 7f9959400640 1 --2- 172.21.15.62:0/3434637449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99541056d0 0x7f9954105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.833+0000 7f9959400640 1 --2- 172.21.15.62:0/3434637449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9954102d20 0x7f9954105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.833+0000 7f9959400640 1 --2- 172.21.15.62:0/3434637449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99541003b0 0x7f99541027e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.833+0000 7f9959400640 1 -- 172.21.15.62:0/3434637449 >> 172.21.15.62:0/3434637449 conn(0x7f99540fbfb0 msgr2=0x7f99540fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:30.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.833+0000 7f9959400640 1 -- 172.21.15.62:0/3434637449 shutdown_connections 2024-04-02T14:41:30.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.833+0000 7f9959400640 1 -- 172.21.15.62:0/3434637449 wait complete. 2024-04-02T14:41:30.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.834+0000 7f9959400640 1 Processor -- start 2024-04-02T14:41:30.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.834+0000 7f9959400640 1 -- start start 2024-04-02T14:41:30.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9959400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99541003b0 0x7f995419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:30.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9959400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9954102d20 0x7f995419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:30.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9959400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99541056d0 0x7f99541a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9959400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9954113eb0 con 0x7f9954102d20 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9959400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9954113d30 con 0x7f99541056d0 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9959400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9954114030 con 0x7f99541003b0 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f994be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9954102d20 0x7f995419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9953200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99541056d0 0x7f99541a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9953200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99541056d0 0x7f99541a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37000/0 (socket says 172.21.15.62:37000) 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9953200640 1 -- 172.21.15.62:0/2068270418 learned_addr learned my addr 172.21.15.62:0/2068270418 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f994be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9954102d20 0x7f995419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54994/0 (socket says 172.21.15.62:54994) 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9952800640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99541003b0 0x7f995419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9953200640 1 -- 172.21.15.62:0/2068270418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99541003b0 msgr2=0x7f995419e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9953200640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99541003b0 0x7f995419e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9953200640 1 -- 172.21.15.62:0/2068270418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9954102d20 msgr2=0x7f995419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9953200640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9954102d20 0x7f995419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.835+0000 7f9953200640 1 -- 172.21.15.62:0/2068270418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99541a6150 con 0x7f99541056d0 2024-04-02T14:41:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.836+0000 7f9953200640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99541056d0 0x7f99541a5ae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f994c00eb10 tx=0x7f994c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.836+0000 7f994b400640 1 -- 172.21.15.62:0/2068270418 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c004b20 con 0x7f99541056d0 2024-04-02T14:41:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.836+0000 7f994b400640 1 -- 172.21.15.62:0/2068270418 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f994c004cc0 con 0x7f99541056d0 2024-04-02T14:41:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.836+0000 7f994b400640 1 -- 172.21.15.62:0/2068270418 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c0107f0 con 0x7f99541056d0 2024-04-02T14:41:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.836+0000 7f9959400640 1 -- 172.21.15.62:0/2068270418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99541a6440 con 0x7f99541056d0 2024-04-02T14:41:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.836+0000 7f9959400640 1 -- 172.21.15.62:0/2068270418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99541a6940 con 0x7f99541056d0 2024-04-02T14:41:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.837+0000 7f9959400640 1 -- 172.21.15.62:0/2068270418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99540fdaa0 con 0x7f99541056d0 2024-04-02T14:41:30.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.839+0000 7f994b400640 1 -- 172.21.15.62:0/2068270418 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f994c0109b0 con 0x7f99541056d0 2024-04-02T14:41:30.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.840+0000 7f994b400640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f99380a3cc0 0x7f99380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:30.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.841+0000 7f994b400640 1 -- 172.21.15.62:0/2068270418 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f994c0d0b20 con 0x7f99541056d0 2024-04-02T14:41:30.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.841+0000 7f9952800640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f99380a3cc0 0x7f99380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:30.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.841+0000 7f9952800640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f99380a3cc0 0x7f99380a6180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9944005ab0 tx=0x7f9944005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:30.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:30.843+0000 7f994b400640 1 -- 172.21.15.62:0/2068270418 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f994c09d450 con 0x7f99541056d0 2024-04-02T14:41:31.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.095+0000 7f9959400640 1 -- 172.21.15.62:0/2068270418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f995402d370 con 0x7f99541056d0 2024-04-02T14:41:31.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.096+0000 7f994b400640 1 -- 172.21.15.62:0/2068270418 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f994c09cd50 con 0x7f99541056d0 2024-04-02T14:41:31.095 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:31.096 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:41:30.090009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24553,24565,24533,24535,24584]},"id":1}]} 2024-04-02T14:41:31.096 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-02T14:41:31.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 -- 172.21.15.62:0/2068270418 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f99380a3cc0 msgr2=0x7f99380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:31.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f99380a3cc0 0x7f99380a6180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9944005ab0 tx=0x7f9944005fd0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 -- 172.21.15.62:0/2068270418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99541056d0 msgr2=0x7f99541a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:31.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99541056d0 0x7f99541a5ae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f994c00eb10 tx=0x7f994c00c2d0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 -- 172.21.15.62:0/2068270418 shutdown_connections 2024-04-02T14:41:31.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f99380a3cc0 0x7f99380a6180 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99541056d0 0x7f99541a5ae0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9954102d20 0x7f995419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 --2- 172.21.15.62:0/2068270418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99541003b0 0x7f995419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 -- 172.21.15.62:0/2068270418 >> 172.21.15.62:0/2068270418 conn(0x7f99540fbfb0 msgr2=0x7f9954100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:31.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 -- 172.21.15.62:0/2068270418 shutdown_connections 2024-04-02T14:41:31.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.099+0000 7f9959400640 1 -- 172.21.15.62:0/2068270418 wait complete. 2024-04-02T14:41:31.120 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:41:31.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.338+0000 7faccc600640 1 -- 172.21.15.62:0/1145929238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc4104f30 msgr2=0x7facc41053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:31.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.338+0000 7faccc600640 1 --2- 172.21.15.62:0/1145929238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc4104f30 0x7facc41053b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7facb0009900 tx=0x7facb002bc30 comp rx=0 tx=0).stop 2024-04-02T14:41:31.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.339+0000 7faccc600640 1 -- 172.21.15.62:0/1145929238 shutdown_connections 2024-04-02T14:41:31.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.339+0000 7faccc600640 1 --2- 172.21.15.62:0/1145929238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc41058f0 0x7facc410c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.339+0000 7faccc600640 1 --2- 172.21.15.62:0/1145929238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc4104f30 0x7facc41053b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.339+0000 7faccc600640 1 --2- 172.21.15.62:0/1145929238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc4103d40 0x7facc4104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.339+0000 7faccc600640 1 -- 172.21.15.62:0/1145929238 >> 172.21.15.62:0/1145929238 conn(0x7facc40ff2f0 msgr2=0x7facc4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:31.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.339+0000 7faccc600640 1 -- 172.21.15.62:0/1145929238 shutdown_connections 2024-04-02T14:41:31.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.339+0000 7faccc600640 1 -- 172.21.15.62:0/1145929238 wait complete. 2024-04-02T14:41:31.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.340+0000 7faccc600640 1 Processor -- start 2024-04-02T14:41:31.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.340+0000 7faccc600640 1 -- start start 2024-04-02T14:41:31.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.340+0000 7faccc600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc4103d40 0x7facc419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:31.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.340+0000 7faccc600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc4104f30 0x7facc419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:31.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7faccc600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc41058f0 0x7facc41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:31.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7faccc600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7facc410faa0 con 0x7facc41058f0 2024-04-02T14:41:31.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7faccc600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7facc410f920 con 0x7facc4104f30 2024-04-02T14:41:31.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7faccc600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7facc410fc20 con 0x7facc4103d40 2024-04-02T14:41:31.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7facca600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc41058f0 0x7facc41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:31.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7facc9200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc4104f30 0x7facc419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:31.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7facca600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc41058f0 0x7facc41a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55006/0 (socket says 172.21.15.62:55006) 2024-04-02T14:41:31.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7facc9200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc4104f30 0x7facc419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37026/0 (socket says 172.21.15.62:37026) 2024-04-02T14:41:31.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7facca600640 1 -- 172.21.15.62:0/2582324913 learned_addr learned my addr 172.21.15.62:0/2582324913 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:31.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7facc9c00640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc4103d40 0x7facc419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:31.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7facc9200640 1 -- 172.21.15.62:0/2582324913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc4103d40 msgr2=0x7facc419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:31.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7facc9200640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc4103d40 0x7facc419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7facc9200640 1 -- 172.21.15.62:0/2582324913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc41058f0 msgr2=0x7facc41a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:31.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7facc9200640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc41058f0 0x7facc41a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.341+0000 7facc9200640 1 -- 172.21.15.62:0/2582324913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc41a1d00 con 0x7facc4104f30 2024-04-02T14:41:31.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.342+0000 7facc9200640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc4104f30 0x7facc419a620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7facb002c170 tx=0x7facb0004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:31.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.342+0000 7facc2000640 1 -- 172.21.15.62:0/2582324913 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0043020 con 0x7facc4104f30 2024-04-02T14:41:31.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.342+0000 7facc2000640 1 -- 172.21.15.62:0/2582324913 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7facb0004250 con 0x7facc4104f30 2024-04-02T14:41:31.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.342+0000 7faccc600640 1 -- 172.21.15.62:0/2582324913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc41a1f90 con 0x7facc4104f30 2024-04-02T14:41:31.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.342+0000 7facc2000640 1 -- 172.21.15.62:0/2582324913 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0003bd0 con 0x7facc4104f30 2024-04-02T14:41:31.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.342+0000 7faccc600640 1 -- 172.21.15.62:0/2582324913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc41a24a0 con 0x7facc4104f30 2024-04-02T14:41:31.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:31 smithi062 ceph-mon[32116]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s wr, 10 op/s 2024-04-02T14:41:31.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:31 smithi062 ceph-mon[32116]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:31.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:31 smithi062 ceph-mon[32116]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:31.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:31 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] up:active 2024-04-02T14:41:31.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:31 smithi062 ceph-mon[32116]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-02T14:41:31.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:31 smithi062 ceph-mon[32116]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-02T14:41:31.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:31 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:41:31.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.343+0000 7faccc600640 1 -- 172.21.15.62:0/2582324913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc410b060 con 0x7facc4104f30 2024-04-02T14:41:31.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.345+0000 7facc2000640 1 -- 172.21.15.62:0/2582324913 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7facb0046020 con 0x7facc4104f30 2024-04-02T14:41:31.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.346+0000 7facc2000640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7facac0a3cc0 0x7facac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:31.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.346+0000 7facc2000640 1 -- 172.21.15.62:0/2582324913 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7facb00f2390 con 0x7facc4104f30 2024-04-02T14:41:31.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.347+0000 7facc9c00640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7facac0a3cc0 0x7facac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:31.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.347+0000 7facc9c00640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7facac0a3cc0 0x7facac0a6180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7facc4104d90 tx=0x7facb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:31.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.350+0000 7facc2000640 1 -- 172.21.15.62:0/2582324913 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7facb004d020 con 0x7facc4104f30 2024-04-02T14:41:31.369 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:41:31 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b[67157]: 2024-04-02T14:41:31.002+0000 7f8f507db700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-02T14:41:31.369 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:41:31 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b[67157]: 2024-04-02T14:41:31.002+0000 7f8f507db700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-02T14:41:31.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:31 smithi113 ceph-mon[36135]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s wr, 10 op/s 2024-04-02T14:41:31.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:31 smithi113 ceph-mon[36135]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:31.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:31 smithi113 ceph-mon[36135]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:31.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:31 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] up:active 2024-04-02T14:41:31.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:31 smithi113 ceph-mon[36135]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-02T14:41:31.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:31 smithi113 ceph-mon[36135]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-02T14:41:31.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:31 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:41:31.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:31 smithi204 ceph-mon[35948]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s wr, 10 op/s 2024-04-02T14:41:31.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:31 smithi204 ceph-mon[35948]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:31.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:31 smithi204 ceph-mon[35948]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:31.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:31 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] up:active 2024-04-02T14:41:31.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:31 smithi204 ceph-mon[35948]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-02T14:41:31.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:31 smithi204 ceph-mon[35948]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-02T14:41:31.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:31 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:41:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.559+0000 7faccc600640 1 -- 172.21.15.62:0/2582324913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facc402d590 con 0x7facc4104f30 2024-04-02T14:41:31.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.561+0000 7facc2000640 1 -- 172.21.15.62:0/2582324913 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22197 (secure 0 0 0) 0x7facb00bedd0 con 0x7facc4104f30 2024-04-02T14:41:31.560 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:31.560 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:41:23.500989+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:41:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.563+0000 7faccc600640 1 -- 172.21.15.62:0/2582324913 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7facac0a3cc0 msgr2=0x7facac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.563+0000 7faccc600640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7facac0a3cc0 0x7facac0a6180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7facc4104d90 tx=0x7facb8005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.563+0000 7faccc600640 1 -- 172.21.15.62:0/2582324913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc4104f30 msgr2=0x7facc419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.563+0000 7faccc600640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc4104f30 0x7facc419a620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7facb002c170 tx=0x7facb0004a30 comp rx=0 tx=0).stop 2024-04-02T14:41:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.564+0000 7faccc600640 1 -- 172.21.15.62:0/2582324913 shutdown_connections 2024-04-02T14:41:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.564+0000 7faccc600640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7facac0a3cc0 0x7facac0a6180 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.564+0000 7faccc600640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc41058f0 0x7facc41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.564+0000 7faccc600640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc4104f30 0x7facc419a620 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.564+0000 7faccc600640 1 --2- 172.21.15.62:0/2582324913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc4103d40 0x7facc419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.564+0000 7faccc600640 1 -- 172.21.15.62:0/2582324913 >> 172.21.15.62:0/2582324913 conn(0x7facc40ff2f0 msgr2=0x7facc4108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:31.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.564+0000 7faccc600640 1 -- 172.21.15.62:0/2582324913 shutdown_connections 2024-04-02T14:41:31.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.564+0000 7faccc600640 1 -- 172.21.15.62:0/2582324913 wait complete. 2024-04-02T14:41:31.584 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-02T14:41:31.584 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T14:41:31.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.813+0000 7f1b94800640 1 -- 172.21.15.62:0/967886016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b8c1058f0 msgr2=0x7f1b8c10c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:31.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.813+0000 7f1b94800640 1 --2- 172.21.15.62:0/967886016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b8c1058f0 0x7f1b8c10c180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009ca0 tx=0x7f1b8802c100 comp rx=0 tx=0).stop 2024-04-02T14:41:31.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.813+0000 7f1b94800640 1 -- 172.21.15.62:0/967886016 shutdown_connections 2024-04-02T14:41:31.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.813+0000 7f1b94800640 1 --2- 172.21.15.62:0/967886016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b8c1058f0 0x7f1b8c10c180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.813+0000 7f1b94800640 1 --2- 172.21.15.62:0/967886016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b8c104f30 0x7f1b8c1053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.813+0000 7f1b94800640 1 --2- 172.21.15.62:0/967886016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b8c103d40 0x7f1b8c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.813+0000 7f1b94800640 1 -- 172.21.15.62:0/967886016 >> 172.21.15.62:0/967886016 conn(0x7f1b8c0ff2f0 msgr2=0x7f1b8c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:31.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.813+0000 7f1b94800640 1 -- 172.21.15.62:0/967886016 shutdown_connections 2024-04-02T14:41:31.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.813+0000 7f1b94800640 1 -- 172.21.15.62:0/967886016 wait complete. 2024-04-02T14:41:31.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.814+0000 7f1b94800640 1 Processor -- start 2024-04-02T14:41:31.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.814+0000 7f1b94800640 1 -- start start 2024-04-02T14:41:31.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.814+0000 7f1b94800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b8c103d40 0x7f1b8c19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:31.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b94800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b8c104f30 0x7f1b8c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b94800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b8c1058f0 0x7f1b8c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b94800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c10faf0 con 0x7f1b8c1058f0 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b94800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c10f970 con 0x7f1b8c103d40 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b94800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c10fc70 con 0x7f1b8c104f30 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b92800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b8c1058f0 0x7f1b8c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b91e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b8c103d40 0x7f1b8c19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b91e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b8c103d40 0x7f1b8c19a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37056/0 (socket says 172.21.15.62:37056) 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b91e00640 1 -- 172.21.15.62:0/2554905275 learned_addr learned my addr 172.21.15.62:0/2554905275 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b92800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b8c1058f0 0x7f1b8c1a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55012/0 (socket says 172.21.15.62:55012) 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b91400640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b8c104f30 0x7f1b8c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b91e00640 1 -- 172.21.15.62:0/2554905275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b8c104f30 msgr2=0x7f1b8c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b91e00640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b8c104f30 0x7f1b8c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b91e00640 1 -- 172.21.15.62:0/2554905275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b8c1058f0 msgr2=0x7f1b8c1a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:31.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b91e00640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b8c1058f0 0x7f1b8c1a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:31.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.815+0000 7f1b91e00640 1 -- 172.21.15.62:0/2554905275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8c1a1d40 con 0x7f1b8c103d40 2024-04-02T14:41:31.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.816+0000 7f1b91e00640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b8c103d40 0x7f1b8c19a120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800e9c0 tx=0x7f1b7800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:31.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.816+0000 7f1b86a00640 1 -- 172.21.15.62:0/2554905275 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800cd80 con 0x7f1b8c103d40 2024-04-02T14:41:31.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.816+0000 7f1b94800640 1 -- 172.21.15.62:0/2554905275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8c1a1fd0 con 0x7f1b8c103d40 2024-04-02T14:41:31.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.816+0000 7f1b86a00640 1 -- 172.21.15.62:0/2554905275 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1b780035f0 con 0x7f1b8c103d40 2024-04-02T14:41:31.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.816+0000 7f1b86a00640 1 -- 172.21.15.62:0/2554905275 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78010690 con 0x7f1b8c103d40 2024-04-02T14:41:31.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.816+0000 7f1b94800640 1 -- 172.21.15.62:0/2554905275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8c1a24e0 con 0x7f1b8c103d40 2024-04-02T14:41:31.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.817+0000 7f1b94800640 1 -- 172.21.15.62:0/2554905275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8c10af90 con 0x7f1b8c103d40 2024-04-02T14:41:31.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.819+0000 7f1b86a00640 1 -- 172.21.15.62:0/2554905275 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1b780037a0 con 0x7f1b8c103d40 2024-04-02T14:41:31.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.820+0000 7f1b86a00640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1b740a3c40 0x7f1b740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:31.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.821+0000 7f1b86a00640 1 -- 172.21.15.62:0/2554905275 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f1b780d08a0 con 0x7f1b8c103d40 2024-04-02T14:41:31.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.821+0000 7f1b91400640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1b740a3c40 0x7f1b740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:31.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.821+0000 7f1b91400640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1b740a3c40 0x7f1b740a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c19b630 tx=0x7f1b80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:31.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:31.825+0000 7f1b86a00640 1 -- 172.21.15.62:0/2554905275 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1b7809d1d0 con 0x7f1b8c103d40 2024-04-02T14:41:32.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.028+0000 7f1b94800640 1 -- 172.21.15.62:0/2554905275 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1b8c061f40 con 0x7f1b740a3c40 2024-04-02T14:41:32.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.048+0000 7f1b86a00640 1 -- 172.21.15.62:0/2554905275 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258766 (secure 0 0 0) 0x7f1b8c061f40 con 0x7f1b740a3c40 2024-04-02T14:41:32.051 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:32.052 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-02T14:41:32.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.054+0000 7f1b94800640 1 -- 172.21.15.62:0/2554905275 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1b740a3c40 msgr2=0x7f1b740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:32.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.054+0000 7f1b94800640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1b740a3c40 0x7f1b740a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c19b630 tx=0x7f1b80005f90 comp rx=0 tx=0).stop 2024-04-02T14:41:32.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.054+0000 7f1b94800640 1 -- 172.21.15.62:0/2554905275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b8c103d40 msgr2=0x7f1b8c19a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:32.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.054+0000 7f1b94800640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b8c103d40 0x7f1b8c19a120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800e9c0 tx=0x7f1b7800ee80 comp rx=0 tx=0).stop 2024-04-02T14:41:32.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.054+0000 7f1b94800640 1 -- 172.21.15.62:0/2554905275 shutdown_connections 2024-04-02T14:41:32.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.054+0000 7f1b94800640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1b740a3c40 0x7f1b740a6100 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.054+0000 7f1b94800640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b8c1058f0 0x7f1b8c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.054+0000 7f1b94800640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b8c104f30 0x7f1b8c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.054+0000 7f1b94800640 1 --2- 172.21.15.62:0/2554905275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b8c103d40 0x7f1b8c19a120 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.054+0000 7f1b94800640 1 -- 172.21.15.62:0/2554905275 >> 172.21.15.62:0/2554905275 conn(0x7f1b8c0ff2f0 msgr2=0x7f1b8c108160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:32.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.054+0000 7f1b94800640 1 -- 172.21.15.62:0/2554905275 shutdown_connections 2024-04-02T14:41:32.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.055+0000 7f1b94800640 1 -- 172.21.15.62:0/2554905275 wait complete. 2024-04-02T14:41:32.074 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-02T14:41:31.355510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":618193,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":141,"num_read_kb":123,"num_write":392,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":403924,"kb_used_data":13172,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537900,"statfs":{"total":1151940427776,"available":1151526809600,"internally_reserved":0,"allocated":13488128,"data_stored":7251701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27825,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":211,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.851583"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506303+0000","last_change":"2024-04-02T14:41:22.546468+0000","last_active":"2024-04-02T14:41:23.506303+0000","last_peered":"2024-04-02T14:41:23.506303+0000","last_clean":"2024-04-02T14:41:23.506303+0000","last_became_active":"2024-04-02T14:41:22.545991+0000","last_became_peered":"2024-04-02T14:41:22.545991+0000","last_unstale":"2024-04-02T14:41:23.506303+0000","last_undegraded":"2024-04-02T14:41:23.506303+0000","last_fullsized":"2024-04-02T14:41:23.506303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:34:24.520404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782453+0000","last_change":"2024-04-02T14:41:22.547738+0000","last_active":"2024-04-02T14:41:23.782453+0000","last_peered":"2024-04-02T14:41:23.782453+0000","last_clean":"2024-04-02T14:41:23.782453+0000","last_became_active":"2024-04-02T14:41:22.542372+0000","last_became_peered":"2024-04-02T14:41:22.542372+0000","last_unstale":"2024-04-02T14:41:23.782453+0000","last_undegraded":"2024-04-02T14:41:23.782453+0000","last_fullsized":"2024-04-02T14:41:23.782453+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:07:26.951136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782322+0000","last_change":"2024-04-02T14:41:22.540173+0000","last_active":"2024-04-02T14:41:23.782322+0000","last_peered":"2024-04-02T14:41:23.782322+0000","last_clean":"2024-04-02T14:41:23.782322+0000","last_became_active":"2024-04-02T14:41:22.536279+0000","last_became_peered":"2024-04-02T14:41:22.536279+0000","last_unstale":"2024-04-02T14:41:23.782322+0000","last_undegraded":"2024-04-02T14:41:23.782322+0000","last_fullsized":"2024-04-02T14:41:23.782322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:43:40.826934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506406+0000","last_change":"2024-04-02T14:41:22.521186+0000","last_active":"2024-04-02T14:41:23.506406+0000","last_peered":"2024-04-02T14:41:23.506406+0000","last_clean":"2024-04-02T14:41:23.506406+0000","last_became_active":"2024-04-02T14:41:22.520825+0000","last_became_peered":"2024-04-02T14:41:22.520825+0000","last_unstale":"2024-04-02T14:41:23.506406+0000","last_undegraded":"2024-04-02T14:41:23.506406+0000","last_fullsized":"2024-04-02T14:41:23.506406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:38:45.652767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793158+0000","last_change":"2024-04-02T14:41:22.554301+0000","last_active":"2024-04-02T14:41:23.793158+0000","last_peered":"2024-04-02T14:41:23.793158+0000","last_clean":"2024-04-02T14:41:23.793158+0000","last_became_active":"2024-04-02T14:41:22.554028+0000","last_became_peered":"2024-04-02T14:41:22.554028+0000","last_unstale":"2024-04-02T14:41:23.793158+0000","last_undegraded":"2024-04-02T14:41:23.793158+0000","last_fullsized":"2024-04-02T14:41:23.793158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:51:59.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792569+0000","last_change":"2024-04-02T14:41:22.543934+0000","last_active":"2024-04-02T14:41:23.792569+0000","last_peered":"2024-04-02T14:41:23.792569+0000","last_clean":"2024-04-02T14:41:23.792569+0000","last_became_active":"2024-04-02T14:41:22.543392+0000","last_became_peered":"2024-04-02T14:41:22.543392+0000","last_unstale":"2024-04-02T14:41:23.792569+0000","last_undegraded":"2024-04-02T14:41:23.792569+0000","last_fullsized":"2024-04-02T14:41:23.792569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:28:30.234356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792603+0000","last_change":"2024-04-02T14:41:22.547460+0000","last_active":"2024-04-02T14:41:23.792603+0000","last_peered":"2024-04-02T14:41:23.792603+0000","last_clean":"2024-04-02T14:41:23.792603+0000","last_became_active":"2024-04-02T14:41:22.547123+0000","last_became_peered":"2024-04-02T14:41:22.547123+0000","last_unstale":"2024-04-02T14:41:23.792603+0000","last_undegraded":"2024-04-02T14:41:23.792603+0000","last_fullsized":"2024-04-02T14:41:23.792603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:24:34.302530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012662+0000","last_change":"2024-04-02T14:41:22.551407+0000","last_active":"2024-04-02T14:41:24.012662+0000","last_peered":"2024-04-02T14:41:24.012662+0000","last_clean":"2024-04-02T14:41:24.012662+0000","last_became_active":"2024-04-02T14:41:22.538304+0000","last_became_peered":"2024-04-02T14:41:22.538304+0000","last_unstale":"2024-04-02T14:41:24.012662+0000","last_undegraded":"2024-04-02T14:41:24.012662+0000","last_fullsized":"2024-04-02T14:41:24.012662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:27.933380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793078+0000","last_change":"2024-04-02T14:41:22.523931+0000","last_active":"2024-04-02T14:41:23.793078+0000","last_peered":"2024-04-02T14:41:23.793078+0000","last_clean":"2024-04-02T14:41:23.793078+0000","last_became_active":"2024-04-02T14:41:22.523622+0000","last_became_peered":"2024-04-02T14:41:22.523622+0000","last_unstale":"2024-04-02T14:41:23.793078+0000","last_undegraded":"2024-04-02T14:41:23.793078+0000","last_fullsized":"2024-04-02T14:41:23.793078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:17.808290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793048+0000","last_change":"2024-04-02T14:41:22.525791+0000","last_active":"2024-04-02T14:41:23.793048+0000","last_peered":"2024-04-02T14:41:23.793048+0000","last_clean":"2024-04-02T14:41:23.793048+0000","last_became_active":"2024-04-02T14:41:22.525434+0000","last_became_peered":"2024-04-02T14:41:22.525434+0000","last_unstale":"2024-04-02T14:41:23.793048+0000","last_undegraded":"2024-04-02T14:41:23.793048+0000","last_fullsized":"2024-04-02T14:41:23.793048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:59.905337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793550+0000","last_change":"2024-04-02T14:41:22.527240+0000","last_active":"2024-04-02T14:41:23.793550+0000","last_peered":"2024-04-02T14:41:23.793550+0000","last_clean":"2024-04-02T14:41:23.793550+0000","last_became_active":"2024-04-02T14:41:22.526412+0000","last_became_peered":"2024-04-02T14:41:22.526412+0000","last_unstale":"2024-04-02T14:41:23.793550+0000","last_undegraded":"2024-04-02T14:41:23.793550+0000","last_fullsized":"2024-04-02T14:41:23.793550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:47.397012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506389+0000","last_change":"2024-04-02T14:41:22.527281+0000","last_active":"2024-04-02T14:41:23.506389+0000","last_peered":"2024-04-02T14:41:23.506389+0000","last_clean":"2024-04-02T14:41:23.506389+0000","last_became_active":"2024-04-02T14:41:22.526946+0000","last_became_peered":"2024-04-02T14:41:22.526946+0000","last_unstale":"2024-04-02T14:41:23.506389+0000","last_undegraded":"2024-04-02T14:41:23.506389+0000","last_fullsized":"2024-04-02T14:41:23.506389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:42.798282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793797+0000","last_change":"2024-04-02T14:41:22.534079+0000","last_active":"2024-04-02T14:41:23.793797+0000","last_peered":"2024-04-02T14:41:23.793797+0000","last_clean":"2024-04-02T14:41:23.793797+0000","last_became_active":"2024-04-02T14:41:22.532233+0000","last_became_peered":"2024-04-02T14:41:22.532233+0000","last_unstale":"2024-04-02T14:41:23.793797+0000","last_undegraded":"2024-04-02T14:41:23.793797+0000","last_fullsized":"2024-04-02T14:41:23.793797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:27:47.122933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.781626+0000","last_change":"2024-04-02T14:41:22.544936+0000","last_active":"2024-04-02T14:41:23.781626+0000","last_peered":"2024-04-02T14:41:23.781626+0000","last_clean":"2024-04-02T14:41:23.781626+0000","last_became_active":"2024-04-02T14:41:22.541805+0000","last_became_peered":"2024-04-02T14:41:22.541805+0000","last_unstale":"2024-04-02T14:41:23.781626+0000","last_undegraded":"2024-04-02T14:41:23.781626+0000","last_fullsized":"2024-04-02T14:41:23.781626+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:23.992807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.781791+0000","last_change":"2024-04-02T14:41:22.534860+0000","last_active":"2024-04-02T14:41:23.781791+0000","last_peered":"2024-04-02T14:41:23.781791+0000","last_clean":"2024-04-02T14:41:23.781791+0000","last_became_active":"2024-04-02T14:41:22.534147+0000","last_became_peered":"2024-04-02T14:41:22.534147+0000","last_unstale":"2024-04-02T14:41:23.781791+0000","last_undegraded":"2024-04-02T14:41:23.781791+0000","last_fullsized":"2024-04-02T14:41:23.781791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:25.249364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792582+0000","last_change":"2024-04-02T14:41:22.537654+0000","last_active":"2024-04-02T14:41:23.792582+0000","last_peered":"2024-04-02T14:41:23.792582+0000","last_clean":"2024-04-02T14:41:23.792582+0000","last_became_active":"2024-04-02T14:41:22.537304+0000","last_became_peered":"2024-04-02T14:41:22.537304+0000","last_unstale":"2024-04-02T14:41:23.792582+0000","last_undegraded":"2024-04-02T14:41:23.792582+0000","last_fullsized":"2024-04-02T14:41:23.792582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:25.599724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506322+0000","last_change":"2024-04-02T14:41:22.546775+0000","last_active":"2024-04-02T14:41:23.506322+0000","last_peered":"2024-04-02T14:41:23.506322+0000","last_clean":"2024-04-02T14:41:23.506322+0000","last_became_active":"2024-04-02T14:41:22.546100+0000","last_became_peered":"2024-04-02T14:41:22.546100+0000","last_unstale":"2024-04-02T14:41:23.506322+0000","last_undegraded":"2024-04-02T14:41:23.506322+0000","last_fullsized":"2024-04-02T14:41:23.506322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:42:02.559335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.795010+0000","last_change":"2024-04-02T14:41:22.543988+0000","last_active":"2024-04-02T14:41:23.795010+0000","last_peered":"2024-04-02T14:41:23.795010+0000","last_clean":"2024-04-02T14:41:23.795010+0000","last_became_active":"2024-04-02T14:41:22.543036+0000","last_became_peered":"2024-04-02T14:41:22.543036+0000","last_unstale":"2024-04-02T14:41:23.795010+0000","last_undegraded":"2024-04-02T14:41:23.795010+0000","last_fullsized":"2024-04-02T14:41:23.795010+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:29.470929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793394+0000","last_change":"2024-04-02T14:41:22.543954+0000","last_active":"2024-04-02T14:41:23.793394+0000","last_peered":"2024-04-02T14:41:23.793394+0000","last_clean":"2024-04-02T14:41:23.793394+0000","last_became_active":"2024-04-02T14:41:22.543624+0000","last_became_peered":"2024-04-02T14:41:22.543624+0000","last_unstale":"2024-04-02T14:41:23.793394+0000","last_undegraded":"2024-04-02T14:41:23.793394+0000","last_fullsized":"2024-04-02T14:41:23.793394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:46.933877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.091677+0000","last_change":"2024-04-02T14:41:21.477341+0000","last_active":"2024-04-02T14:41:29.091677+0000","last_peered":"2024-04-02T14:41:29.091677+0000","last_clean":"2024-04-02T14:41:29.091677+0000","last_became_active":"2024-04-02T14:41:21.476708+0000","last_became_peered":"2024-04-02T14:41:21.476708+0000","last_unstale":"2024-04-02T14:41:29.091677+0000","last_undegraded":"2024-04-02T14:41:29.091677+0000","last_fullsized":"2024-04-02T14:41:29.091677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:34.867570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.013166+0000","last_change":"2024-04-02T14:41:22.551293+0000","last_active":"2024-04-02T14:41:24.013166+0000","last_peered":"2024-04-02T14:41:24.013166+0000","last_clean":"2024-04-02T14:41:24.013166+0000","last_became_active":"2024-04-02T14:41:22.547819+0000","last_became_peered":"2024-04-02T14:41:22.547819+0000","last_unstale":"2024-04-02T14:41:24.013166+0000","last_undegraded":"2024-04-02T14:41:24.013166+0000","last_fullsized":"2024-04-02T14:41:24.013166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:40.393005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.013338+0000","last_change":"2024-04-02T14:41:21.487096+0000","last_active":"2024-04-02T14:41:24.013338+0000","last_peered":"2024-04-02T14:41:24.013338+0000","last_clean":"2024-04-02T14:41:24.013338+0000","last_became_active":"2024-04-02T14:41:21.485883+0000","last_became_peered":"2024-04-02T14:41:21.485883+0000","last_unstale":"2024-04-02T14:41:24.013338+0000","last_undegraded":"2024-04-02T14:41:24.013338+0000","last_fullsized":"2024-04-02T14:41:24.013338+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:24.017671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782636+0000","last_change":"2024-04-02T14:41:22.549234+0000","last_active":"2024-04-02T14:41:23.782636+0000","last_peered":"2024-04-02T14:41:23.782636+0000","last_clean":"2024-04-02T14:41:23.782636+0000","last_became_active":"2024-04-02T14:41:22.548855+0000","last_became_peered":"2024-04-02T14:41:22.548855+0000","last_unstale":"2024-04-02T14:41:23.782636+0000","last_undegraded":"2024-04-02T14:41:23.782636+0000","last_fullsized":"2024-04-02T14:41:23.782636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:05.047383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'11","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.090840+0000","last_change":"2024-04-02T14:41:21.471673+0000","last_active":"2024-04-02T14:41:29.090840+0000","last_peered":"2024-04-02T14:41:29.090840+0000","last_clean":"2024-04-02T14:41:29.090840+0000","last_became_active":"2024-04-02T14:41:21.471292+0000","last_became_peered":"2024-04-02T14:41:21.471292+0000","last_unstale":"2024-04-02T14:41:29.090840+0000","last_undegraded":"2024-04-02T14:41:29.090840+0000","last_fullsized":"2024-04-02T14:41:29.090840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:42:55.665938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792271+0000","last_change":"2024-04-02T14:41:22.537665+0000","last_active":"2024-04-02T14:41:23.792271+0000","last_peered":"2024-04-02T14:41:23.792271+0000","last_clean":"2024-04-02T14:41:23.792271+0000","last_became_active":"2024-04-02T14:41:22.537274+0000","last_became_peered":"2024-04-02T14:41:22.537274+0000","last_unstale":"2024-04-02T14:41:23.792271+0000","last_undegraded":"2024-04-02T14:41:23.792271+0000","last_fullsized":"2024-04-02T14:41:23.792271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:14:39.200735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.090241+0000","last_change":"2024-04-02T14:41:21.466547+0000","last_active":"2024-04-02T14:41:29.090241+0000","last_peered":"2024-04-02T14:41:29.090241+0000","last_clean":"2024-04-02T14:41:29.090241+0000","last_became_active":"2024-04-02T14:41:21.466063+0000","last_became_peered":"2024-04-02T14:41:21.466063+0000","last_unstale":"2024-04-02T14:41:29.090241+0000","last_undegraded":"2024-04-02T14:41:29.090241+0000","last_fullsized":"2024-04-02T14:41:29.090241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:39.201543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793982+0000","last_change":"2024-04-02T14:41:22.523984+0000","last_active":"2024-04-02T14:41:23.793982+0000","last_peered":"2024-04-02T14:41:23.793982+0000","last_clean":"2024-04-02T14:41:23.793982+0000","last_became_active":"2024-04-02T14:41:22.523524+0000","last_became_peered":"2024-04-02T14:41:22.523524+0000","last_unstale":"2024-04-02T14:41:23.793982+0000","last_undegraded":"2024-04-02T14:41:23.793982+0000","last_fullsized":"2024-04-02T14:41:23.793982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:31.542588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.050732+0000","last_change":"2024-04-02T14:41:21.470877+0000","last_active":"2024-04-02T14:41:26.050732+0000","last_peered":"2024-04-02T14:41:26.050732+0000","last_clean":"2024-04-02T14:41:26.050732+0000","last_became_active":"2024-04-02T14:41:21.470046+0000","last_became_peered":"2024-04-02T14:41:21.470046+0000","last_unstale":"2024-04-02T14:41:26.050732+0000","last_undegraded":"2024-04-02T14:41:26.050732+0000","last_fullsized":"2024-04-02T14:41:26.050732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:01.839043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792816+0000","last_change":"2024-04-02T14:41:22.544107+0000","last_active":"2024-04-02T14:41:23.792816+0000","last_peered":"2024-04-02T14:41:23.792816+0000","last_clean":"2024-04-02T14:41:23.792816+0000","last_became_active":"2024-04-02T14:41:22.543739+0000","last_became_peered":"2024-04-02T14:41:22.543739+0000","last_unstale":"2024-04-02T14:41:23.792816+0000","last_undegraded":"2024-04-02T14:41:23.792816+0000","last_fullsized":"2024-04-02T14:41:23.792816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:49:08.921668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.078281+0000","last_change":"2024-04-02T14:41:21.479714+0000","last_active":"2024-04-02T14:41:28.078281+0000","last_peered":"2024-04-02T14:41:28.078281+0000","last_clean":"2024-04-02T14:41:28.078281+0000","last_became_active":"2024-04-02T14:41:21.479329+0000","last_became_peered":"2024-04-02T14:41:21.479329+0000","last_unstale":"2024-04-02T14:41:28.078281+0000","last_undegraded":"2024-04-02T14:41:28.078281+0000","last_fullsized":"2024-04-02T14:41:28.078281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:09.021017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793781+0000","last_change":"2024-04-02T14:41:22.550072+0000","last_active":"2024-04-02T14:41:23.793781+0000","last_peered":"2024-04-02T14:41:23.793781+0000","last_clean":"2024-04-02T14:41:23.793781+0000","last_became_active":"2024-04-02T14:41:22.549722+0000","last_became_peered":"2024-04-02T14:41:22.549722+0000","last_unstale":"2024-04-02T14:41:23.793781+0000","last_undegraded":"2024-04-02T14:41:23.793781+0000","last_fullsized":"2024-04-02T14:41:23.793781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:30:56.376705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:27.063381+0000","last_change":"2024-04-02T14:41:21.484054+0000","last_active":"2024-04-02T14:41:27.063381+0000","last_peered":"2024-04-02T14:41:27.063381+0000","last_clean":"2024-04-02T14:41:27.063381+0000","last_became_active":"2024-04-02T14:41:21.483688+0000","last_became_peered":"2024-04-02T14:41:21.483688+0000","last_unstale":"2024-04-02T14:41:27.063381+0000","last_undegraded":"2024-04-02T14:41:27.063381+0000","last_fullsized":"2024-04-02T14:41:27.063381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:24.114180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012433+0000","last_change":"2024-04-02T14:41:22.553274+0000","last_active":"2024-04-02T14:41:24.012433+0000","last_peered":"2024-04-02T14:41:24.012433+0000","last_clean":"2024-04-02T14:41:24.012433+0000","last_became_active":"2024-04-02T14:41:22.545412+0000","last_became_peered":"2024-04-02T14:41:22.545412+0000","last_unstale":"2024-04-02T14:41:24.012433+0000","last_undegraded":"2024-04-02T14:41:24.012433+0000","last_fullsized":"2024-04-02T14:41:24.012433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:21.917119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.046893+0000","last_change":"2024-04-02T14:41:21.468582+0000","last_active":"2024-04-02T14:41:26.046893+0000","last_peered":"2024-04-02T14:41:26.046893+0000","last_clean":"2024-04-02T14:41:26.046893+0000","last_became_active":"2024-04-02T14:41:21.468081+0000","last_became_peered":"2024-04-02T14:41:21.468081+0000","last_unstale":"2024-04-02T14:41:26.046893+0000","last_undegraded":"2024-04-02T14:41:26.046893+0000","last_fullsized":"2024-04-02T14:41:26.046893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:19.593081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.794640+0000","last_change":"2024-04-02T14:41:22.555627+0000","last_active":"2024-04-02T14:41:23.794640+0000","last_peered":"2024-04-02T14:41:23.794640+0000","last_clean":"2024-04-02T14:41:23.794640+0000","last_became_active":"2024-04-02T14:41:22.555171+0000","last_became_peered":"2024-04-02T14:41:22.555171+0000","last_unstale":"2024-04-02T14:41:23.794640+0000","last_undegraded":"2024-04-02T14:41:23.794640+0000","last_fullsized":"2024-04-02T14:41:23.794640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:21:28.161869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.049246+0000","last_change":"2024-04-02T14:41:21.475004+0000","last_active":"2024-04-02T14:41:26.049246+0000","last_peered":"2024-04-02T14:41:26.049246+0000","last_clean":"2024-04-02T14:41:26.049246+0000","last_became_active":"2024-04-02T14:41:21.474621+0000","last_became_peered":"2024-04-02T14:41:21.474621+0000","last_unstale":"2024-04-02T14:41:26.049246+0000","last_undegraded":"2024-04-02T14:41:26.049246+0000","last_fullsized":"2024-04-02T14:41:26.049246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:38.779276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792608+0000","last_change":"2024-04-02T14:41:22.543100+0000","last_active":"2024-04-02T14:41:23.792608+0000","last_peered":"2024-04-02T14:41:23.792608+0000","last_clean":"2024-04-02T14:41:23.792608+0000","last_became_active":"2024-04-02T14:41:22.542697+0000","last_became_peered":"2024-04-02T14:41:22.542697+0000","last_unstale":"2024-04-02T14:41:23.792608+0000","last_undegraded":"2024-04-02T14:41:23.792608+0000","last_fullsized":"2024-04-02T14:41:23.792608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:50:52.412879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.090683+0000","last_change":"2024-04-02T14:41:21.468626+0000","last_active":"2024-04-02T14:41:29.090683+0000","last_peered":"2024-04-02T14:41:29.090683+0000","last_clean":"2024-04-02T14:41:29.090683+0000","last_became_active":"2024-04-02T14:41:21.468041+0000","last_became_peered":"2024-04-02T14:41:21.468041+0000","last_unstale":"2024-04-02T14:41:29.090683+0000","last_undegraded":"2024-04-02T14:41:29.090683+0000","last_fullsized":"2024-04-02T14:41:29.090683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:55.064033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782267+0000","last_change":"2024-04-02T14:41:22.539584+0000","last_active":"2024-04-02T14:41:23.782267+0000","last_peered":"2024-04-02T14:41:23.782267+0000","last_clean":"2024-04-02T14:41:23.782267+0000","last_became_active":"2024-04-02T14:41:22.539218+0000","last_became_peered":"2024-04-02T14:41:22.539218+0000","last_unstale":"2024-04-02T14:41:23.782267+0000","last_undegraded":"2024-04-02T14:41:23.782267+0000","last_fullsized":"2024-04-02T14:41:23.782267+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:47:56.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.048650+0000","last_change":"2024-04-02T14:41:21.481208+0000","last_active":"2024-04-02T14:41:26.048650+0000","last_peered":"2024-04-02T14:41:26.048650+0000","last_clean":"2024-04-02T14:41:26.048650+0000","last_became_active":"2024-04-02T14:41:21.480782+0000","last_became_peered":"2024-04-02T14:41:21.480782+0000","last_unstale":"2024-04-02T14:41:26.048650+0000","last_undegraded":"2024-04-02T14:41:26.048650+0000","last_fullsized":"2024-04-02T14:41:26.048650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:12.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793209+0000","last_change":"2024-04-02T14:41:22.517397+0000","last_active":"2024-04-02T14:41:23.793209+0000","last_peered":"2024-04-02T14:41:23.793209+0000","last_clean":"2024-04-02T14:41:23.793209+0000","last_became_active":"2024-04-02T14:41:22.517027+0000","last_became_peered":"2024-04-02T14:41:22.517027+0000","last_unstale":"2024-04-02T14:41:23.793209+0000","last_undegraded":"2024-04-02T14:41:23.793209+0000","last_fullsized":"2024-04-02T14:41:23.793209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:53.129949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'17","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.090658+0000","last_change":"2024-04-02T14:41:21.469643+0000","last_active":"2024-04-02T14:41:29.090658+0000","last_peered":"2024-04-02T14:41:29.090658+0000","last_clean":"2024-04-02T14:41:29.090658+0000","last_became_active":"2024-04-02T14:41:21.468730+0000","last_became_peered":"2024-04-02T14:41:21.468730+0000","last_unstale":"2024-04-02T14:41:29.090658+0000","last_undegraded":"2024-04-02T14:41:29.090658+0000","last_fullsized":"2024-04-02T14:41:29.090658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:10.983551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793609+0000","last_change":"2024-04-02T14:41:22.525052+0000","last_active":"2024-04-02T14:41:23.793609+0000","last_peered":"2024-04-02T14:41:23.793609+0000","last_clean":"2024-04-02T14:41:23.793609+0000","last_became_active":"2024-04-02T14:41:22.524691+0000","last_became_peered":"2024-04-02T14:41:22.524691+0000","last_unstale":"2024-04-02T14:41:23.793609+0000","last_undegraded":"2024-04-02T14:41:23.793609+0000","last_fullsized":"2024-04-02T14:41:23.793609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:16.386020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:30.110974+0000","last_change":"2024-04-02T14:41:21.476988+0000","last_active":"2024-04-02T14:41:30.110974+0000","last_peered":"2024-04-02T14:41:30.110974+0000","last_clean":"2024-04-02T14:41:30.110974+0000","last_became_active":"2024-04-02T14:41:21.476549+0000","last_became_peered":"2024-04-02T14:41:21.476549+0000","last_unstale":"2024-04-02T14:41:30.110974+0000","last_undegraded":"2024-04-02T14:41:30.110974+0000","last_fullsized":"2024-04-02T14:41:30.110974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:47.498626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793023+0000","last_change":"2024-04-02T14:41:22.544746+0000","last_active":"2024-04-02T14:41:23.793023+0000","last_peered":"2024-04-02T14:41:23.793023+0000","last_clean":"2024-04-02T14:41:23.793023+0000","last_became_active":"2024-04-02T14:41:22.544334+0000","last_became_peered":"2024-04-02T14:41:22.544334+0000","last_unstale":"2024-04-02T14:41:23.793023+0000","last_undegraded":"2024-04-02T14:41:23.793023+0000","last_fullsized":"2024-04-02T14:41:23.793023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:45:20.094455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.048477+0000","last_change":"2024-04-02T14:41:21.472458+0000","last_active":"2024-04-02T14:41:26.048477+0000","last_peered":"2024-04-02T14:41:26.048477+0000","last_clean":"2024-04-02T14:41:26.048477+0000","last_became_active":"2024-04-02T14:41:21.472105+0000","last_became_peered":"2024-04-02T14:41:21.472105+0000","last_unstale":"2024-04-02T14:41:26.048477+0000","last_undegraded":"2024-04-02T14:41:26.048477+0000","last_fullsized":"2024-04-02T14:41:26.048477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:44.075807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793812+0000","last_change":"2024-04-02T14:41:22.537492+0000","last_active":"2024-04-02T14:41:23.793812+0000","last_peered":"2024-04-02T14:41:23.793812+0000","last_clean":"2024-04-02T14:41:23.793812+0000","last_became_active":"2024-04-02T14:41:22.537067+0000","last_became_peered":"2024-04-02T14:41:22.537067+0000","last_unstale":"2024-04-02T14:41:23.793812+0000","last_undegraded":"2024-04-02T14:41:23.793812+0000","last_fullsized":"2024-04-02T14:41:23.793812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:08.704340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.071934+0000","last_change":"2024-04-02T14:41:21.466017+0000","last_active":"2024-04-02T14:41:28.071934+0000","last_peered":"2024-04-02T14:41:28.071934+0000","last_clean":"2024-04-02T14:41:28.071934+0000","last_became_active":"2024-04-02T14:41:21.465600+0000","last_became_peered":"2024-04-02T14:41:21.465600+0000","last_unstale":"2024-04-02T14:41:28.071934+0000","last_undegraded":"2024-04-02T14:41:28.071934+0000","last_fullsized":"2024-04-02T14:41:28.071934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:52:08.131125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793318+0000","last_change":"2024-04-02T14:41:22.526628+0000","last_active":"2024-04-02T14:41:23.793318+0000","last_peered":"2024-04-02T14:41:23.793318+0000","last_clean":"2024-04-02T14:41:23.793318+0000","last_became_active":"2024-04-02T14:41:22.526283+0000","last_became_peered":"2024-04-02T14:41:22.526283+0000","last_unstale":"2024-04-02T14:41:23.793318+0000","last_undegraded":"2024-04-02T14:41:23.793318+0000","last_fullsized":"2024-04-02T14:41:23.793318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:41.812330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.074839+0000","last_change":"2024-04-02T14:41:21.475853+0000","last_active":"2024-04-02T14:41:28.074839+0000","last_peered":"2024-04-02T14:41:28.074839+0000","last_clean":"2024-04-02T14:41:28.074839+0000","last_became_active":"2024-04-02T14:41:21.475555+0000","last_became_peered":"2024-04-02T14:41:21.475555+0000","last_unstale":"2024-04-02T14:41:28.074839+0000","last_undegraded":"2024-04-02T14:41:28.074839+0000","last_fullsized":"2024-04-02T14:41:28.074839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:49.170906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792966+0000","last_change":"2024-04-02T14:41:22.523848+0000","last_active":"2024-04-02T14:41:23.792966+0000","last_peered":"2024-04-02T14:41:23.792966+0000","last_clean":"2024-04-02T14:41:23.792966+0000","last_became_active":"2024-04-02T14:41:22.523476+0000","last_became_peered":"2024-04-02T14:41:22.523476+0000","last_unstale":"2024-04-02T14:41:23.792966+0000","last_undegraded":"2024-04-02T14:41:23.792966+0000","last_fullsized":"2024-04-02T14:41:23.792966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:47:46.933264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.992747+0000","last_change":"2024-04-02T14:41:21.468556+0000","last_active":"2024-04-02T14:41:24.992747+0000","last_peered":"2024-04-02T14:41:24.992747+0000","last_clean":"2024-04-02T14:41:24.992747+0000","last_became_active":"2024-04-02T14:41:21.468215+0000","last_became_peered":"2024-04-02T14:41:21.468215+0000","last_unstale":"2024-04-02T14:41:24.992747+0000","last_undegraded":"2024-04-02T14:41:24.992747+0000","last_fullsized":"2024-04-02T14:41:24.992747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:04.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506120+0000","last_change":"2024-04-02T14:41:22.524228+0000","last_active":"2024-04-02T14:41:23.506120+0000","last_peered":"2024-04-02T14:41:23.506120+0000","last_clean":"2024-04-02T14:41:23.506120+0000","last_became_active":"2024-04-02T14:41:22.523839+0000","last_became_peered":"2024-04-02T14:41:22.523839+0000","last_unstale":"2024-04-02T14:41:23.506120+0000","last_undegraded":"2024-04-02T14:41:23.506120+0000","last_fullsized":"2024-04-02T14:41:23.506120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:13.094123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.877700+0000","last_change":"2024-04-02T14:41:21.485518+0000","last_active":"2024-04-02T14:41:24.877700+0000","last_peered":"2024-04-02T14:41:24.877700+0000","last_clean":"2024-04-02T14:41:24.877700+0000","last_became_active":"2024-04-02T14:41:21.483016+0000","last_became_peered":"2024-04-02T14:41:21.483016+0000","last_unstale":"2024-04-02T14:41:24.877700+0000","last_undegraded":"2024-04-02T14:41:24.877700+0000","last_fullsized":"2024-04-02T14:41:24.877700+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:57.177512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.507870+0000","last_change":"2024-04-02T14:41:22.542847+0000","last_active":"2024-04-02T14:41:24.507870+0000","last_peered":"2024-04-02T14:41:24.507870+0000","last_clean":"2024-04-02T14:41:24.507870+0000","last_became_active":"2024-04-02T14:41:22.542476+0000","last_became_peered":"2024-04-02T14:41:22.542476+0000","last_unstale":"2024-04-02T14:41:24.507870+0000","last_undegraded":"2024-04-02T14:41:24.507870+0000","last_fullsized":"2024-04-02T14:41:24.507870+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:27:12.102931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.093525+0000","last_change":"2024-04-02T14:41:21.483382+0000","last_active":"2024-04-02T14:41:29.093525+0000","last_peered":"2024-04-02T14:41:29.093525+0000","last_clean":"2024-04-02T14:41:29.093525+0000","last_became_active":"2024-04-02T14:41:21.482996+0000","last_became_peered":"2024-04-02T14:41:21.482996+0000","last_unstale":"2024-04-02T14:41:29.093525+0000","last_undegraded":"2024-04-02T14:41:29.093525+0000","last_fullsized":"2024-04-02T14:41:29.093525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:37.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793725+0000","last_change":"2024-04-02T14:41:22.544121+0000","last_active":"2024-04-02T14:41:23.793725+0000","last_peered":"2024-04-02T14:41:23.793725+0000","last_clean":"2024-04-02T14:41:23.793725+0000","last_became_active":"2024-04-02T14:41:22.543765+0000","last_became_peered":"2024-04-02T14:41:22.543765+0000","last_unstale":"2024-04-02T14:41:23.793725+0000","last_undegraded":"2024-04-02T14:41:23.793725+0000","last_fullsized":"2024-04-02T14:41:23.793725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:45.715217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792807+0000","last_change":"2024-04-02T14:41:22.525853+0000","last_active":"2024-04-02T14:41:23.792807+0000","last_peered":"2024-04-02T14:41:23.792807+0000","last_clean":"2024-04-02T14:41:23.792807+0000","last_became_active":"2024-04-02T14:41:22.525553+0000","last_became_peered":"2024-04-02T14:41:22.525553+0000","last_unstale":"2024-04-02T14:41:23.792807+0000","last_undegraded":"2024-04-02T14:41:23.792807+0000","last_fullsized":"2024-04-02T14:41:23.792807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:25:30.340974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:27.063769+0000","last_change":"2024-04-02T14:41:21.490632+0000","last_active":"2024-04-02T14:41:27.063769+0000","last_peered":"2024-04-02T14:41:27.063769+0000","last_clean":"2024-04-02T14:41:27.063769+0000","last_became_active":"2024-04-02T14:41:21.490208+0000","last_became_peered":"2024-04-02T14:41:21.490208+0000","last_unstale":"2024-04-02T14:41:27.063769+0000","last_undegraded":"2024-04-02T14:41:27.063769+0000","last_fullsized":"2024-04-02T14:41:27.063769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:18.566261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012817+0000","last_change":"2024-04-02T14:41:22.544872+0000","last_active":"2024-04-02T14:41:24.012817+0000","last_peered":"2024-04-02T14:41:24.012817+0000","last_clean":"2024-04-02T14:41:24.012817+0000","last_became_active":"2024-04-02T14:41:22.544268+0000","last_became_peered":"2024-04-02T14:41:22.544268+0000","last_unstale":"2024-04-02T14:41:24.012817+0000","last_undegraded":"2024-04-02T14:41:24.012817+0000","last_fullsized":"2024-04-02T14:41:24.012817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:54.963084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.076419+0000","last_change":"2024-04-02T14:41:21.474411+0000","last_active":"2024-04-02T14:41:28.076419+0000","last_peered":"2024-04-02T14:41:28.076419+0000","last_clean":"2024-04-02T14:41:28.076419+0000","last_became_active":"2024-04-02T14:41:21.473950+0000","last_became_peered":"2024-04-02T14:41:21.473950+0000","last_unstale":"2024-04-02T14:41:28.076419+0000","last_undegraded":"2024-04-02T14:41:28.076419+0000","last_fullsized":"2024-04-02T14:41:28.076419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:46:05.892446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792697+0000","last_change":"2024-04-02T14:40:03.537813+0000","last_active":"2024-04-02T14:41:23.792697+0000","last_peered":"2024-04-02T14:41:23.792697+0000","last_clean":"2024-04-02T14:41:23.792697+0000","last_became_active":"2024-04-02T14:40:03.537223+0000","last_became_peered":"2024-04-02T14:40:03.537223+0000","last_unstale":"2024-04-02T14:41:23.792697+0000","last_undegraded":"2024-04-02T14:41:23.792697+0000","last_fullsized":"2024-04-02T14:41:23.792697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_clean_scrub_stamp":"2024-04-02T14:36:11.612815+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:19:10.346096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792813+0000","last_change":"2024-04-02T14:41:22.537906+0000","last_active":"2024-04-02T14:41:23.792813+0000","last_peered":"2024-04-02T14:41:23.792813+0000","last_clean":"2024-04-02T14:41:23.792813+0000","last_became_active":"2024-04-02T14:41:22.537238+0000","last_became_peered":"2024-04-02T14:41:22.537238+0000","last_unstale":"2024-04-02T14:41:23.792813+0000","last_undegraded":"2024-04-02T14:41:23.792813+0000","last_fullsized":"2024-04-02T14:41:23.792813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:57.089563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:27.059407+0000","last_change":"2024-04-02T14:41:21.466830+0000","last_active":"2024-04-02T14:41:27.059407+0000","last_peered":"2024-04-02T14:41:27.059407+0000","last_clean":"2024-04-02T14:41:27.059407+0000","last_became_active":"2024-04-02T14:41:21.466490+0000","last_became_peered":"2024-04-02T14:41:21.466490+0000","last_unstale":"2024-04-02T14:41:27.059407+0000","last_undegraded":"2024-04-02T14:41:27.059407+0000","last_fullsized":"2024-04-02T14:41:27.059407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:15.597854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.078356+0000","last_change":"2024-04-02T14:41:21.481139+0000","last_active":"2024-04-02T14:41:28.078356+0000","last_peered":"2024-04-02T14:41:28.078356+0000","last_clean":"2024-04-02T14:41:28.078356+0000","last_became_active":"2024-04-02T14:41:21.480754+0000","last_became_peered":"2024-04-02T14:41:21.480754+0000","last_unstale":"2024-04-02T14:41:28.078356+0000","last_undegraded":"2024-04-02T14:41:28.078356+0000","last_fullsized":"2024-04-02T14:41:28.078356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:03.716223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.794700+0000","last_change":"2024-04-02T14:41:22.548483+0000","last_active":"2024-04-02T14:41:23.794700+0000","last_peered":"2024-04-02T14:41:23.794700+0000","last_clean":"2024-04-02T14:41:23.794700+0000","last_became_active":"2024-04-02T14:41:22.547463+0000","last_became_peered":"2024-04-02T14:41:22.547463+0000","last_unstale":"2024-04-02T14:41:23.794700+0000","last_undegraded":"2024-04-02T14:41:23.794700+0000","last_fullsized":"2024-04-02T14:41:23.794700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:33:58.076844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.079853+0000","last_change":"2024-04-02T14:41:21.491105+0000","last_active":"2024-04-02T14:41:28.079853+0000","last_peered":"2024-04-02T14:41:28.079853+0000","last_clean":"2024-04-02T14:41:28.079853+0000","last_became_active":"2024-04-02T14:41:21.490635+0000","last_became_peered":"2024-04-02T14:41:21.490635+0000","last_unstale":"2024-04-02T14:41:28.079853+0000","last_undegraded":"2024-04-02T14:41:28.079853+0000","last_fullsized":"2024-04-02T14:41:28.079853+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:16.531064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.013024+0000","last_change":"2024-04-02T14:41:22.553497+0000","last_active":"2024-04-02T14:41:24.013024+0000","last_peered":"2024-04-02T14:41:24.013024+0000","last_clean":"2024-04-02T14:41:24.013024+0000","last_became_active":"2024-04-02T14:41:22.550474+0000","last_became_peered":"2024-04-02T14:41:22.550474+0000","last_unstale":"2024-04-02T14:41:24.013024+0000","last_undegraded":"2024-04-02T14:41:24.013024+0000","last_fullsized":"2024-04-02T14:41:24.013024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:52.212454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.092740+0000","last_change":"2024-04-02T14:41:21.479516+0000","last_active":"2024-04-02T14:41:29.092740+0000","last_peered":"2024-04-02T14:41:29.092740+0000","last_clean":"2024-04-02T14:41:29.092740+0000","last_became_active":"2024-04-02T14:41:21.478841+0000","last_became_peered":"2024-04-02T14:41:21.478841+0000","last_unstale":"2024-04-02T14:41:29.092740+0000","last_undegraded":"2024-04-02T14:41:29.092740+0000","last_fullsized":"2024-04-02T14:41:29.092740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:39.222316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782763+0000","last_change":"2024-04-02T14:41:22.550095+0000","last_active":"2024-04-02T14:41:23.782763+0000","last_peered":"2024-04-02T14:41:23.782763+0000","last_clean":"2024-04-02T14:41:23.782763+0000","last_became_active":"2024-04-02T14:41:22.547438+0000","last_became_peered":"2024-04-02T14:41:22.547438+0000","last_unstale":"2024-04-02T14:41:23.782763+0000","last_undegraded":"2024-04-02T14:41:23.782763+0000","last_fullsized":"2024-04-02T14:41:23.782763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:03:03.668289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.048298+0000","last_change":"2024-04-02T14:41:21.482926+0000","last_active":"2024-04-02T14:41:26.048298+0000","last_peered":"2024-04-02T14:41:26.048298+0000","last_clean":"2024-04-02T14:41:26.048298+0000","last_became_active":"2024-04-02T14:41:21.482517+0000","last_became_peered":"2024-04-02T14:41:21.482517+0000","last_unstale":"2024-04-02T14:41:26.048298+0000","last_undegraded":"2024-04-02T14:41:26.048298+0000","last_fullsized":"2024-04-02T14:41:26.048298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:43.052390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792578+0000","last_change":"2024-04-02T14:41:22.527758+0000","last_active":"2024-04-02T14:41:23.792578+0000","last_peered":"2024-04-02T14:41:23.792578+0000","last_clean":"2024-04-02T14:41:23.792578+0000","last_became_active":"2024-04-02T14:41:22.526954+0000","last_became_peered":"2024-04-02T14:41:22.526954+0000","last_unstale":"2024-04-02T14:41:23.792578+0000","last_undegraded":"2024-04-02T14:41:23.792578+0000","last_fullsized":"2024-04-02T14:41:23.792578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:34.988661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.078624+0000","last_change":"2024-04-02T14:41:21.487534+0000","last_active":"2024-04-02T14:41:28.078624+0000","last_peered":"2024-04-02T14:41:28.078624+0000","last_clean":"2024-04-02T14:41:28.078624+0000","last_became_active":"2024-04-02T14:41:21.487168+0000","last_became_peered":"2024-04-02T14:41:21.487168+0000","last_unstale":"2024-04-02T14:41:28.078624+0000","last_undegraded":"2024-04-02T14:41:28.078624+0000","last_fullsized":"2024-04-02T14:41:28.078624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:03:20.636028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.505969+0000","last_change":"2024-04-02T14:41:22.539246+0000","last_active":"2024-04-02T14:41:23.505969+0000","last_peered":"2024-04-02T14:41:23.505969+0000","last_clean":"2024-04-02T14:41:23.505969+0000","last_became_active":"2024-04-02T14:41:22.538864+0000","last_became_peered":"2024-04-02T14:41:22.538864+0000","last_unstale":"2024-04-02T14:41:23.505969+0000","last_undegraded":"2024-04-02T14:41:23.505969+0000","last_fullsized":"2024-04-02T14:41:23.505969+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:40:13.418752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.075733+0000","last_change":"2024-04-02T14:41:21.477043+0000","last_active":"2024-04-02T14:41:28.075733+0000","last_peered":"2024-04-02T14:41:28.075733+0000","last_clean":"2024-04-02T14:41:28.075733+0000","last_became_active":"2024-04-02T14:41:21.476580+0000","last_became_peered":"2024-04-02T14:41:21.476580+0000","last_unstale":"2024-04-02T14:41:28.075733+0000","last_undegraded":"2024-04-02T14:41:28.075733+0000","last_fullsized":"2024-04-02T14:41:28.075733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:18.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012673+0000","last_change":"2024-04-02T14:41:22.549758+0000","last_active":"2024-04-02T14:41:24.012673+0000","last_peered":"2024-04-02T14:41:24.012673+0000","last_clean":"2024-04-02T14:41:24.012673+0000","last_became_active":"2024-04-02T14:41:22.545582+0000","last_became_peered":"2024-04-02T14:41:22.545582+0000","last_unstale":"2024-04-02T14:41:24.012673+0000","last_undegraded":"2024-04-02T14:41:24.012673+0000","last_fullsized":"2024-04-02T14:41:24.012673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:42.311921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.046962+0000","last_change":"2024-04-02T14:41:21.472328+0000","last_active":"2024-04-02T14:41:26.046962+0000","last_peered":"2024-04-02T14:41:26.046962+0000","last_clean":"2024-04-02T14:41:26.046962+0000","last_became_active":"2024-04-02T14:41:21.471919+0000","last_became_peered":"2024-04-02T14:41:21.471919+0000","last_unstale":"2024-04-02T14:41:26.046962+0000","last_undegraded":"2024-04-02T14:41:26.046962+0000","last_fullsized":"2024-04-02T14:41:26.046962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:10:37.897485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793268+0000","last_change":"2024-04-02T14:41:22.531509+0000","last_active":"2024-04-02T14:41:23.793268+0000","last_peered":"2024-04-02T14:41:23.793268+0000","last_clean":"2024-04-02T14:41:23.793268+0000","last_became_active":"2024-04-02T14:41:22.531206+0000","last_became_peered":"2024-04-02T14:41:22.531206+0000","last_unstale":"2024-04-02T14:41:23.793268+0000","last_undegraded":"2024-04-02T14:41:23.793268+0000","last_fullsized":"2024-04-02T14:41:23.793268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:39.099177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:30.114614+0000","last_change":"2024-04-02T14:41:21.485271+0000","last_active":"2024-04-02T14:41:30.114614+0000","last_peered":"2024-04-02T14:41:30.114614+0000","last_clean":"2024-04-02T14:41:30.114614+0000","last_became_active":"2024-04-02T14:41:21.484895+0000","last_became_peered":"2024-04-02T14:41:21.484895+0000","last_unstale":"2024-04-02T14:41:30.114614+0000","last_undegraded":"2024-04-02T14:41:30.114614+0000","last_fullsized":"2024-04-02T14:41:30.114614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.552092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793618+0000","last_change":"2024-04-02T14:41:22.524853+0000","last_active":"2024-04-02T14:41:23.793618+0000","last_peered":"2024-04-02T14:41:23.793618+0000","last_clean":"2024-04-02T14:41:23.793618+0000","last_became_active":"2024-04-02T14:41:22.524403+0000","last_became_peered":"2024-04-02T14:41:22.524403+0000","last_unstale":"2024-04-02T14:41:23.793618+0000","last_undegraded":"2024-04-02T14:41:23.793618+0000","last_fullsized":"2024-04-02T14:41:23.793618+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:31.322626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793961+0000","last_change":"2024-04-02T14:41:21.474340+0000","last_active":"2024-04-02T14:41:23.793961+0000","last_peered":"2024-04-02T14:41:23.793961+0000","last_clean":"2024-04-02T14:41:23.793961+0000","last_became_active":"2024-04-02T14:41:21.473934+0000","last_became_peered":"2024-04-02T14:41:21.473934+0000","last_unstale":"2024-04-02T14:41:23.793961+0000","last_undegraded":"2024-04-02T14:41:23.793961+0000","last_fullsized":"2024-04-02T14:41:23.793961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:28:42.711817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792744+0000","last_change":"2024-04-02T14:41:22.543018+0000","last_active":"2024-04-02T14:41:23.792744+0000","last_peered":"2024-04-02T14:41:23.792744+0000","last_clean":"2024-04-02T14:41:23.792744+0000","last_became_active":"2024-04-02T14:41:22.542621+0000","last_became_peered":"2024-04-02T14:41:22.542621+0000","last_unstale":"2024-04-02T14:41:23.792744+0000","last_undegraded":"2024-04-02T14:41:23.792744+0000","last_fullsized":"2024-04-02T14:41:23.792744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:55:03.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.093726+0000","last_change":"2024-04-02T14:41:21.493060+0000","last_active":"2024-04-02T14:41:29.093726+0000","last_peered":"2024-04-02T14:41:29.093726+0000","last_clean":"2024-04-02T14:41:29.093726+0000","last_became_active":"2024-04-02T14:41:21.492627+0000","last_became_peered":"2024-04-02T14:41:21.492627+0000","last_unstale":"2024-04-02T14:41:29.093726+0000","last_undegraded":"2024-04-02T14:41:29.093726+0000","last_fullsized":"2024-04-02T14:41:29.093726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:29:10.443866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793264+0000","last_change":"2024-04-02T14:41:22.527742+0000","last_active":"2024-04-02T14:41:23.793264+0000","last_peered":"2024-04-02T14:41:23.793264+0000","last_clean":"2024-04-02T14:41:23.793264+0000","last_became_active":"2024-04-02T14:41:22.527436+0000","last_became_peered":"2024-04-02T14:41:22.527436+0000","last_unstale":"2024-04-02T14:41:23.793264+0000","last_undegraded":"2024-04-02T14:41:23.793264+0000","last_fullsized":"2024-04-02T14:41:23.793264+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:04:24.754907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506134+0000","last_change":"2024-04-02T14:41:22.526235+0000","last_active":"2024-04-02T14:41:23.506134+0000","last_peered":"2024-04-02T14:41:23.506134+0000","last_clean":"2024-04-02T14:41:23.506134+0000","last_became_active":"2024-04-02T14:41:22.525879+0000","last_became_peered":"2024-04-02T14:41:22.525879+0000","last_unstale":"2024-04-02T14:41:23.506134+0000","last_undegraded":"2024-04-02T14:41:23.506134+0000","last_fullsized":"2024-04-02T14:41:23.506134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:20:54.431223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792579+0000","last_change":"2024-04-02T14:41:22.522498+0000","last_active":"2024-04-02T14:41:23.792579+0000","last_peered":"2024-04-02T14:41:23.792579+0000","last_clean":"2024-04-02T14:41:23.792579+0000","last_became_active":"2024-04-02T14:41:22.522162+0000","last_became_peered":"2024-04-02T14:41:22.522162+0000","last_unstale":"2024-04-02T14:41:23.792579+0000","last_undegraded":"2024-04-02T14:41:23.792579+0000","last_fullsized":"2024-04-02T14:41:23.792579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:42:04.701228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792330+0000","last_change":"2024-04-02T14:41:22.543920+0000","last_active":"2024-04-02T14:41:23.792330+0000","last_peered":"2024-04-02T14:41:23.792330+0000","last_clean":"2024-04-02T14:41:23.792330+0000","last_became_active":"2024-04-02T14:41:22.543552+0000","last_became_peered":"2024-04-02T14:41:22.543552+0000","last_unstale":"2024-04-02T14:41:23.792330+0000","last_undegraded":"2024-04-02T14:41:23.792330+0000","last_fullsized":"2024-04-02T14:41:23.792330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:28.761770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782057+0000","last_change":"2024-04-02T14:41:22.550507+0000","last_active":"2024-04-02T14:41:23.782057+0000","last_peered":"2024-04-02T14:41:23.782057+0000","last_clean":"2024-04-02T14:41:23.782057+0000","last_became_active":"2024-04-02T14:41:22.545824+0000","last_became_peered":"2024-04-02T14:41:22.545824+0000","last_unstale":"2024-04-02T14:41:23.782057+0000","last_undegraded":"2024-04-02T14:41:23.782057+0000","last_fullsized":"2024-04-02T14:41:23.782057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:11.175138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.794810+0000","last_change":"2024-04-02T14:41:22.541362+0000","last_active":"2024-04-02T14:41:23.794810+0000","last_peered":"2024-04-02T14:41:23.794810+0000","last_clean":"2024-04-02T14:41:23.794810+0000","last_became_active":"2024-04-02T14:41:22.540927+0000","last_became_peered":"2024-04-02T14:41:22.540927+0000","last_unstale":"2024-04-02T14:41:23.794810+0000","last_undegraded":"2024-04-02T14:41:23.794810+0000","last_fullsized":"2024-04-02T14:41:23.794810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:43:00.746274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506162+0000","last_change":"2024-04-02T14:41:22.527146+0000","last_active":"2024-04-02T14:41:23.506162+0000","last_peered":"2024-04-02T14:41:23.506162+0000","last_clean":"2024-04-02T14:41:23.506162+0000","last_became_active":"2024-04-02T14:41:22.526810+0000","last_became_peered":"2024-04-02T14:41:22.526810+0000","last_unstale":"2024-04-02T14:41:23.506162+0000","last_undegraded":"2024-04-02T14:41:23.506162+0000","last_fullsized":"2024-04-02T14:41:23.506162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:18:46.464701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793579+0000","last_change":"2024-04-02T14:41:22.524458+0000","last_active":"2024-04-02T14:41:23.793579+0000","last_peered":"2024-04-02T14:41:23.793579+0000","last_clean":"2024-04-02T14:41:23.793579+0000","last_became_active":"2024-04-02T14:41:22.524138+0000","last_became_peered":"2024-04-02T14:41:22.524138+0000","last_unstale":"2024-04-02T14:41:23.793579+0000","last_undegraded":"2024-04-02T14:41:23.793579+0000","last_fullsized":"2024-04-02T14:41:23.793579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:50:04.667964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012817+0000","last_change":"2024-04-02T14:41:22.550905+0000","last_active":"2024-04-02T14:41:24.012817+0000","last_peered":"2024-04-02T14:41:24.012817+0000","last_clean":"2024-04-02T14:41:24.012817+0000","last_became_active":"2024-04-02T14:41:22.548518+0000","last_became_peered":"2024-04-02T14:41:22.548518+0000","last_unstale":"2024-04-02T14:41:24.012817+0000","last_undegraded":"2024-04-02T14:41:24.012817+0000","last_fullsized":"2024-04-02T14:41:24.012817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:46:56.717745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792658+0000","last_change":"2024-04-02T14:41:22.542319+0000","last_active":"2024-04-02T14:41:23.792658+0000","last_peered":"2024-04-02T14:41:23.792658+0000","last_clean":"2024-04-02T14:41:23.792658+0000","last_became_active":"2024-04-02T14:41:22.541959+0000","last_became_peered":"2024-04-02T14:41:22.541959+0000","last_unstale":"2024-04-02T14:41:23.792658+0000","last_undegraded":"2024-04-02T14:41:23.792658+0000","last_fullsized":"2024-04-02T14:41:23.792658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:49.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012482+0000","last_change":"2024-04-02T14:41:22.549932+0000","last_active":"2024-04-02T14:41:24.012482+0000","last_peered":"2024-04-02T14:41:24.012482+0000","last_clean":"2024-04-02T14:41:24.012482+0000","last_became_active":"2024-04-02T14:41:22.549231+0000","last_became_peered":"2024-04-02T14:41:22.549231+0000","last_unstale":"2024-04-02T14:41:24.012482+0000","last_undegraded":"2024-04-02T14:41:24.012482+0000","last_fullsized":"2024-04-02T14:41:24.012482+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:01.485343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792984+0000","last_change":"2024-04-02T14:41:22.544712+0000","last_active":"2024-04-02T14:41:23.792984+0000","last_peered":"2024-04-02T14:41:23.792984+0000","last_clean":"2024-04-02T14:41:23.792984+0000","last_became_active":"2024-04-02T14:41:22.544043+0000","last_became_peered":"2024-04-02T14:41:22.544043+0000","last_unstale":"2024-04-02T14:41:23.792984+0000","last_undegraded":"2024-04-02T14:41:23.792984+0000","last_fullsized":"2024-04-02T14:41:23.792984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:07.931377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793631+0000","last_change":"2024-04-02T14:41:22.542364+0000","last_active":"2024-04-02T14:41:23.793631+0000","last_peered":"2024-04-02T14:41:23.793631+0000","last_clean":"2024-04-02T14:41:23.793631+0000","last_became_active":"2024-04-02T14:41:22.541929+0000","last_became_peered":"2024-04-02T14:41:22.541929+0000","last_unstale":"2024-04-02T14:41:23.793631+0000","last_undegraded":"2024-04-02T14:41:23.793631+0000","last_fullsized":"2024-04-02T14:41:23.793631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:45.716132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.505990+0000","last_change":"2024-04-02T14:41:22.525295+0000","last_active":"2024-04-02T14:41:23.505990+0000","last_peered":"2024-04-02T14:41:23.505990+0000","last_clean":"2024-04-02T14:41:23.505990+0000","last_became_active":"2024-04-02T14:41:22.524484+0000","last_became_peered":"2024-04-02T14:41:22.524484+0000","last_unstale":"2024-04-02T14:41:23.505990+0000","last_undegraded":"2024-04-02T14:41:23.505990+0000","last_fullsized":"2024-04-02T14:41:23.505990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:01.819082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27708,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":210,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":483328,"data_stored":129184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182,"ondisk_log_size":182,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285582,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":452184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547219,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1051274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710745,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":452114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972384,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":452492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266725,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1044215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430250,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":451831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":473639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":453170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":474250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117050,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":451276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":451658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691888,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1043598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":46590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":22631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:41:32.079 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T14:41:32.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:32 smithi062 ceph-mon[32116]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:32.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:32 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2068270418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:32.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:32 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2582324913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:32.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.343+0000 7f7771c00640 1 -- 172.21.15.62:0/228338279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c1034f0 msgr2=0x7f776c109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:32.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.343+0000 7f7771c00640 1 --2- 172.21.15.62:0/228338279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c1034f0 0x7f776c109d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7764009ca0 tx=0x7f776402c100 comp rx=0 tx=0).stop 2024-04-02T14:41:32.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.343+0000 7f7771c00640 1 -- 172.21.15.62:0/228338279 shutdown_connections 2024-04-02T14:41:32.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.343+0000 7f7771c00640 1 --2- 172.21.15.62:0/228338279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c1034f0 0x7f776c109d80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.343+0000 7f7771c00640 1 --2- 172.21.15.62:0/228338279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f776c102b30 0x7f776c102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.343+0000 7f7771c00640 1 --2- 172.21.15.62:0/228338279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f776c101940 0x7f776c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.343+0000 7f7771c00640 1 -- 172.21.15.62:0/228338279 >> 172.21.15.62:0/228338279 conn(0x7f776c0fceb0 msgr2=0x7f776c0ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:32.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.343+0000 7f7771c00640 1 -- 172.21.15.62:0/228338279 shutdown_connections 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.343+0000 7f7771c00640 1 -- 172.21.15.62:0/228338279 wait complete. 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.344+0000 7f7771c00640 1 Processor -- start 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.344+0000 7f7771c00640 1 -- start start 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.344+0000 7f7771c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f776c101940 0x7f776c197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.344+0000 7f7771c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f776c102b30 0x7f776c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f7771c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c1034f0 0x7f776c19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f7771c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f776c10d6f0 con 0x7f776c101940 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f7771c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f776c10d570 con 0x7f776c102b30 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f7771c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f776c10d870 con 0x7f776c1034f0 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f776be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f776c101940 0x7f776c197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f776be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f776c101940 0x7f776c197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60172/0 (socket says 172.21.15.62:60172) 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f776be00640 1 -- 172.21.15.62:0/2828667969 learned_addr learned my addr 172.21.15.62:0/2828667969 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:32.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f7771200640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c1034f0 0x7f776c19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:32.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f776b400640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f776c102b30 0x7f776c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:32.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f776be00640 1 -- 172.21.15.62:0/2828667969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c1034f0 msgr2=0x7f776c19f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:32.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f776be00640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c1034f0 0x7f776c19f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f776be00640 1 -- 172.21.15.62:0/2828667969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f776c102b30 msgr2=0x7f776c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:32.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f776be00640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f776c102b30 0x7f776c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.345+0000 7f776be00640 1 -- 172.21.15.62:0/2828667969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c19f8f0 con 0x7f776c101940 2024-04-02T14:41:32.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.346+0000 7f776be00640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f776c101940 0x7f776c197cb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f775c00e9c0 tx=0x7f775c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:32.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.346+0000 7f7768c00640 1 -- 172.21.15.62:0/2828667969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c00cd80 con 0x7f776c101940 2024-04-02T14:41:32.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.346+0000 7f7768c00640 1 -- 172.21.15.62:0/2828667969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f775c0035f0 con 0x7f776c101940 2024-04-02T14:41:32.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.346+0000 7f7771c00640 1 -- 172.21.15.62:0/2828667969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c19fbe0 con 0x7f776c101940 2024-04-02T14:41:32.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.346+0000 7f7768c00640 1 -- 172.21.15.62:0/2828667969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c010690 con 0x7f776c101940 2024-04-02T14:41:32.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.346+0000 7f7771c00640 1 -- 172.21.15.62:0/2828667969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c1a0130 con 0x7f776c101940 2024-04-02T14:41:32.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.347+0000 7f7771c00640 1 -- 172.21.15.62:0/2828667969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776c108c60 con 0x7f776c101940 2024-04-02T14:41:32.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.349+0000 7f7768c00640 1 -- 172.21.15.62:0/2828667969 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f775c0037a0 con 0x7f776c101940 2024-04-02T14:41:32.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.350+0000 7f7768c00640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f77500a3c40 0x7f77500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:32.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.351+0000 7f7768c00640 1 -- 172.21.15.62:0/2828667969 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f775c0d08f0 con 0x7f776c101940 2024-04-02T14:41:32.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.351+0000 7f776b400640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f77500a3c40 0x7f77500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:32.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.352+0000 7f776b400640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f77500a3c40 0x7f77500a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f776c1991b0 tx=0x7f775800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:32.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.355+0000 7f7768c00640 1 -- 172.21.15.62:0/2828667969 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f775c09d220 con 0x7f776c101940 2024-04-02T14:41:32.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:32 smithi113 ceph-mon[36135]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:32.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:32 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2068270418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:32.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:32 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2582324913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:32.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:32 smithi204 ceph-mon[35948]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:41:32.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:32 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2068270418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:41:32.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:32 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2582324913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.558+0000 7f7771c00640 1 -- 172.21.15.62:0/2828667969 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f776c061f40 con 0x7f77500a3c40 2024-04-02T14:41:32.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.582+0000 7f7768c00640 1 -- 172.21.15.62:0/2828667969 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258766 (secure 0 0 0) 0x7f776c061f40 con 0x7f77500a3c40 2024-04-02T14:41:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:32.585 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-02T14:41:32.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 -- 172.21.15.62:0/2828667969 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f77500a3c40 msgr2=0x7f77500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:32.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f77500a3c40 0x7f77500a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f776c1991b0 tx=0x7f775800b410 comp rx=0 tx=0).stop 2024-04-02T14:41:32.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 -- 172.21.15.62:0/2828667969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f776c101940 msgr2=0x7f776c197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:32.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f776c101940 0x7f776c197cb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f775c00e9c0 tx=0x7f775c00ee80 comp rx=0 tx=0).stop 2024-04-02T14:41:32.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 -- 172.21.15.62:0/2828667969 shutdown_connections 2024-04-02T14:41:32.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f77500a3c40 0x7f77500a6100 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c1034f0 0x7f776c19f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f776c102b30 0x7f776c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 --2- 172.21.15.62:0/2828667969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f776c101940 0x7f776c197cb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:32.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 -- 172.21.15.62:0/2828667969 >> 172.21.15.62:0/2828667969 conn(0x7f776c0fceb0 msgr2=0x7f776c105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:32.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 -- 172.21.15.62:0/2828667969 shutdown_connections 2024-04-02T14:41:32.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:32.587+0000 7f7771c00640 1 -- 172.21.15.62:0/2828667969 wait complete. 2024-04-02T14:41:32.607 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-02T14:41:31.355510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":618193,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":141,"num_read_kb":123,"num_write":392,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":403924,"kb_used_data":13172,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537900,"statfs":{"total":1151940427776,"available":1151526809600,"internally_reserved":0,"allocated":13488128,"data_stored":7251701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27825,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":211,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.851583"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506303+0000","last_change":"2024-04-02T14:41:22.546468+0000","last_active":"2024-04-02T14:41:23.506303+0000","last_peered":"2024-04-02T14:41:23.506303+0000","last_clean":"2024-04-02T14:41:23.506303+0000","last_became_active":"2024-04-02T14:41:22.545991+0000","last_became_peered":"2024-04-02T14:41:22.545991+0000","last_unstale":"2024-04-02T14:41:23.506303+0000","last_undegraded":"2024-04-02T14:41:23.506303+0000","last_fullsized":"2024-04-02T14:41:23.506303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:34:24.520404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782453+0000","last_change":"2024-04-02T14:41:22.547738+0000","last_active":"2024-04-02T14:41:23.782453+0000","last_peered":"2024-04-02T14:41:23.782453+0000","last_clean":"2024-04-02T14:41:23.782453+0000","last_became_active":"2024-04-02T14:41:22.542372+0000","last_became_peered":"2024-04-02T14:41:22.542372+0000","last_unstale":"2024-04-02T14:41:23.782453+0000","last_undegraded":"2024-04-02T14:41:23.782453+0000","last_fullsized":"2024-04-02T14:41:23.782453+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:07:26.951136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782322+0000","last_change":"2024-04-02T14:41:22.540173+0000","last_active":"2024-04-02T14:41:23.782322+0000","last_peered":"2024-04-02T14:41:23.782322+0000","last_clean":"2024-04-02T14:41:23.782322+0000","last_became_active":"2024-04-02T14:41:22.536279+0000","last_became_peered":"2024-04-02T14:41:22.536279+0000","last_unstale":"2024-04-02T14:41:23.782322+0000","last_undegraded":"2024-04-02T14:41:23.782322+0000","last_fullsized":"2024-04-02T14:41:23.782322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:43:40.826934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506406+0000","last_change":"2024-04-02T14:41:22.521186+0000","last_active":"2024-04-02T14:41:23.506406+0000","last_peered":"2024-04-02T14:41:23.506406+0000","last_clean":"2024-04-02T14:41:23.506406+0000","last_became_active":"2024-04-02T14:41:22.520825+0000","last_became_peered":"2024-04-02T14:41:22.520825+0000","last_unstale":"2024-04-02T14:41:23.506406+0000","last_undegraded":"2024-04-02T14:41:23.506406+0000","last_fullsized":"2024-04-02T14:41:23.506406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:38:45.652767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793158+0000","last_change":"2024-04-02T14:41:22.554301+0000","last_active":"2024-04-02T14:41:23.793158+0000","last_peered":"2024-04-02T14:41:23.793158+0000","last_clean":"2024-04-02T14:41:23.793158+0000","last_became_active":"2024-04-02T14:41:22.554028+0000","last_became_peered":"2024-04-02T14:41:22.554028+0000","last_unstale":"2024-04-02T14:41:23.793158+0000","last_undegraded":"2024-04-02T14:41:23.793158+0000","last_fullsized":"2024-04-02T14:41:23.793158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:51:59.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792569+0000","last_change":"2024-04-02T14:41:22.543934+0000","last_active":"2024-04-02T14:41:23.792569+0000","last_peered":"2024-04-02T14:41:23.792569+0000","last_clean":"2024-04-02T14:41:23.792569+0000","last_became_active":"2024-04-02T14:41:22.543392+0000","last_became_peered":"2024-04-02T14:41:22.543392+0000","last_unstale":"2024-04-02T14:41:23.792569+0000","last_undegraded":"2024-04-02T14:41:23.792569+0000","last_fullsized":"2024-04-02T14:41:23.792569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:28:30.234356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792603+0000","last_change":"2024-04-02T14:41:22.547460+0000","last_active":"2024-04-02T14:41:23.792603+0000","last_peered":"2024-04-02T14:41:23.792603+0000","last_clean":"2024-04-02T14:41:23.792603+0000","last_became_active":"2024-04-02T14:41:22.547123+0000","last_became_peered":"2024-04-02T14:41:22.547123+0000","last_unstale":"2024-04-02T14:41:23.792603+0000","last_undegraded":"2024-04-02T14:41:23.792603+0000","last_fullsized":"2024-04-02T14:41:23.792603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:24:34.302530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012662+0000","last_change":"2024-04-02T14:41:22.551407+0000","last_active":"2024-04-02T14:41:24.012662+0000","last_peered":"2024-04-02T14:41:24.012662+0000","last_clean":"2024-04-02T14:41:24.012662+0000","last_became_active":"2024-04-02T14:41:22.538304+0000","last_became_peered":"2024-04-02T14:41:22.538304+0000","last_unstale":"2024-04-02T14:41:24.012662+0000","last_undegraded":"2024-04-02T14:41:24.012662+0000","last_fullsized":"2024-04-02T14:41:24.012662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:27.933380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793078+0000","last_change":"2024-04-02T14:41:22.523931+0000","last_active":"2024-04-02T14:41:23.793078+0000","last_peered":"2024-04-02T14:41:23.793078+0000","last_clean":"2024-04-02T14:41:23.793078+0000","last_became_active":"2024-04-02T14:41:22.523622+0000","last_became_peered":"2024-04-02T14:41:22.523622+0000","last_unstale":"2024-04-02T14:41:23.793078+0000","last_undegraded":"2024-04-02T14:41:23.793078+0000","last_fullsized":"2024-04-02T14:41:23.793078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:17.808290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793048+0000","last_change":"2024-04-02T14:41:22.525791+0000","last_active":"2024-04-02T14:41:23.793048+0000","last_peered":"2024-04-02T14:41:23.793048+0000","last_clean":"2024-04-02T14:41:23.793048+0000","last_became_active":"2024-04-02T14:41:22.525434+0000","last_became_peered":"2024-04-02T14:41:22.525434+0000","last_unstale":"2024-04-02T14:41:23.793048+0000","last_undegraded":"2024-04-02T14:41:23.793048+0000","last_fullsized":"2024-04-02T14:41:23.793048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:59.905337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793550+0000","last_change":"2024-04-02T14:41:22.527240+0000","last_active":"2024-04-02T14:41:23.793550+0000","last_peered":"2024-04-02T14:41:23.793550+0000","last_clean":"2024-04-02T14:41:23.793550+0000","last_became_active":"2024-04-02T14:41:22.526412+0000","last_became_peered":"2024-04-02T14:41:22.526412+0000","last_unstale":"2024-04-02T14:41:23.793550+0000","last_undegraded":"2024-04-02T14:41:23.793550+0000","last_fullsized":"2024-04-02T14:41:23.793550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:47.397012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506389+0000","last_change":"2024-04-02T14:41:22.527281+0000","last_active":"2024-04-02T14:41:23.506389+0000","last_peered":"2024-04-02T14:41:23.506389+0000","last_clean":"2024-04-02T14:41:23.506389+0000","last_became_active":"2024-04-02T14:41:22.526946+0000","last_became_peered":"2024-04-02T14:41:22.526946+0000","last_unstale":"2024-04-02T14:41:23.506389+0000","last_undegraded":"2024-04-02T14:41:23.506389+0000","last_fullsized":"2024-04-02T14:41:23.506389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:42.798282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793797+0000","last_change":"2024-04-02T14:41:22.534079+0000","last_active":"2024-04-02T14:41:23.793797+0000","last_peered":"2024-04-02T14:41:23.793797+0000","last_clean":"2024-04-02T14:41:23.793797+0000","last_became_active":"2024-04-02T14:41:22.532233+0000","last_became_peered":"2024-04-02T14:41:22.532233+0000","last_unstale":"2024-04-02T14:41:23.793797+0000","last_undegraded":"2024-04-02T14:41:23.793797+0000","last_fullsized":"2024-04-02T14:41:23.793797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:27:47.122933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.781626+0000","last_change":"2024-04-02T14:41:22.544936+0000","last_active":"2024-04-02T14:41:23.781626+0000","last_peered":"2024-04-02T14:41:23.781626+0000","last_clean":"2024-04-02T14:41:23.781626+0000","last_became_active":"2024-04-02T14:41:22.541805+0000","last_became_peered":"2024-04-02T14:41:22.541805+0000","last_unstale":"2024-04-02T14:41:23.781626+0000","last_undegraded":"2024-04-02T14:41:23.781626+0000","last_fullsized":"2024-04-02T14:41:23.781626+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:23.992807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.781791+0000","last_change":"2024-04-02T14:41:22.534860+0000","last_active":"2024-04-02T14:41:23.781791+0000","last_peered":"2024-04-02T14:41:23.781791+0000","last_clean":"2024-04-02T14:41:23.781791+0000","last_became_active":"2024-04-02T14:41:22.534147+0000","last_became_peered":"2024-04-02T14:41:22.534147+0000","last_unstale":"2024-04-02T14:41:23.781791+0000","last_undegraded":"2024-04-02T14:41:23.781791+0000","last_fullsized":"2024-04-02T14:41:23.781791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:25.249364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792582+0000","last_change":"2024-04-02T14:41:22.537654+0000","last_active":"2024-04-02T14:41:23.792582+0000","last_peered":"2024-04-02T14:41:23.792582+0000","last_clean":"2024-04-02T14:41:23.792582+0000","last_became_active":"2024-04-02T14:41:22.537304+0000","last_became_peered":"2024-04-02T14:41:22.537304+0000","last_unstale":"2024-04-02T14:41:23.792582+0000","last_undegraded":"2024-04-02T14:41:23.792582+0000","last_fullsized":"2024-04-02T14:41:23.792582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:25.599724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506322+0000","last_change":"2024-04-02T14:41:22.546775+0000","last_active":"2024-04-02T14:41:23.506322+0000","last_peered":"2024-04-02T14:41:23.506322+0000","last_clean":"2024-04-02T14:41:23.506322+0000","last_became_active":"2024-04-02T14:41:22.546100+0000","last_became_peered":"2024-04-02T14:41:22.546100+0000","last_unstale":"2024-04-02T14:41:23.506322+0000","last_undegraded":"2024-04-02T14:41:23.506322+0000","last_fullsized":"2024-04-02T14:41:23.506322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:42:02.559335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.795010+0000","last_change":"2024-04-02T14:41:22.543988+0000","last_active":"2024-04-02T14:41:23.795010+0000","last_peered":"2024-04-02T14:41:23.795010+0000","last_clean":"2024-04-02T14:41:23.795010+0000","last_became_active":"2024-04-02T14:41:22.543036+0000","last_became_peered":"2024-04-02T14:41:22.543036+0000","last_unstale":"2024-04-02T14:41:23.795010+0000","last_undegraded":"2024-04-02T14:41:23.795010+0000","last_fullsized":"2024-04-02T14:41:23.795010+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:29.470929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793394+0000","last_change":"2024-04-02T14:41:22.543954+0000","last_active":"2024-04-02T14:41:23.793394+0000","last_peered":"2024-04-02T14:41:23.793394+0000","last_clean":"2024-04-02T14:41:23.793394+0000","last_became_active":"2024-04-02T14:41:22.543624+0000","last_became_peered":"2024-04-02T14:41:22.543624+0000","last_unstale":"2024-04-02T14:41:23.793394+0000","last_undegraded":"2024-04-02T14:41:23.793394+0000","last_fullsized":"2024-04-02T14:41:23.793394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:46.933877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.091677+0000","last_change":"2024-04-02T14:41:21.477341+0000","last_active":"2024-04-02T14:41:29.091677+0000","last_peered":"2024-04-02T14:41:29.091677+0000","last_clean":"2024-04-02T14:41:29.091677+0000","last_became_active":"2024-04-02T14:41:21.476708+0000","last_became_peered":"2024-04-02T14:41:21.476708+0000","last_unstale":"2024-04-02T14:41:29.091677+0000","last_undegraded":"2024-04-02T14:41:29.091677+0000","last_fullsized":"2024-04-02T14:41:29.091677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:34.867570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.013166+0000","last_change":"2024-04-02T14:41:22.551293+0000","last_active":"2024-04-02T14:41:24.013166+0000","last_peered":"2024-04-02T14:41:24.013166+0000","last_clean":"2024-04-02T14:41:24.013166+0000","last_became_active":"2024-04-02T14:41:22.547819+0000","last_became_peered":"2024-04-02T14:41:22.547819+0000","last_unstale":"2024-04-02T14:41:24.013166+0000","last_undegraded":"2024-04-02T14:41:24.013166+0000","last_fullsized":"2024-04-02T14:41:24.013166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:40.393005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.013338+0000","last_change":"2024-04-02T14:41:21.487096+0000","last_active":"2024-04-02T14:41:24.013338+0000","last_peered":"2024-04-02T14:41:24.013338+0000","last_clean":"2024-04-02T14:41:24.013338+0000","last_became_active":"2024-04-02T14:41:21.485883+0000","last_became_peered":"2024-04-02T14:41:21.485883+0000","last_unstale":"2024-04-02T14:41:24.013338+0000","last_undegraded":"2024-04-02T14:41:24.013338+0000","last_fullsized":"2024-04-02T14:41:24.013338+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:24.017671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782636+0000","last_change":"2024-04-02T14:41:22.549234+0000","last_active":"2024-04-02T14:41:23.782636+0000","last_peered":"2024-04-02T14:41:23.782636+0000","last_clean":"2024-04-02T14:41:23.782636+0000","last_became_active":"2024-04-02T14:41:22.548855+0000","last_became_peered":"2024-04-02T14:41:22.548855+0000","last_unstale":"2024-04-02T14:41:23.782636+0000","last_undegraded":"2024-04-02T14:41:23.782636+0000","last_fullsized":"2024-04-02T14:41:23.782636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:05.047383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'11","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.090840+0000","last_change":"2024-04-02T14:41:21.471673+0000","last_active":"2024-04-02T14:41:29.090840+0000","last_peered":"2024-04-02T14:41:29.090840+0000","last_clean":"2024-04-02T14:41:29.090840+0000","last_became_active":"2024-04-02T14:41:21.471292+0000","last_became_peered":"2024-04-02T14:41:21.471292+0000","last_unstale":"2024-04-02T14:41:29.090840+0000","last_undegraded":"2024-04-02T14:41:29.090840+0000","last_fullsized":"2024-04-02T14:41:29.090840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:42:55.665938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792271+0000","last_change":"2024-04-02T14:41:22.537665+0000","last_active":"2024-04-02T14:41:23.792271+0000","last_peered":"2024-04-02T14:41:23.792271+0000","last_clean":"2024-04-02T14:41:23.792271+0000","last_became_active":"2024-04-02T14:41:22.537274+0000","last_became_peered":"2024-04-02T14:41:22.537274+0000","last_unstale":"2024-04-02T14:41:23.792271+0000","last_undegraded":"2024-04-02T14:41:23.792271+0000","last_fullsized":"2024-04-02T14:41:23.792271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:14:39.200735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.090241+0000","last_change":"2024-04-02T14:41:21.466547+0000","last_active":"2024-04-02T14:41:29.090241+0000","last_peered":"2024-04-02T14:41:29.090241+0000","last_clean":"2024-04-02T14:41:29.090241+0000","last_became_active":"2024-04-02T14:41:21.466063+0000","last_became_peered":"2024-04-02T14:41:21.466063+0000","last_unstale":"2024-04-02T14:41:29.090241+0000","last_undegraded":"2024-04-02T14:41:29.090241+0000","last_fullsized":"2024-04-02T14:41:29.090241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:39.201543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793982+0000","last_change":"2024-04-02T14:41:22.523984+0000","last_active":"2024-04-02T14:41:23.793982+0000","last_peered":"2024-04-02T14:41:23.793982+0000","last_clean":"2024-04-02T14:41:23.793982+0000","last_became_active":"2024-04-02T14:41:22.523524+0000","last_became_peered":"2024-04-02T14:41:22.523524+0000","last_unstale":"2024-04-02T14:41:23.793982+0000","last_undegraded":"2024-04-02T14:41:23.793982+0000","last_fullsized":"2024-04-02T14:41:23.793982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:31.542588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.050732+0000","last_change":"2024-04-02T14:41:21.470877+0000","last_active":"2024-04-02T14:41:26.050732+0000","last_peered":"2024-04-02T14:41:26.050732+0000","last_clean":"2024-04-02T14:41:26.050732+0000","last_became_active":"2024-04-02T14:41:21.470046+0000","last_became_peered":"2024-04-02T14:41:21.470046+0000","last_unstale":"2024-04-02T14:41:26.050732+0000","last_undegraded":"2024-04-02T14:41:26.050732+0000","last_fullsized":"2024-04-02T14:41:26.050732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:01.839043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792816+0000","last_change":"2024-04-02T14:41:22.544107+0000","last_active":"2024-04-02T14:41:23.792816+0000","last_peered":"2024-04-02T14:41:23.792816+0000","last_clean":"2024-04-02T14:41:23.792816+0000","last_became_active":"2024-04-02T14:41:22.543739+0000","last_became_peered":"2024-04-02T14:41:22.543739+0000","last_unstale":"2024-04-02T14:41:23.792816+0000","last_undegraded":"2024-04-02T14:41:23.792816+0000","last_fullsized":"2024-04-02T14:41:23.792816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:49:08.921668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.078281+0000","last_change":"2024-04-02T14:41:21.479714+0000","last_active":"2024-04-02T14:41:28.078281+0000","last_peered":"2024-04-02T14:41:28.078281+0000","last_clean":"2024-04-02T14:41:28.078281+0000","last_became_active":"2024-04-02T14:41:21.479329+0000","last_became_peered":"2024-04-02T14:41:21.479329+0000","last_unstale":"2024-04-02T14:41:28.078281+0000","last_undegraded":"2024-04-02T14:41:28.078281+0000","last_fullsized":"2024-04-02T14:41:28.078281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:09.021017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793781+0000","last_change":"2024-04-02T14:41:22.550072+0000","last_active":"2024-04-02T14:41:23.793781+0000","last_peered":"2024-04-02T14:41:23.793781+0000","last_clean":"2024-04-02T14:41:23.793781+0000","last_became_active":"2024-04-02T14:41:22.549722+0000","last_became_peered":"2024-04-02T14:41:22.549722+0000","last_unstale":"2024-04-02T14:41:23.793781+0000","last_undegraded":"2024-04-02T14:41:23.793781+0000","last_fullsized":"2024-04-02T14:41:23.793781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:30:56.376705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:27.063381+0000","last_change":"2024-04-02T14:41:21.484054+0000","last_active":"2024-04-02T14:41:27.063381+0000","last_peered":"2024-04-02T14:41:27.063381+0000","last_clean":"2024-04-02T14:41:27.063381+0000","last_became_active":"2024-04-02T14:41:21.483688+0000","last_became_peered":"2024-04-02T14:41:21.483688+0000","last_unstale":"2024-04-02T14:41:27.063381+0000","last_undegraded":"2024-04-02T14:41:27.063381+0000","last_fullsized":"2024-04-02T14:41:27.063381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:24.114180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012433+0000","last_change":"2024-04-02T14:41:22.553274+0000","last_active":"2024-04-02T14:41:24.012433+0000","last_peered":"2024-04-02T14:41:24.012433+0000","last_clean":"2024-04-02T14:41:24.012433+0000","last_became_active":"2024-04-02T14:41:22.545412+0000","last_became_peered":"2024-04-02T14:41:22.545412+0000","last_unstale":"2024-04-02T14:41:24.012433+0000","last_undegraded":"2024-04-02T14:41:24.012433+0000","last_fullsized":"2024-04-02T14:41:24.012433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:21.917119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.046893+0000","last_change":"2024-04-02T14:41:21.468582+0000","last_active":"2024-04-02T14:41:26.046893+0000","last_peered":"2024-04-02T14:41:26.046893+0000","last_clean":"2024-04-02T14:41:26.046893+0000","last_became_active":"2024-04-02T14:41:21.468081+0000","last_became_peered":"2024-04-02T14:41:21.468081+0000","last_unstale":"2024-04-02T14:41:26.046893+0000","last_undegraded":"2024-04-02T14:41:26.046893+0000","last_fullsized":"2024-04-02T14:41:26.046893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:19.593081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.794640+0000","last_change":"2024-04-02T14:41:22.555627+0000","last_active":"2024-04-02T14:41:23.794640+0000","last_peered":"2024-04-02T14:41:23.794640+0000","last_clean":"2024-04-02T14:41:23.794640+0000","last_became_active":"2024-04-02T14:41:22.555171+0000","last_became_peered":"2024-04-02T14:41:22.555171+0000","last_unstale":"2024-04-02T14:41:23.794640+0000","last_undegraded":"2024-04-02T14:41:23.794640+0000","last_fullsized":"2024-04-02T14:41:23.794640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:21:28.161869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.049246+0000","last_change":"2024-04-02T14:41:21.475004+0000","last_active":"2024-04-02T14:41:26.049246+0000","last_peered":"2024-04-02T14:41:26.049246+0000","last_clean":"2024-04-02T14:41:26.049246+0000","last_became_active":"2024-04-02T14:41:21.474621+0000","last_became_peered":"2024-04-02T14:41:21.474621+0000","last_unstale":"2024-04-02T14:41:26.049246+0000","last_undegraded":"2024-04-02T14:41:26.049246+0000","last_fullsized":"2024-04-02T14:41:26.049246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:38.779276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792608+0000","last_change":"2024-04-02T14:41:22.543100+0000","last_active":"2024-04-02T14:41:23.792608+0000","last_peered":"2024-04-02T14:41:23.792608+0000","last_clean":"2024-04-02T14:41:23.792608+0000","last_became_active":"2024-04-02T14:41:22.542697+0000","last_became_peered":"2024-04-02T14:41:22.542697+0000","last_unstale":"2024-04-02T14:41:23.792608+0000","last_undegraded":"2024-04-02T14:41:23.792608+0000","last_fullsized":"2024-04-02T14:41:23.792608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:50:52.412879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.090683+0000","last_change":"2024-04-02T14:41:21.468626+0000","last_active":"2024-04-02T14:41:29.090683+0000","last_peered":"2024-04-02T14:41:29.090683+0000","last_clean":"2024-04-02T14:41:29.090683+0000","last_became_active":"2024-04-02T14:41:21.468041+0000","last_became_peered":"2024-04-02T14:41:21.468041+0000","last_unstale":"2024-04-02T14:41:29.090683+0000","last_undegraded":"2024-04-02T14:41:29.090683+0000","last_fullsized":"2024-04-02T14:41:29.090683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:55.064033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782267+0000","last_change":"2024-04-02T14:41:22.539584+0000","last_active":"2024-04-02T14:41:23.782267+0000","last_peered":"2024-04-02T14:41:23.782267+0000","last_clean":"2024-04-02T14:41:23.782267+0000","last_became_active":"2024-04-02T14:41:22.539218+0000","last_became_peered":"2024-04-02T14:41:22.539218+0000","last_unstale":"2024-04-02T14:41:23.782267+0000","last_undegraded":"2024-04-02T14:41:23.782267+0000","last_fullsized":"2024-04-02T14:41:23.782267+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:47:56.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.048650+0000","last_change":"2024-04-02T14:41:21.481208+0000","last_active":"2024-04-02T14:41:26.048650+0000","last_peered":"2024-04-02T14:41:26.048650+0000","last_clean":"2024-04-02T14:41:26.048650+0000","last_became_active":"2024-04-02T14:41:21.480782+0000","last_became_peered":"2024-04-02T14:41:21.480782+0000","last_unstale":"2024-04-02T14:41:26.048650+0000","last_undegraded":"2024-04-02T14:41:26.048650+0000","last_fullsized":"2024-04-02T14:41:26.048650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:12.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793209+0000","last_change":"2024-04-02T14:41:22.517397+0000","last_active":"2024-04-02T14:41:23.793209+0000","last_peered":"2024-04-02T14:41:23.793209+0000","last_clean":"2024-04-02T14:41:23.793209+0000","last_became_active":"2024-04-02T14:41:22.517027+0000","last_became_peered":"2024-04-02T14:41:22.517027+0000","last_unstale":"2024-04-02T14:41:23.793209+0000","last_undegraded":"2024-04-02T14:41:23.793209+0000","last_fullsized":"2024-04-02T14:41:23.793209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:53.129949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'17","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.090658+0000","last_change":"2024-04-02T14:41:21.469643+0000","last_active":"2024-04-02T14:41:29.090658+0000","last_peered":"2024-04-02T14:41:29.090658+0000","last_clean":"2024-04-02T14:41:29.090658+0000","last_became_active":"2024-04-02T14:41:21.468730+0000","last_became_peered":"2024-04-02T14:41:21.468730+0000","last_unstale":"2024-04-02T14:41:29.090658+0000","last_undegraded":"2024-04-02T14:41:29.090658+0000","last_fullsized":"2024-04-02T14:41:29.090658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:10.983551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793609+0000","last_change":"2024-04-02T14:41:22.525052+0000","last_active":"2024-04-02T14:41:23.793609+0000","last_peered":"2024-04-02T14:41:23.793609+0000","last_clean":"2024-04-02T14:41:23.793609+0000","last_became_active":"2024-04-02T14:41:22.524691+0000","last_became_peered":"2024-04-02T14:41:22.524691+0000","last_unstale":"2024-04-02T14:41:23.793609+0000","last_undegraded":"2024-04-02T14:41:23.793609+0000","last_fullsized":"2024-04-02T14:41:23.793609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:16.386020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:30.110974+0000","last_change":"2024-04-02T14:41:21.476988+0000","last_active":"2024-04-02T14:41:30.110974+0000","last_peered":"2024-04-02T14:41:30.110974+0000","last_clean":"2024-04-02T14:41:30.110974+0000","last_became_active":"2024-04-02T14:41:21.476549+0000","last_became_peered":"2024-04-02T14:41:21.476549+0000","last_unstale":"2024-04-02T14:41:30.110974+0000","last_undegraded":"2024-04-02T14:41:30.110974+0000","last_fullsized":"2024-04-02T14:41:30.110974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:47.498626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793023+0000","last_change":"2024-04-02T14:41:22.544746+0000","last_active":"2024-04-02T14:41:23.793023+0000","last_peered":"2024-04-02T14:41:23.793023+0000","last_clean":"2024-04-02T14:41:23.793023+0000","last_became_active":"2024-04-02T14:41:22.544334+0000","last_became_peered":"2024-04-02T14:41:22.544334+0000","last_unstale":"2024-04-02T14:41:23.793023+0000","last_undegraded":"2024-04-02T14:41:23.793023+0000","last_fullsized":"2024-04-02T14:41:23.793023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:45:20.094455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.048477+0000","last_change":"2024-04-02T14:41:21.472458+0000","last_active":"2024-04-02T14:41:26.048477+0000","last_peered":"2024-04-02T14:41:26.048477+0000","last_clean":"2024-04-02T14:41:26.048477+0000","last_became_active":"2024-04-02T14:41:21.472105+0000","last_became_peered":"2024-04-02T14:41:21.472105+0000","last_unstale":"2024-04-02T14:41:26.048477+0000","last_undegraded":"2024-04-02T14:41:26.048477+0000","last_fullsized":"2024-04-02T14:41:26.048477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:44.075807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793812+0000","last_change":"2024-04-02T14:41:22.537492+0000","last_active":"2024-04-02T14:41:23.793812+0000","last_peered":"2024-04-02T14:41:23.793812+0000","last_clean":"2024-04-02T14:41:23.793812+0000","last_became_active":"2024-04-02T14:41:22.537067+0000","last_became_peered":"2024-04-02T14:41:22.537067+0000","last_unstale":"2024-04-02T14:41:23.793812+0000","last_undegraded":"2024-04-02T14:41:23.793812+0000","last_fullsized":"2024-04-02T14:41:23.793812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:08.704340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.071934+0000","last_change":"2024-04-02T14:41:21.466017+0000","last_active":"2024-04-02T14:41:28.071934+0000","last_peered":"2024-04-02T14:41:28.071934+0000","last_clean":"2024-04-02T14:41:28.071934+0000","last_became_active":"2024-04-02T14:41:21.465600+0000","last_became_peered":"2024-04-02T14:41:21.465600+0000","last_unstale":"2024-04-02T14:41:28.071934+0000","last_undegraded":"2024-04-02T14:41:28.071934+0000","last_fullsized":"2024-04-02T14:41:28.071934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:52:08.131125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793318+0000","last_change":"2024-04-02T14:41:22.526628+0000","last_active":"2024-04-02T14:41:23.793318+0000","last_peered":"2024-04-02T14:41:23.793318+0000","last_clean":"2024-04-02T14:41:23.793318+0000","last_became_active":"2024-04-02T14:41:22.526283+0000","last_became_peered":"2024-04-02T14:41:22.526283+0000","last_unstale":"2024-04-02T14:41:23.793318+0000","last_undegraded":"2024-04-02T14:41:23.793318+0000","last_fullsized":"2024-04-02T14:41:23.793318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:41.812330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.074839+0000","last_change":"2024-04-02T14:41:21.475853+0000","last_active":"2024-04-02T14:41:28.074839+0000","last_peered":"2024-04-02T14:41:28.074839+0000","last_clean":"2024-04-02T14:41:28.074839+0000","last_became_active":"2024-04-02T14:41:21.475555+0000","last_became_peered":"2024-04-02T14:41:21.475555+0000","last_unstale":"2024-04-02T14:41:28.074839+0000","last_undegraded":"2024-04-02T14:41:28.074839+0000","last_fullsized":"2024-04-02T14:41:28.074839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:49.170906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792966+0000","last_change":"2024-04-02T14:41:22.523848+0000","last_active":"2024-04-02T14:41:23.792966+0000","last_peered":"2024-04-02T14:41:23.792966+0000","last_clean":"2024-04-02T14:41:23.792966+0000","last_became_active":"2024-04-02T14:41:22.523476+0000","last_became_peered":"2024-04-02T14:41:22.523476+0000","last_unstale":"2024-04-02T14:41:23.792966+0000","last_undegraded":"2024-04-02T14:41:23.792966+0000","last_fullsized":"2024-04-02T14:41:23.792966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:47:46.933264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.992747+0000","last_change":"2024-04-02T14:41:21.468556+0000","last_active":"2024-04-02T14:41:24.992747+0000","last_peered":"2024-04-02T14:41:24.992747+0000","last_clean":"2024-04-02T14:41:24.992747+0000","last_became_active":"2024-04-02T14:41:21.468215+0000","last_became_peered":"2024-04-02T14:41:21.468215+0000","last_unstale":"2024-04-02T14:41:24.992747+0000","last_undegraded":"2024-04-02T14:41:24.992747+0000","last_fullsized":"2024-04-02T14:41:24.992747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:04.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506120+0000","last_change":"2024-04-02T14:41:22.524228+0000","last_active":"2024-04-02T14:41:23.506120+0000","last_peered":"2024-04-02T14:41:23.506120+0000","last_clean":"2024-04-02T14:41:23.506120+0000","last_became_active":"2024-04-02T14:41:22.523839+0000","last_became_peered":"2024-04-02T14:41:22.523839+0000","last_unstale":"2024-04-02T14:41:23.506120+0000","last_undegraded":"2024-04-02T14:41:23.506120+0000","last_fullsized":"2024-04-02T14:41:23.506120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:13.094123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.877700+0000","last_change":"2024-04-02T14:41:21.485518+0000","last_active":"2024-04-02T14:41:24.877700+0000","last_peered":"2024-04-02T14:41:24.877700+0000","last_clean":"2024-04-02T14:41:24.877700+0000","last_became_active":"2024-04-02T14:41:21.483016+0000","last_became_peered":"2024-04-02T14:41:21.483016+0000","last_unstale":"2024-04-02T14:41:24.877700+0000","last_undegraded":"2024-04-02T14:41:24.877700+0000","last_fullsized":"2024-04-02T14:41:24.877700+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:57.177512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.507870+0000","last_change":"2024-04-02T14:41:22.542847+0000","last_active":"2024-04-02T14:41:24.507870+0000","last_peered":"2024-04-02T14:41:24.507870+0000","last_clean":"2024-04-02T14:41:24.507870+0000","last_became_active":"2024-04-02T14:41:22.542476+0000","last_became_peered":"2024-04-02T14:41:22.542476+0000","last_unstale":"2024-04-02T14:41:24.507870+0000","last_undegraded":"2024-04-02T14:41:24.507870+0000","last_fullsized":"2024-04-02T14:41:24.507870+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:27:12.102931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.093525+0000","last_change":"2024-04-02T14:41:21.483382+0000","last_active":"2024-04-02T14:41:29.093525+0000","last_peered":"2024-04-02T14:41:29.093525+0000","last_clean":"2024-04-02T14:41:29.093525+0000","last_became_active":"2024-04-02T14:41:21.482996+0000","last_became_peered":"2024-04-02T14:41:21.482996+0000","last_unstale":"2024-04-02T14:41:29.093525+0000","last_undegraded":"2024-04-02T14:41:29.093525+0000","last_fullsized":"2024-04-02T14:41:29.093525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:37.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793725+0000","last_change":"2024-04-02T14:41:22.544121+0000","last_active":"2024-04-02T14:41:23.793725+0000","last_peered":"2024-04-02T14:41:23.793725+0000","last_clean":"2024-04-02T14:41:23.793725+0000","last_became_active":"2024-04-02T14:41:22.543765+0000","last_became_peered":"2024-04-02T14:41:22.543765+0000","last_unstale":"2024-04-02T14:41:23.793725+0000","last_undegraded":"2024-04-02T14:41:23.793725+0000","last_fullsized":"2024-04-02T14:41:23.793725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:45.715217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792807+0000","last_change":"2024-04-02T14:41:22.525853+0000","last_active":"2024-04-02T14:41:23.792807+0000","last_peered":"2024-04-02T14:41:23.792807+0000","last_clean":"2024-04-02T14:41:23.792807+0000","last_became_active":"2024-04-02T14:41:22.525553+0000","last_became_peered":"2024-04-02T14:41:22.525553+0000","last_unstale":"2024-04-02T14:41:23.792807+0000","last_undegraded":"2024-04-02T14:41:23.792807+0000","last_fullsized":"2024-04-02T14:41:23.792807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:25:30.340974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:27.063769+0000","last_change":"2024-04-02T14:41:21.490632+0000","last_active":"2024-04-02T14:41:27.063769+0000","last_peered":"2024-04-02T14:41:27.063769+0000","last_clean":"2024-04-02T14:41:27.063769+0000","last_became_active":"2024-04-02T14:41:21.490208+0000","last_became_peered":"2024-04-02T14:41:21.490208+0000","last_unstale":"2024-04-02T14:41:27.063769+0000","last_undegraded":"2024-04-02T14:41:27.063769+0000","last_fullsized":"2024-04-02T14:41:27.063769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:18.566261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012817+0000","last_change":"2024-04-02T14:41:22.544872+0000","last_active":"2024-04-02T14:41:24.012817+0000","last_peered":"2024-04-02T14:41:24.012817+0000","last_clean":"2024-04-02T14:41:24.012817+0000","last_became_active":"2024-04-02T14:41:22.544268+0000","last_became_peered":"2024-04-02T14:41:22.544268+0000","last_unstale":"2024-04-02T14:41:24.012817+0000","last_undegraded":"2024-04-02T14:41:24.012817+0000","last_fullsized":"2024-04-02T14:41:24.012817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:54.963084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.076419+0000","last_change":"2024-04-02T14:41:21.474411+0000","last_active":"2024-04-02T14:41:28.076419+0000","last_peered":"2024-04-02T14:41:28.076419+0000","last_clean":"2024-04-02T14:41:28.076419+0000","last_became_active":"2024-04-02T14:41:21.473950+0000","last_became_peered":"2024-04-02T14:41:21.473950+0000","last_unstale":"2024-04-02T14:41:28.076419+0000","last_undegraded":"2024-04-02T14:41:28.076419+0000","last_fullsized":"2024-04-02T14:41:28.076419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:46:05.892446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792697+0000","last_change":"2024-04-02T14:40:03.537813+0000","last_active":"2024-04-02T14:41:23.792697+0000","last_peered":"2024-04-02T14:41:23.792697+0000","last_clean":"2024-04-02T14:41:23.792697+0000","last_became_active":"2024-04-02T14:40:03.537223+0000","last_became_peered":"2024-04-02T14:40:03.537223+0000","last_unstale":"2024-04-02T14:41:23.792697+0000","last_undegraded":"2024-04-02T14:41:23.792697+0000","last_fullsized":"2024-04-02T14:41:23.792697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_clean_scrub_stamp":"2024-04-02T14:36:11.612815+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:19:10.346096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792813+0000","last_change":"2024-04-02T14:41:22.537906+0000","last_active":"2024-04-02T14:41:23.792813+0000","last_peered":"2024-04-02T14:41:23.792813+0000","last_clean":"2024-04-02T14:41:23.792813+0000","last_became_active":"2024-04-02T14:41:22.537238+0000","last_became_peered":"2024-04-02T14:41:22.537238+0000","last_unstale":"2024-04-02T14:41:23.792813+0000","last_undegraded":"2024-04-02T14:41:23.792813+0000","last_fullsized":"2024-04-02T14:41:23.792813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:57.089563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:27.059407+0000","last_change":"2024-04-02T14:41:21.466830+0000","last_active":"2024-04-02T14:41:27.059407+0000","last_peered":"2024-04-02T14:41:27.059407+0000","last_clean":"2024-04-02T14:41:27.059407+0000","last_became_active":"2024-04-02T14:41:21.466490+0000","last_became_peered":"2024-04-02T14:41:21.466490+0000","last_unstale":"2024-04-02T14:41:27.059407+0000","last_undegraded":"2024-04-02T14:41:27.059407+0000","last_fullsized":"2024-04-02T14:41:27.059407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:15.597854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.078356+0000","last_change":"2024-04-02T14:41:21.481139+0000","last_active":"2024-04-02T14:41:28.078356+0000","last_peered":"2024-04-02T14:41:28.078356+0000","last_clean":"2024-04-02T14:41:28.078356+0000","last_became_active":"2024-04-02T14:41:21.480754+0000","last_became_peered":"2024-04-02T14:41:21.480754+0000","last_unstale":"2024-04-02T14:41:28.078356+0000","last_undegraded":"2024-04-02T14:41:28.078356+0000","last_fullsized":"2024-04-02T14:41:28.078356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:03.716223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.794700+0000","last_change":"2024-04-02T14:41:22.548483+0000","last_active":"2024-04-02T14:41:23.794700+0000","last_peered":"2024-04-02T14:41:23.794700+0000","last_clean":"2024-04-02T14:41:23.794700+0000","last_became_active":"2024-04-02T14:41:22.547463+0000","last_became_peered":"2024-04-02T14:41:22.547463+0000","last_unstale":"2024-04-02T14:41:23.794700+0000","last_undegraded":"2024-04-02T14:41:23.794700+0000","last_fullsized":"2024-04-02T14:41:23.794700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:33:58.076844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.079853+0000","last_change":"2024-04-02T14:41:21.491105+0000","last_active":"2024-04-02T14:41:28.079853+0000","last_peered":"2024-04-02T14:41:28.079853+0000","last_clean":"2024-04-02T14:41:28.079853+0000","last_became_active":"2024-04-02T14:41:21.490635+0000","last_became_peered":"2024-04-02T14:41:21.490635+0000","last_unstale":"2024-04-02T14:41:28.079853+0000","last_undegraded":"2024-04-02T14:41:28.079853+0000","last_fullsized":"2024-04-02T14:41:28.079853+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:16.531064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.013024+0000","last_change":"2024-04-02T14:41:22.553497+0000","last_active":"2024-04-02T14:41:24.013024+0000","last_peered":"2024-04-02T14:41:24.013024+0000","last_clean":"2024-04-02T14:41:24.013024+0000","last_became_active":"2024-04-02T14:41:22.550474+0000","last_became_peered":"2024-04-02T14:41:22.550474+0000","last_unstale":"2024-04-02T14:41:24.013024+0000","last_undegraded":"2024-04-02T14:41:24.013024+0000","last_fullsized":"2024-04-02T14:41:24.013024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:52.212454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.092740+0000","last_change":"2024-04-02T14:41:21.479516+0000","last_active":"2024-04-02T14:41:29.092740+0000","last_peered":"2024-04-02T14:41:29.092740+0000","last_clean":"2024-04-02T14:41:29.092740+0000","last_became_active":"2024-04-02T14:41:21.478841+0000","last_became_peered":"2024-04-02T14:41:21.478841+0000","last_unstale":"2024-04-02T14:41:29.092740+0000","last_undegraded":"2024-04-02T14:41:29.092740+0000","last_fullsized":"2024-04-02T14:41:29.092740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:39.222316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782763+0000","last_change":"2024-04-02T14:41:22.550095+0000","last_active":"2024-04-02T14:41:23.782763+0000","last_peered":"2024-04-02T14:41:23.782763+0000","last_clean":"2024-04-02T14:41:23.782763+0000","last_became_active":"2024-04-02T14:41:22.547438+0000","last_became_peered":"2024-04-02T14:41:22.547438+0000","last_unstale":"2024-04-02T14:41:23.782763+0000","last_undegraded":"2024-04-02T14:41:23.782763+0000","last_fullsized":"2024-04-02T14:41:23.782763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:03:03.668289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.048298+0000","last_change":"2024-04-02T14:41:21.482926+0000","last_active":"2024-04-02T14:41:26.048298+0000","last_peered":"2024-04-02T14:41:26.048298+0000","last_clean":"2024-04-02T14:41:26.048298+0000","last_became_active":"2024-04-02T14:41:21.482517+0000","last_became_peered":"2024-04-02T14:41:21.482517+0000","last_unstale":"2024-04-02T14:41:26.048298+0000","last_undegraded":"2024-04-02T14:41:26.048298+0000","last_fullsized":"2024-04-02T14:41:26.048298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:43.052390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792578+0000","last_change":"2024-04-02T14:41:22.527758+0000","last_active":"2024-04-02T14:41:23.792578+0000","last_peered":"2024-04-02T14:41:23.792578+0000","last_clean":"2024-04-02T14:41:23.792578+0000","last_became_active":"2024-04-02T14:41:22.526954+0000","last_became_peered":"2024-04-02T14:41:22.526954+0000","last_unstale":"2024-04-02T14:41:23.792578+0000","last_undegraded":"2024-04-02T14:41:23.792578+0000","last_fullsized":"2024-04-02T14:41:23.792578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:34.988661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.078624+0000","last_change":"2024-04-02T14:41:21.487534+0000","last_active":"2024-04-02T14:41:28.078624+0000","last_peered":"2024-04-02T14:41:28.078624+0000","last_clean":"2024-04-02T14:41:28.078624+0000","last_became_active":"2024-04-02T14:41:21.487168+0000","last_became_peered":"2024-04-02T14:41:21.487168+0000","last_unstale":"2024-04-02T14:41:28.078624+0000","last_undegraded":"2024-04-02T14:41:28.078624+0000","last_fullsized":"2024-04-02T14:41:28.078624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:03:20.636028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.505969+0000","last_change":"2024-04-02T14:41:22.539246+0000","last_active":"2024-04-02T14:41:23.505969+0000","last_peered":"2024-04-02T14:41:23.505969+0000","last_clean":"2024-04-02T14:41:23.505969+0000","last_became_active":"2024-04-02T14:41:22.538864+0000","last_became_peered":"2024-04-02T14:41:22.538864+0000","last_unstale":"2024-04-02T14:41:23.505969+0000","last_undegraded":"2024-04-02T14:41:23.505969+0000","last_fullsized":"2024-04-02T14:41:23.505969+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:40:13.418752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:28.075733+0000","last_change":"2024-04-02T14:41:21.477043+0000","last_active":"2024-04-02T14:41:28.075733+0000","last_peered":"2024-04-02T14:41:28.075733+0000","last_clean":"2024-04-02T14:41:28.075733+0000","last_became_active":"2024-04-02T14:41:21.476580+0000","last_became_peered":"2024-04-02T14:41:21.476580+0000","last_unstale":"2024-04-02T14:41:28.075733+0000","last_undegraded":"2024-04-02T14:41:28.075733+0000","last_fullsized":"2024-04-02T14:41:28.075733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:18.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012673+0000","last_change":"2024-04-02T14:41:22.549758+0000","last_active":"2024-04-02T14:41:24.012673+0000","last_peered":"2024-04-02T14:41:24.012673+0000","last_clean":"2024-04-02T14:41:24.012673+0000","last_became_active":"2024-04-02T14:41:22.545582+0000","last_became_peered":"2024-04-02T14:41:22.545582+0000","last_unstale":"2024-04-02T14:41:24.012673+0000","last_undegraded":"2024-04-02T14:41:24.012673+0000","last_fullsized":"2024-04-02T14:41:24.012673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:42.311921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:26.046962+0000","last_change":"2024-04-02T14:41:21.472328+0000","last_active":"2024-04-02T14:41:26.046962+0000","last_peered":"2024-04-02T14:41:26.046962+0000","last_clean":"2024-04-02T14:41:26.046962+0000","last_became_active":"2024-04-02T14:41:21.471919+0000","last_became_peered":"2024-04-02T14:41:21.471919+0000","last_unstale":"2024-04-02T14:41:26.046962+0000","last_undegraded":"2024-04-02T14:41:26.046962+0000","last_fullsized":"2024-04-02T14:41:26.046962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:10:37.897485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793268+0000","last_change":"2024-04-02T14:41:22.531509+0000","last_active":"2024-04-02T14:41:23.793268+0000","last_peered":"2024-04-02T14:41:23.793268+0000","last_clean":"2024-04-02T14:41:23.793268+0000","last_became_active":"2024-04-02T14:41:22.531206+0000","last_became_peered":"2024-04-02T14:41:22.531206+0000","last_unstale":"2024-04-02T14:41:23.793268+0000","last_undegraded":"2024-04-02T14:41:23.793268+0000","last_fullsized":"2024-04-02T14:41:23.793268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:39.099177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:30.114614+0000","last_change":"2024-04-02T14:41:21.485271+0000","last_active":"2024-04-02T14:41:30.114614+0000","last_peered":"2024-04-02T14:41:30.114614+0000","last_clean":"2024-04-02T14:41:30.114614+0000","last_became_active":"2024-04-02T14:41:21.484895+0000","last_became_peered":"2024-04-02T14:41:21.484895+0000","last_unstale":"2024-04-02T14:41:30.114614+0000","last_undegraded":"2024-04-02T14:41:30.114614+0000","last_fullsized":"2024-04-02T14:41:30.114614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.552092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793618+0000","last_change":"2024-04-02T14:41:22.524853+0000","last_active":"2024-04-02T14:41:23.793618+0000","last_peered":"2024-04-02T14:41:23.793618+0000","last_clean":"2024-04-02T14:41:23.793618+0000","last_became_active":"2024-04-02T14:41:22.524403+0000","last_became_peered":"2024-04-02T14:41:22.524403+0000","last_unstale":"2024-04-02T14:41:23.793618+0000","last_undegraded":"2024-04-02T14:41:23.793618+0000","last_fullsized":"2024-04-02T14:41:23.793618+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:31.322626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793961+0000","last_change":"2024-04-02T14:41:21.474340+0000","last_active":"2024-04-02T14:41:23.793961+0000","last_peered":"2024-04-02T14:41:23.793961+0000","last_clean":"2024-04-02T14:41:23.793961+0000","last_became_active":"2024-04-02T14:41:21.473934+0000","last_became_peered":"2024-04-02T14:41:21.473934+0000","last_unstale":"2024-04-02T14:41:23.793961+0000","last_undegraded":"2024-04-02T14:41:23.793961+0000","last_fullsized":"2024-04-02T14:41:23.793961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:28:42.711817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792744+0000","last_change":"2024-04-02T14:41:22.543018+0000","last_active":"2024-04-02T14:41:23.792744+0000","last_peered":"2024-04-02T14:41:23.792744+0000","last_clean":"2024-04-02T14:41:23.792744+0000","last_became_active":"2024-04-02T14:41:22.542621+0000","last_became_peered":"2024-04-02T14:41:22.542621+0000","last_unstale":"2024-04-02T14:41:23.792744+0000","last_undegraded":"2024-04-02T14:41:23.792744+0000","last_fullsized":"2024-04-02T14:41:23.792744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:55:03.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:29.093726+0000","last_change":"2024-04-02T14:41:21.493060+0000","last_active":"2024-04-02T14:41:29.093726+0000","last_peered":"2024-04-02T14:41:29.093726+0000","last_clean":"2024-04-02T14:41:29.093726+0000","last_became_active":"2024-04-02T14:41:21.492627+0000","last_became_peered":"2024-04-02T14:41:21.492627+0000","last_unstale":"2024-04-02T14:41:29.093726+0000","last_undegraded":"2024-04-02T14:41:29.093726+0000","last_fullsized":"2024-04-02T14:41:29.093726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:29:10.443866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793264+0000","last_change":"2024-04-02T14:41:22.527742+0000","last_active":"2024-04-02T14:41:23.793264+0000","last_peered":"2024-04-02T14:41:23.793264+0000","last_clean":"2024-04-02T14:41:23.793264+0000","last_became_active":"2024-04-02T14:41:22.527436+0000","last_became_peered":"2024-04-02T14:41:22.527436+0000","last_unstale":"2024-04-02T14:41:23.793264+0000","last_undegraded":"2024-04-02T14:41:23.793264+0000","last_fullsized":"2024-04-02T14:41:23.793264+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:04:24.754907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506134+0000","last_change":"2024-04-02T14:41:22.526235+0000","last_active":"2024-04-02T14:41:23.506134+0000","last_peered":"2024-04-02T14:41:23.506134+0000","last_clean":"2024-04-02T14:41:23.506134+0000","last_became_active":"2024-04-02T14:41:22.525879+0000","last_became_peered":"2024-04-02T14:41:22.525879+0000","last_unstale":"2024-04-02T14:41:23.506134+0000","last_undegraded":"2024-04-02T14:41:23.506134+0000","last_fullsized":"2024-04-02T14:41:23.506134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:20:54.431223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792579+0000","last_change":"2024-04-02T14:41:22.522498+0000","last_active":"2024-04-02T14:41:23.792579+0000","last_peered":"2024-04-02T14:41:23.792579+0000","last_clean":"2024-04-02T14:41:23.792579+0000","last_became_active":"2024-04-02T14:41:22.522162+0000","last_became_peered":"2024-04-02T14:41:22.522162+0000","last_unstale":"2024-04-02T14:41:23.792579+0000","last_undegraded":"2024-04-02T14:41:23.792579+0000","last_fullsized":"2024-04-02T14:41:23.792579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:42:04.701228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792330+0000","last_change":"2024-04-02T14:41:22.543920+0000","last_active":"2024-04-02T14:41:23.792330+0000","last_peered":"2024-04-02T14:41:23.792330+0000","last_clean":"2024-04-02T14:41:23.792330+0000","last_became_active":"2024-04-02T14:41:22.543552+0000","last_became_peered":"2024-04-02T14:41:22.543552+0000","last_unstale":"2024-04-02T14:41:23.792330+0000","last_undegraded":"2024-04-02T14:41:23.792330+0000","last_fullsized":"2024-04-02T14:41:23.792330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:28.761770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.782057+0000","last_change":"2024-04-02T14:41:22.550507+0000","last_active":"2024-04-02T14:41:23.782057+0000","last_peered":"2024-04-02T14:41:23.782057+0000","last_clean":"2024-04-02T14:41:23.782057+0000","last_became_active":"2024-04-02T14:41:22.545824+0000","last_became_peered":"2024-04-02T14:41:22.545824+0000","last_unstale":"2024-04-02T14:41:23.782057+0000","last_undegraded":"2024-04-02T14:41:23.782057+0000","last_fullsized":"2024-04-02T14:41:23.782057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:11.175138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.794810+0000","last_change":"2024-04-02T14:41:22.541362+0000","last_active":"2024-04-02T14:41:23.794810+0000","last_peered":"2024-04-02T14:41:23.794810+0000","last_clean":"2024-04-02T14:41:23.794810+0000","last_became_active":"2024-04-02T14:41:22.540927+0000","last_became_peered":"2024-04-02T14:41:22.540927+0000","last_unstale":"2024-04-02T14:41:23.794810+0000","last_undegraded":"2024-04-02T14:41:23.794810+0000","last_fullsized":"2024-04-02T14:41:23.794810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:43:00.746274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.506162+0000","last_change":"2024-04-02T14:41:22.527146+0000","last_active":"2024-04-02T14:41:23.506162+0000","last_peered":"2024-04-02T14:41:23.506162+0000","last_clean":"2024-04-02T14:41:23.506162+0000","last_became_active":"2024-04-02T14:41:22.526810+0000","last_became_peered":"2024-04-02T14:41:22.526810+0000","last_unstale":"2024-04-02T14:41:23.506162+0000","last_undegraded":"2024-04-02T14:41:23.506162+0000","last_fullsized":"2024-04-02T14:41:23.506162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:18:46.464701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793579+0000","last_change":"2024-04-02T14:41:22.524458+0000","last_active":"2024-04-02T14:41:23.793579+0000","last_peered":"2024-04-02T14:41:23.793579+0000","last_clean":"2024-04-02T14:41:23.793579+0000","last_became_active":"2024-04-02T14:41:22.524138+0000","last_became_peered":"2024-04-02T14:41:22.524138+0000","last_unstale":"2024-04-02T14:41:23.793579+0000","last_undegraded":"2024-04-02T14:41:23.793579+0000","last_fullsized":"2024-04-02T14:41:23.793579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:50:04.667964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012817+0000","last_change":"2024-04-02T14:41:22.550905+0000","last_active":"2024-04-02T14:41:24.012817+0000","last_peered":"2024-04-02T14:41:24.012817+0000","last_clean":"2024-04-02T14:41:24.012817+0000","last_became_active":"2024-04-02T14:41:22.548518+0000","last_became_peered":"2024-04-02T14:41:22.548518+0000","last_unstale":"2024-04-02T14:41:24.012817+0000","last_undegraded":"2024-04-02T14:41:24.012817+0000","last_fullsized":"2024-04-02T14:41:24.012817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:46:56.717745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792658+0000","last_change":"2024-04-02T14:41:22.542319+0000","last_active":"2024-04-02T14:41:23.792658+0000","last_peered":"2024-04-02T14:41:23.792658+0000","last_clean":"2024-04-02T14:41:23.792658+0000","last_became_active":"2024-04-02T14:41:22.541959+0000","last_became_peered":"2024-04-02T14:41:22.541959+0000","last_unstale":"2024-04-02T14:41:23.792658+0000","last_undegraded":"2024-04-02T14:41:23.792658+0000","last_fullsized":"2024-04-02T14:41:23.792658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:49.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:24.012482+0000","last_change":"2024-04-02T14:41:22.549932+0000","last_active":"2024-04-02T14:41:24.012482+0000","last_peered":"2024-04-02T14:41:24.012482+0000","last_clean":"2024-04-02T14:41:24.012482+0000","last_became_active":"2024-04-02T14:41:22.549231+0000","last_became_peered":"2024-04-02T14:41:22.549231+0000","last_unstale":"2024-04-02T14:41:24.012482+0000","last_undegraded":"2024-04-02T14:41:24.012482+0000","last_fullsized":"2024-04-02T14:41:24.012482+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:01.485343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.792984+0000","last_change":"2024-04-02T14:41:22.544712+0000","last_active":"2024-04-02T14:41:23.792984+0000","last_peered":"2024-04-02T14:41:23.792984+0000","last_clean":"2024-04-02T14:41:23.792984+0000","last_became_active":"2024-04-02T14:41:22.544043+0000","last_became_peered":"2024-04-02T14:41:22.544043+0000","last_unstale":"2024-04-02T14:41:23.792984+0000","last_undegraded":"2024-04-02T14:41:23.792984+0000","last_fullsized":"2024-04-02T14:41:23.792984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:07.931377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.793631+0000","last_change":"2024-04-02T14:41:22.542364+0000","last_active":"2024-04-02T14:41:23.793631+0000","last_peered":"2024-04-02T14:41:23.793631+0000","last_clean":"2024-04-02T14:41:23.793631+0000","last_became_active":"2024-04-02T14:41:22.541929+0000","last_became_peered":"2024-04-02T14:41:22.541929+0000","last_unstale":"2024-04-02T14:41:23.793631+0000","last_undegraded":"2024-04-02T14:41:23.793631+0000","last_fullsized":"2024-04-02T14:41:23.793631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:45.716132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-02T14:41:23.505990+0000","last_change":"2024-04-02T14:41:22.525295+0000","last_active":"2024-04-02T14:41:23.505990+0000","last_peered":"2024-04-02T14:41:23.505990+0000","last_clean":"2024-04-02T14:41:23.505990+0000","last_became_active":"2024-04-02T14:41:22.524484+0000","last_became_peered":"2024-04-02T14:41:22.524484+0000","last_unstale":"2024-04-02T14:41:23.505990+0000","last_undegraded":"2024-04-02T14:41:23.505990+0000","last_fullsized":"2024-04-02T14:41:23.505990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:01.819082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27708,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":210,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":483328,"data_stored":129184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182,"ondisk_log_size":182,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285582,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":452184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547219,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1051274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710745,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":452114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972384,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":452492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266725,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1044215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430250,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":451831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":473639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":453170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":474250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117050,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":451276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":451658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691888,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1043598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":46590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":22631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:41:32.614 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-02T14:41:32.614 INFO:tasks.cephadm:Setting up client nodes... 2024-04-02T14:41:32.614 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-02T14:41:33.125 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:33.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:33 smithi113 ceph-mon[36135]: pgmap v324: 97 pgs: 97 active+clean; 604 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 462 B/s rd, 9.7 KiB/s wr, 24 op/s 2024-04-02T14:41:33.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:33 smithi113 ceph-mon[36135]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:41:33.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:33 smithi204 ceph-mon[35948]: pgmap v324: 97 pgs: 97 active+clean; 604 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 462 B/s rd, 9.7 KiB/s wr, 24 op/s 2024-04-02T14:41:33.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:33 smithi204 ceph-mon[35948]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:41:33.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:33 smithi062 ceph-mon[32116]: pgmap v324: 97 pgs: 97 active+clean; 604 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 462 B/s rd, 9.7 KiB/s wr, 24 op/s 2024-04-02T14:41:33.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:33 smithi062 ceph-mon[32116]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:41:34.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:34 smithi113 ceph-mon[36135]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:41:34.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:34 smithi062 ceph-mon[32116]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:41:34.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:34 smithi204 ceph-mon[35948]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:41:35.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:35 smithi062 ceph-mon[32116]: pgmap v325: 97 pgs: 97 active+clean; 604 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s rd, 8.6 KiB/s wr, 21 op/s 2024-04-02T14:41:35.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:35 smithi113 ceph-mon[36135]: pgmap v325: 97 pgs: 97 active+clean; 604 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s rd, 8.6 KiB/s wr, 21 op/s 2024-04-02T14:41:35.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:35 smithi204 ceph-mon[35948]: pgmap v325: 97 pgs: 97 active+clean; 604 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s rd, 8.6 KiB/s wr, 21 op/s 2024-04-02T14:41:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.677+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3428095884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc4076620 msgr2=0x7f0dc4074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.677+0000 7f0dc8a00700 1 --2- 172.21.15.62:0/3428095884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc4076620 0x7f0dc4074a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0db40098f0 tx=0x7f0db4009c00 comp rx=0 tx=0).stop 2024-04-02T14:41:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.677+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3428095884 shutdown_connections 2024-04-02T14:41:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.677+0000 7f0dc8a00700 1 --2- 172.21.15.62:0/3428095884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc410a360 0x7f0dc410c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.677+0000 7f0dc8a00700 1 --2- 172.21.15.62:0/3428095884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc4076620 0x7f0dc4074a80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.677+0000 7f0dc8a00700 1 --2- 172.21.15.62:0/3428095884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dc4075050 0x7f0dc40754d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.677+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3428095884 >> 172.21.15.62:0/3428095884 conn(0x7f0dc40fda30 msgr2=0x7f0dc40ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.677+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3428095884 shutdown_connections 2024-04-02T14:41:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.677+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3428095884 wait complete. 2024-04-02T14:41:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.679+0000 7f0dc8a00700 1 Processor -- start 2024-04-02T14:41:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.679+0000 7f0dc8a00700 1 -- start start 2024-04-02T14:41:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.679+0000 7f0dc8a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dc4075050 0x7f0dc419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.679+0000 7f0dc8a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc4076620 0x7f0dc419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.680+0000 7f0dc8a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc410a360 0x7f0dc41a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.680+0000 7f0dc8a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4110390 con 0x7f0dc4076620 2024-04-02T14:41:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.680+0000 7f0dc8a00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4110210 con 0x7f0dc4075050 2024-04-02T14:41:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.680+0000 7f0dc8a00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4110510 con 0x7f0dc410a360 2024-04-02T14:41:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.680+0000 7f0dc2a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc4076620 0x7f0dc419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.680+0000 7f0dc2a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc4076620 0x7f0dc419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60192/0 (socket says 172.21.15.62:60192) 2024-04-02T14:41:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.680+0000 7f0dc2a00700 1 -- 172.21.15.62:0/3556340296 learned_addr learned my addr 172.21.15.62:0/3556340296 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.680+0000 7f0dc3400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dc4075050 0x7f0dc419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:35.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.680+0000 7f0dc3e00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc410a360 0x7f0dc41a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.680+0000 7f0dc3400700 1 -- 172.21.15.62:0/3556340296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc410a360 msgr2=0x7f0dc41a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.681+0000 7f0dc3400700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc410a360 0x7f0dc41a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.681+0000 7f0dc3400700 1 -- 172.21.15.62:0/3556340296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc4076620 msgr2=0x7f0dc419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.681+0000 7f0dc3400700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc4076620 0x7f0dc419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.681+0000 7f0dc3400700 1 -- 172.21.15.62:0/3556340296 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc41a25a0 con 0x7f0dc4075050 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.681+0000 7f0dc2a00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc4076620 0x7f0dc419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.681+0000 7f0dc3e00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc410a360 0x7f0dc41a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.681+0000 7f0dc3400700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dc4075050 0x7f0dc419a940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009230 tx=0x7f0db4005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.681+0000 7f0dbbe00700 1 -- 172.21.15.62:0/3556340296 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4005460 con 0x7f0dc4075050 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.681+0000 7f0dbbe00700 1 -- 172.21.15.62:0/3556340296 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0db4005600 con 0x7f0dc4075050 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.681+0000 7f0dbbe00700 1 -- 172.21.15.62:0/3556340296 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4003d60 con 0x7f0dc4075050 2024-04-02T14:41:35.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.681+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3556340296 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc41a2830 con 0x7f0dc4075050 2024-04-02T14:41:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.682+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3556340296 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc41a2ce0 con 0x7f0dc4075050 2024-04-02T14:41:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.683+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3556340296 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc4063c20 con 0x7f0dc4075050 2024-04-02T14:41:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.685+0000 7f0dbbe00700 1 -- 172.21.15.62:0/3556340296 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0db400b560 con 0x7f0dc4075050 2024-04-02T14:41:35.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.686+0000 7f0dbbe00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0da80a3e00 0x7f0da80a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:35.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.686+0000 7f0dbbe00700 1 -- 172.21.15.62:0/3556340296 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f0db40d1640 con 0x7f0dc4075050 2024-04-02T14:41:35.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.686+0000 7f0dc2a00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0da80a3e00 0x7f0da80a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:35.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.687+0000 7f0dc2a00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0da80a3e00 0x7f0da80a62c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0db0005950 tx=0x7f0db0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:35.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:35.689+0000 7f0dbbe00700 1 -- 172.21.15.62:0/3556340296 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0db4013030 con 0x7f0dc4075050 2024-04-02T14:41:36.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.013+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3556340296 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f0dc402dd60 con 0x7f0dc4075050 2024-04-02T14:41:36.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.019+0000 7f0dbbe00700 1 -- 172.21.15.62:0/3556340296 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f0db409e120 con 0x7f0dc4075050 2024-04-02T14:41:36.019 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-02T14:41:36.019 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAgGQxmUHbjABAAH5cTsOLrqEwjvbvaI+DhaQ== 2024-04-02T14:41:36.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.023+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3556340296 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0da80a3e00 msgr2=0x7f0da80a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:36.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.023+0000 7f0dc8a00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0da80a3e00 0x7f0da80a62c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0db0005950 tx=0x7f0db0009260 comp rx=0 tx=0).stop 2024-04-02T14:41:36.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.023+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3556340296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dc4075050 msgr2=0x7f0dc419a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:36.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.023+0000 7f0dc8a00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dc4075050 0x7f0dc419a940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009230 tx=0x7f0db4005270 comp rx=0 tx=0).stop 2024-04-02T14:41:36.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.023+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3556340296 shutdown_connections 2024-04-02T14:41:36.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.023+0000 7f0dc8a00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0da80a3e00 0x7f0da80a62c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:36.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.023+0000 7f0dc8a00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dc4075050 0x7f0dc419a940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:36.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.023+0000 7f0dc8a00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc410a360 0x7f0dc41a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:36.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.023+0000 7f0dc8a00700 1 --2- 172.21.15.62:0/3556340296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc4076620 0x7f0dc419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:36.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.023+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3556340296 >> 172.21.15.62:0/3556340296 conn(0x7f0dc40fda30 msgr2=0x7f0dc4106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:36.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.024+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3556340296 shutdown_connections 2024-04-02T14:41:36.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:36.024+0000 7f0dc8a00700 1 -- 172.21.15.62:0/3556340296 wait complete. 2024-04-02T14:41:36.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:36 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3556340296' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T14:41:36.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:36 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T14:41:36.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:36 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T14:41:36.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:36 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3556340296' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T14:41:36.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:36 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T14:41:36.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:36 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T14:41:36.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:36 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3556340296' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T14:41:36.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:36 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T14:41:36.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:36 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T14:41:37.127 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:41:37.127 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-02T14:41:37.127 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-02T14:41:37.295 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-02T14:41:37.295 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-02T14:41:37.295 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph mgr dump --format=json 2024-04-02T14:41:37.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:37 smithi062 ceph-mon[32116]: pgmap v326: 97 pgs: 97 active+clean; 627 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-04-02T14:41:37.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:37 smithi204 ceph-mon[35948]: pgmap v326: 97 pgs: 97 active+clean; 627 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-04-02T14:41:37.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:37 smithi113 ceph-mon[36135]: pgmap v326: 97 pgs: 97 active+clean; 627 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-04-02T14:41:37.802 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:39.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:39 smithi062 ceph-mon[32116]: pgmap v327: 97 pgs: 97 active+clean; 627 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 KiB/s wr, 22 op/s 2024-04-02T14:41:39.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:39 smithi204 ceph-mon[35948]: pgmap v327: 97 pgs: 97 active+clean; 627 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 KiB/s wr, 22 op/s 2024-04-02T14:41:39.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:39 smithi113 ceph-mon[36135]: pgmap v327: 97 pgs: 97 active+clean; 627 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 KiB/s wr, 22 op/s 2024-04-02T14:41:40.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.269+0000 7f5829200700 1 -- 172.21.15.62:0/4072789099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5824102eb0 msgr2=0x7f582410f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:40.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.269+0000 7f5829200700 1 --2- 172.21.15.62:0/4072789099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5824102eb0 0x7f582410f370 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f581c00b0d0 tx=0x7f581c00b3e0 comp rx=0 tx=0).stop 2024-04-02T14:41:40.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.269+0000 7f5829200700 1 -- 172.21.15.62:0/4072789099 shutdown_connections 2024-04-02T14:41:40.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.269+0000 7f5829200700 1 --2- 172.21.15.62:0/4072789099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58241085a0 0x7f5824108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:40.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.269+0000 7f5829200700 1 --2- 172.21.15.62:0/4072789099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5824102eb0 0x7f582410f370 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:40.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.269+0000 7f5829200700 1 --2- 172.21.15.62:0/4072789099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58241024f0 0x7f5824102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:40.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.269+0000 7f5829200700 1 -- 172.21.15.62:0/4072789099 >> 172.21.15.62:0/4072789099 conn(0x7f58240fdfc0 msgr2=0x7f58241003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:40.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.269+0000 7f5829200700 1 -- 172.21.15.62:0/4072789099 shutdown_connections 2024-04-02T14:41:40.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.269+0000 7f5829200700 1 -- 172.21.15.62:0/4072789099 wait complete. 2024-04-02T14:41:40.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.270+0000 7f5829200700 1 Processor -- start 2024-04-02T14:41:40.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.270+0000 7f5829200700 1 -- start start 2024-04-02T14:41:40.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.271+0000 7f5829200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58241024f0 0x7f582419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:40.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.271+0000 7f5829200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5824102eb0 0x7f582419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:40.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.271+0000 7f5829200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241085a0 0x7f5824198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:40.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.271+0000 7f5829200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5824112980 con 0x7f58241085a0 2024-04-02T14:41:40.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.271+0000 7f5829200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5824112800 con 0x7f5824102eb0 2024-04-02T14:41:40.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.271+0000 7f5829200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5824112b00 con 0x7f58241024f0 2024-04-02T14:41:40.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.271+0000 7f5823e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241085a0 0x7f5824198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:40.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.271+0000 7f5823e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241085a0 0x7f5824198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60200/0 (socket says 172.21.15.62:60200) 2024-04-02T14:41:40.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.271+0000 7f5823e00700 1 -- 172.21.15.62:0/151731612 learned_addr learned my addr 172.21.15.62:0/151731612 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:40.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.271+0000 7f5823400700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58241024f0 0x7f582419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:40.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.271+0000 7f5822a00700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5824102eb0 0x7f582419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:40.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.272+0000 7f5823400700 1 -- 172.21.15.62:0/151731612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5824102eb0 msgr2=0x7f582419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:40.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.272+0000 7f5823400700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5824102eb0 0x7f582419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:40.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.272+0000 7f5823400700 1 -- 172.21.15.62:0/151731612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241085a0 msgr2=0x7f5824198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:40.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.272+0000 7f5823400700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241085a0 0x7f5824198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:40.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.272+0000 7f5823400700 1 -- 172.21.15.62:0/151731612 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5824199200 con 0x7f58241024f0 2024-04-02T14:41:40.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.272+0000 7f5823e00700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241085a0 0x7f5824198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:40.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.272+0000 7f5823400700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58241024f0 0x7f582419e8e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f581400e9f0 tx=0x7f581400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:40.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.272+0000 7f581be00700 1 -- 172.21.15.62:0/151731612 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581400c9c0 con 0x7f58241024f0 2024-04-02T14:41:40.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.272+0000 7f581be00700 1 -- 172.21.15.62:0/151731612 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f581400cb60 con 0x7f58241024f0 2024-04-02T14:41:40.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.272+0000 7f5829200700 1 -- 172.21.15.62:0/151731612 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58241994f0 con 0x7f58241024f0 2024-04-02T14:41:40.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.272+0000 7f581be00700 1 -- 172.21.15.62:0/151731612 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814018680 con 0x7f58241024f0 2024-04-02T14:41:40.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.273+0000 7f5829200700 1 -- 172.21.15.62:0/151731612 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58241a57b0 con 0x7f58241024f0 2024-04-02T14:41:40.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.273+0000 7f5829200700 1 -- 172.21.15.62:0/151731612 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58240692b0 con 0x7f58241024f0 2024-04-02T14:41:40.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.275+0000 7f581be00700 1 -- 172.21.15.62:0/151731612 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5814010430 con 0x7f58241024f0 2024-04-02T14:41:40.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.277+0000 7f581be00700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58080a3d50 0x7f58080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:40.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.277+0000 7f581be00700 1 -- 172.21.15.62:0/151731612 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f58140d0610 con 0x7f58241024f0 2024-04-02T14:41:40.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.277+0000 7f5822a00700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58080a3d50 0x7f58080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:40.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.278+0000 7f5822a00700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58080a3d50 0x7f58080a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5824107aa0 tx=0x7f5810009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:40.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.279+0000 7f581be00700 1 -- 172.21.15.62:0/151731612 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f581409cf40 con 0x7f58241024f0 2024-04-02T14:41:40.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.578+0000 7f5829200700 1 -- 172.21.15.62:0/151731612 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f58241a59e0 con 0x7f58241024f0 2024-04-02T14:41:40.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.584+0000 7f581be00700 1 -- 172.21.15.62:0/151731612 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264994 (secure 0 0 0) 0x7f581409c840 con 0x7f58241024f0 2024-04-02T14:41:40.587 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.590+0000 7f5829200700 1 -- 172.21.15.62:0/151731612 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58080a3d50 msgr2=0x7f58080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.590+0000 7f5829200700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58080a3d50 0x7f58080a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5824107aa0 tx=0x7f5810009260 comp rx=0 tx=0).stop 2024-04-02T14:41:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.590+0000 7f5829200700 1 -- 172.21.15.62:0/151731612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58241024f0 msgr2=0x7f582419e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.590+0000 7f5829200700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58241024f0 0x7f582419e8e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f581400e9f0 tx=0x7f581400ed00 comp rx=0 tx=0).stop 2024-04-02T14:41:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.590+0000 7f5829200700 1 -- 172.21.15.62:0/151731612 shutdown_connections 2024-04-02T14:41:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.590+0000 7f5829200700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f58080a3d50 0x7f58080a6210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.590+0000 7f5829200700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58241085a0 0x7f5824198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.590+0000 7f5829200700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58241024f0 0x7f582419e8e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.590+0000 7f5829200700 1 --2- 172.21.15.62:0/151731612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5824102eb0 0x7f582419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.590+0000 7f5829200700 1 -- 172.21.15.62:0/151731612 >> 172.21.15.62:0/151731612 conn(0x7f58240fdfc0 msgr2=0x7f58240ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.590+0000 7f5829200700 1 -- 172.21.15.62:0/151731612 shutdown_connections 2024-04-02T14:41:40.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:40.591+0000 7f5829200700 1 -- 172.21.15.62:0/151731612 wait complete. 2024-04-02T14:41:41.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:41 smithi204 ceph-mon[35948]: pgmap v328: 97 pgs: 97 active+clean; 627 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-02T14:41:41.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:41 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/151731612' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T14:41:41.519 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:41 smithi062 ceph-mon[32116]: pgmap v328: 97 pgs: 97 active+clean; 627 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-02T14:41:41.519 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:41 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/151731612' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T14:41:41.593 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1995176952},{"type":"v1","addr":"172.21.15.62:6801","nonce":1995176952}]},"active_addr":"172.21.15.62:6801/1995176952","active_change":"2024-04-02T14:33:19.212067+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24110,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.62:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":129789951}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":566820061}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":79979639}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":3400451184}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":1886123984}]}]} 2024-04-02T14:41:41.598 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-02T14:41:41.598 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-02T14:41:41.598 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T14:41:41.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:41 smithi113 ceph-mon[36135]: pgmap v328: 97 pgs: 97 active+clean; 627 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-02T14:41:41.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:41 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/151731612' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T14:41:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:43.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:43 smithi204 ceph-mon[35948]: pgmap v329: 97 pgs: 97 active+clean; 631 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 23 op/s 2024-04-02T14:41:43.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:43 smithi062 ceph-mon[32116]: pgmap v329: 97 pgs: 97 active+clean; 631 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 23 op/s 2024-04-02T14:41:43.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:43 smithi113 ceph-mon[36135]: pgmap v329: 97 pgs: 97 active+clean; 631 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 23 op/s 2024-04-02T14:41:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.472+0000 7fc6e2800700 1 -- 172.21.15.62:0/527234391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc06a7e0 msgr2=0x7fc6dc06ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.472+0000 7fc6e2800700 1 --2- 172.21.15.62:0/527234391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc06a7e0 0x7fc6dc06ac40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009990 tx=0x7fc6c4009ca0 comp rx=0 tx=0).stop 2024-04-02T14:41:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.472+0000 7fc6e2800700 1 -- 172.21.15.62:0/527234391 shutdown_connections 2024-04-02T14:41:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.472+0000 7fc6e2800700 1 --2- 172.21.15.62:0/527234391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6dc1074e0 0x7fc6dc06a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.472+0000 7fc6e2800700 1 --2- 172.21.15.62:0/527234391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6dc104ff0 0x7fc6dc105480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.472+0000 7fc6e2800700 1 --2- 172.21.15.62:0/527234391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc06a7e0 0x7fc6dc06ac40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.472+0000 7fc6e2800700 1 -- 172.21.15.62:0/527234391 >> 172.21.15.62:0/527234391 conn(0x7fc6dc078420 msgr2=0x7fc6dc078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.472+0000 7fc6e2800700 1 -- 172.21.15.62:0/527234391 shutdown_connections 2024-04-02T14:41:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.472+0000 7fc6e2800700 1 -- 172.21.15.62:0/527234391 wait complete. 2024-04-02T14:41:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.473+0000 7fc6e2800700 1 Processor -- start 2024-04-02T14:41:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.473+0000 7fc6e2800700 1 -- start start 2024-04-02T14:41:44.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.474+0000 7fc6e2800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc06a7e0 0x7fc6dc19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.474+0000 7fc6e2800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6dc104ff0 0x7fc6dc19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.474+0000 7fc6e2800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6dc1074e0 0x7fc6dc1a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.474+0000 7fc6e2800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc119060 con 0x7fc6dc1074e0 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.474+0000 7fc6e2800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc118ee0 con 0x7fc6dc104ff0 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.474+0000 7fc6e2800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc1191e0 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.474+0000 7fc6db400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc06a7e0 0x7fc6dc19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.474+0000 7fc6db400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc06a7e0 0x7fc6dc19f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39364/0 (socket says 172.21.15.62:39364) 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.474+0000 7fc6db400700 1 -- 172.21.15.62:0/703953535 learned_addr learned my addr 172.21.15.62:0/703953535 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.474+0000 7fc6dbe00700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6dc1074e0 0x7fc6dc1a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.474+0000 7fc6d2a00700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6dc104ff0 0x7fc6dc19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.475+0000 7fc6db400700 1 -- 172.21.15.62:0/703953535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6dc104ff0 msgr2=0x7fc6dc19f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.475+0000 7fc6db400700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6dc104ff0 0x7fc6dc19f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.475+0000 7fc6db400700 1 -- 172.21.15.62:0/703953535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6dc1074e0 msgr2=0x7fc6dc1a3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.475+0000 7fc6db400700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6dc1074e0 0x7fc6dc1a3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.475+0000 7fc6db400700 1 -- 172.21.15.62:0/703953535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6dc1a4450 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.475+0000 7fc6d2a00700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6dc104ff0 0x7fc6dc19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.475+0000 7fc6db400700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc06a7e0 0x7fc6dc19f490 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00e9a0 tx=0x7fc6cc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.475+0000 7fc6d8c00700 1 -- 172.21.15.62:0/703953535 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00ca10 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.475+0000 7fc6d8c00700 1 -- 172.21.15.62:0/703953535 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc6cc00cbb0 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.475+0000 7fc6e2800700 1 -- 172.21.15.62:0/703953535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6dc1a4740 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.475+0000 7fc6d8c00700 1 -- 172.21.15.62:0/703953535 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc018710 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.476+0000 7fc6e2800700 1 -- 172.21.15.62:0/703953535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6dc1abfc0 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.477+0000 7fc6e2800700 1 -- 172.21.15.62:0/703953535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6dc0692b0 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.479+0000 7fc6d8c00700 1 -- 172.21.15.62:0/703953535 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc6cc010430 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.480+0000 7fc6d8c00700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc6c00a3d50 0x7fc6c00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:44.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.480+0000 7fc6d8c00700 1 -- 172.21.15.62:0/703953535 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc6cc0d05b0 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.480+0000 7fc6d2a00700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc6c00a3d50 0x7fc6c00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:44.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.481+0000 7fc6d2a00700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc6c00a3d50 0x7fc6c00a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc078ad0 tx=0x7fc6c401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:44.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.484+0000 7fc6d8c00700 1 -- 172.21.15.62:0/703953535 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc6cc09cee0 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.715+0000 7fc6e2800700 1 -- 172.21.15.62:0/703953535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6dc1ac1f0 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.717+0000 7fc6d8c00700 1 -- 172.21.15.62:0/703953535 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22197 (secure 0 0 0) 0x7fc6cc09c7e0 con 0x7fc6dc06a7e0 2024-04-02T14:41:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:44.717 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:41:23.500989+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:41:44.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 -- 172.21.15.62:0/703953535 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc6c00a3d50 msgr2=0x7fc6c00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:44.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc6c00a3d50 0x7fc6c00a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc078ad0 tx=0x7fc6c401a040 comp rx=0 tx=0).stop 2024-04-02T14:41:44.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 -- 172.21.15.62:0/703953535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc06a7e0 msgr2=0x7fc6dc19f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:44.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc06a7e0 0x7fc6dc19f490 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00e9a0 tx=0x7fc6cc00ecb0 comp rx=0 tx=0).stop 2024-04-02T14:41:44.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 -- 172.21.15.62:0/703953535 shutdown_connections 2024-04-02T14:41:44.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc6c00a3d50 0x7fc6c00a6210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:44.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6dc1074e0 0x7fc6dc1a3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:44.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc06a7e0 0x7fc6dc19f490 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:44.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 --2- 172.21.15.62:0/703953535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6dc104ff0 0x7fc6dc19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:44.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 -- 172.21.15.62:0/703953535 >> 172.21.15.62:0/703953535 conn(0x7fc6dc078420 msgr2=0x7fc6dc1004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:44.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 -- 172.21.15.62:0/703953535 shutdown_connections 2024-04-02T14:41:44.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:44.720+0000 7fc6e2800700 1 -- 172.21.15.62:0/703953535 wait complete. 2024-04-02T14:41:45.421 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:45 smithi113 ceph-mon[36135]: pgmap v330: 97 pgs: 97 active+clean; 631 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-04-02T14:41:45.421 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:45 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/703953535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:45.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:45 smithi204 ceph-mon[35948]: pgmap v330: 97 pgs: 97 active+clean; 631 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-04-02T14:41:45.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:45 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/703953535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:45.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:45 smithi062 ceph-mon[32116]: pgmap v330: 97 pgs: 97 active+clean; 631 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-04-02T14:41:45.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:45 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/703953535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:45.787 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-02T14:41:45.788 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T14:41:46.249 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:47.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:47 smithi062 ceph-mon[32116]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 KiB/s wr, 16 op/s 2024-04-02T14:41:47.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:47 smithi204 ceph-mon[35948]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 KiB/s wr, 16 op/s 2024-04-02T14:41:47.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:47 smithi113 ceph-mon[36135]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 KiB/s wr, 16 op/s 2024-04-02T14:41:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.505+0000 7ff37d200700 1 -- 172.21.15.62:0/2375740751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff37810df40 msgr2=0x7ff378115b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.505+0000 7ff37d200700 1 --2- 172.21.15.62:0/2375740751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff37810df40 0x7ff378115b10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff37000b0d0 tx=0x7ff37000b3e0 comp rx=0 tx=0).stop 2024-04-02T14:41:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.505+0000 7ff37d200700 1 -- 172.21.15.62:0/2375740751 shutdown_connections 2024-04-02T14:41:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.505+0000 7ff37d200700 1 --2- 172.21.15.62:0/2375740751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff37810df40 0x7ff378115b10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.505+0000 7ff37d200700 1 --2- 172.21.15.62:0/2375740751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff378108e60 0x7ff378109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.505+0000 7ff37d200700 1 --2- 172.21.15.62:0/2375740751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff378109810 0x7ff37810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.505+0000 7ff37d200700 1 -- 172.21.15.62:0/2375740751 >> 172.21.15.62:0/2375740751 conn(0x7ff378075dc0 msgr2=0x7ff3780781e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.506+0000 7ff37d200700 1 -- 172.21.15.62:0/2375740751 shutdown_connections 2024-04-02T14:41:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.506+0000 7ff37d200700 1 -- 172.21.15.62:0/2375740751 wait complete. 2024-04-02T14:41:48.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.507+0000 7ff37d200700 1 Processor -- start 2024-04-02T14:41:48.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.507+0000 7ff37d200700 1 -- start start 2024-04-02T14:41:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.508+0000 7ff37d200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff378108e60 0x7ff37819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.508+0000 7ff37d200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff378109810 0x7ff37819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.508+0000 7ff37d200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff37810df40 0x7ff3781a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.508+0000 7ff37d200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3781190d0 con 0x7ff378108e60 2024-04-02T14:41:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.508+0000 7ff37d200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff378118f50 con 0x7ff37810df40 2024-04-02T14:41:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.508+0000 7ff37d200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff378119250 con 0x7ff378109810 2024-04-02T14:41:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.508+0000 7ff375e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff378108e60 0x7ff37819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.508+0000 7ff375e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff378108e60 0x7ff37819f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57766/0 (socket says 172.21.15.62:57766) 2024-04-02T14:41:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.508+0000 7ff375e00700 1 -- 172.21.15.62:0/691463863 learned_addr learned my addr 172.21.15.62:0/691463863 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:41:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.508+0000 7ff376800700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff37810df40 0x7ff3781a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff375400700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff378109810 0x7ff37819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff375e00700 1 -- 172.21.15.62:0/691463863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff378109810 msgr2=0x7ff37819f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff375e00700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff378109810 0x7ff37819f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff375e00700 1 -- 172.21.15.62:0/691463863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff37810df40 msgr2=0x7ff3781a3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff375e00700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff37810df40 0x7ff3781a3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff375e00700 1 -- 172.21.15.62:0/691463863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3781a4450 con 0x7ff378108e60 2024-04-02T14:41:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff375400700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff378109810 0x7ff37819f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:41:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff376800700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff37810df40 0x7ff3781a3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:41:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff375e00700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff378108e60 0x7ff37819f440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff36800e9a0 tx=0x7ff36800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff36ea00700 1 -- 172.21.15.62:0/691463863 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36800cbd0 con 0x7ff378108e60 2024-04-02T14:41:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff36ea00700 1 -- 172.21.15.62:0/691463863 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff36800cd70 con 0x7ff378108e60 2024-04-02T14:41:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff37d200700 1 -- 172.21.15.62:0/691463863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3781a4740 con 0x7ff378108e60 2024-04-02T14:41:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.509+0000 7ff36ea00700 1 -- 172.21.15.62:0/691463863 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3680187c0 con 0x7ff378108e60 2024-04-02T14:41:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.510+0000 7ff37d200700 1 -- 172.21.15.62:0/691463863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3781abfc0 con 0x7ff378108e60 2024-04-02T14:41:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.511+0000 7ff37d200700 1 -- 172.21.15.62:0/691463863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3780692b0 con 0x7ff378108e60 2024-04-02T14:41:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.513+0000 7ff36ea00700 1 -- 172.21.15.62:0/691463863 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff368010430 con 0x7ff378108e60 2024-04-02T14:41:48.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.514+0000 7ff36ea00700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff35c0a3d50 0x7ff35c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:41:48.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.515+0000 7ff36ea00700 1 -- 172.21.15.62:0/691463863 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7ff3680d06b0 con 0x7ff378108e60 2024-04-02T14:41:48.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.515+0000 7ff375400700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff35c0a3d50 0x7ff35c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:41:48.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.515+0000 7ff375400700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff35c0a3d50 0x7ff35c0a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff378107da0 tx=0x7ff364009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:41:48.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.519+0000 7ff36ea00700 1 -- 172.21.15.62:0/691463863 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff36809cfe0 con 0x7ff378108e60 2024-04-02T14:41:48.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.755+0000 7ff37d200700 1 -- 172.21.15.62:0/691463863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3781ac1f0 con 0x7ff378108e60 2024-04-02T14:41:48.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.759+0000 7ff36ea00700 1 -- 172.21.15.62:0/691463863 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22197 (secure 0 0 0) 0x7ff36809c8e0 con 0x7ff378108e60 2024-04-02T14:41:48.759 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:41:48.759 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:41:23.500989+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:41:48.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.762+0000 7ff37d200700 1 -- 172.21.15.62:0/691463863 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff35c0a3d50 msgr2=0x7ff35c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:48.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.762+0000 7ff37d200700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff35c0a3d50 0x7ff35c0a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff378107da0 tx=0x7ff364009260 comp rx=0 tx=0).stop 2024-04-02T14:41:48.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.762+0000 7ff37d200700 1 -- 172.21.15.62:0/691463863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff378108e60 msgr2=0x7ff37819f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:41:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.762+0000 7ff37d200700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff378108e60 0x7ff37819f440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff36800e9a0 tx=0x7ff36800ecb0 comp rx=0 tx=0).stop 2024-04-02T14:41:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.762+0000 7ff37d200700 1 -- 172.21.15.62:0/691463863 shutdown_connections 2024-04-02T14:41:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.762+0000 7ff37d200700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff35c0a3d50 0x7ff35c0a6210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.762+0000 7ff37d200700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff37810df40 0x7ff3781a3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.762+0000 7ff37d200700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff378108e60 0x7ff37819f440 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.762+0000 7ff37d200700 1 --2- 172.21.15.62:0/691463863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff378109810 0x7ff37819f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:41:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.762+0000 7ff37d200700 1 -- 172.21.15.62:0/691463863 >> 172.21.15.62:0/691463863 conn(0x7ff378075dc0 msgr2=0x7ff3780766f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:41:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.763+0000 7ff37d200700 1 -- 172.21.15.62:0/691463863 shutdown_connections 2024-04-02T14:41:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:48.763+0000 7ff37d200700 1 -- 172.21.15.62:0/691463863 wait complete. 2024-04-02T14:41:49.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:49 smithi204 ceph-mon[35948]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-02T14:41:49.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:49 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/691463863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:49.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:49 smithi062 ceph-mon[32116]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-02T14:41:49.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:49 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/691463863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:49.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:49 smithi113 ceph-mon[36135]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-02T14:41:49.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:49 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/691463863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:41:49.669 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-04-02T14:41:49.670 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-04-02T14:41:49.671 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-04-02T14:41:49.671 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-04-02T14:41:49.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-04-02T14:41:49.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-04-02T14:41:49.673 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.6 flush_pg_stats 2024-04-02T14:41:49.673 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.7 flush_pg_stats 2024-04-02T14:41:49.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.8 flush_pg_stats 2024-04-02T14:41:49.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.9 flush_pg_stats 2024-04-02T14:41:49.675 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.10 flush_pg_stats 2024-04-02T14:41:49.675 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph tell osd.11 flush_pg_stats 2024-04-02T14:41:50.844 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:50.848 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:50.848 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:50.851 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:50.862 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:50.862 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:50.871 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:50.903 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:41:51.298 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:51 smithi062 ceph-mon[32116]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-02T14:41:51.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:51 smithi204 ceph-mon[35948]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-02T14:41:51.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:51 smithi113 ceph-mon[36135]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-02T14:41:53.310 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:53 smithi062 ceph-mon[32116]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-02T14:41:53.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:53 smithi204 ceph-mon[35948]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-02T14:41:53.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:53 smithi113 ceph-mon[36135]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-02T14:41:55.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:55 smithi062 ceph-mon[32116]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-02T14:41:55.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:55 smithi204 ceph-mon[35948]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-02T14:41:55.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:55 smithi113 ceph-mon[36135]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-02T14:41:57.439 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:57 smithi062 ceph-mon[32116]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-02T14:41:57.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:57 smithi204 ceph-mon[35948]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-02T14:41:57.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:57 smithi113 ceph-mon[36135]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-02T14:41:59.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:41:59 smithi062 ceph-mon[32116]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T14:41:59.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:41:59 smithi204 ceph-mon[35948]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T14:41:59.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:41:59 smithi113 ceph-mon[36135]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T14:42:00.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.786+0000 7f0da2a00700 1 -- 172.21.15.62:0/2426658189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c108650 msgr2=0x7f0d9c108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:00.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.786+0000 7f0da2a00700 1 --2- 172.21.15.62:0/2426658189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c108650 0x7f0d9c108a30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009920 tx=0x7f0d8c009c30 comp rx=0 tx=0).stop 2024-04-02T14:42:00.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.786+0000 7f0da2a00700 1 -- 172.21.15.62:0/2426658189 shutdown_connections 2024-04-02T14:42:00.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.786+0000 7f0da2a00700 1 --2- 172.21.15.62:0/2426658189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d9c06ac60 0x7f0d9c10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.786+0000 7f0da2a00700 1 --2- 172.21.15.62:0/2426658189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c108650 0x7f0d9c108a30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.786+0000 7f0da2a00700 1 --2- 172.21.15.62:0/2426658189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c06a2a0 0x7f0d9c06a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.786+0000 7f0da2a00700 1 -- 172.21.15.62:0/2426658189 >> 172.21.15.62:0/2426658189 conn(0x7f0d9c078420 msgr2=0x7f0d9c078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.786+0000 7f0da2a00700 1 -- 172.21.15.62:0/2426658189 shutdown_connections 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.787+0000 7f0da2a00700 1 -- 172.21.15.62:0/2426658189 wait complete. 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.788+0000 7f0da2a00700 1 Processor -- start 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.788+0000 7f0da2a00700 1 -- start start 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.788+0000 7f0da2a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c06a2a0 0x7f0d9c1015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.788+0000 7f0da2a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d9c06ac60 0x7f0d9c101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.788+0000 7f0da2a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c108650 0x7f0d9c102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.788+0000 7f0da2a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c112a20 con 0x7f0d9c06ac60 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.788+0000 7f0da2a00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1128a0 con 0x7f0d9c06a2a0 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.788+0000 7f0da2a00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c112ba0 con 0x7f0d9c108650 2024-04-02T14:42:00.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0d9b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d9c06ac60 0x7f0d9c101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0d9be00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c06a2a0 0x7f0d9c1015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0d9b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d9c06ac60 0x7f0d9c101b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37454/0 (socket says 172.21.15.62:37454) 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0d9be00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c06a2a0 0x7f0d9c1015c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:35054/0 (socket says 172.21.15.62:35054) 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0d9b400700 1 -- 172.21.15.62:0/526885534 learned_addr learned my addr 172.21.15.62:0/526885534 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0da0a00700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c108650 0x7f0d9c102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0d9be00700 1 -- 172.21.15.62:0/526885534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c108650 msgr2=0x7f0d9c102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0d9be00700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c108650 0x7f0d9c102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0d9be00700 1 -- 172.21.15.62:0/526885534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d9c06ac60 msgr2=0x7f0d9c101b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0d9be00700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d9c06ac60 0x7f0d9c101b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0d9be00700 1 -- 172.21.15.62:0/526885534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c1a7420 con 0x7f0d9c06a2a0 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.789+0000 7f0d9b400700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d9c06ac60 0x7f0d9c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.790+0000 7f0d9be00700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c06a2a0 0x7f0d9c1015c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009230 tx=0x7f0d8c010f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.790+0000 7f0d98c00700 1 -- 172.21.15.62:0/526885534 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00f460 con 0x7f0d9c06a2a0 2024-04-02T14:42:00.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.790+0000 7f0d98c00700 1 -- 172.21.15.62:0/526885534 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0d8c00fb00 con 0x7f0d9c06a2a0 2024-04-02T14:42:00.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.790+0000 7f0da2a00700 1 -- 172.21.15.62:0/526885534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c1a76b0 con 0x7f0d9c06a2a0 2024-04-02T14:42:00.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.790+0000 7f0d98c00700 1 -- 172.21.15.62:0/526885534 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c003a40 con 0x7f0d9c06a2a0 2024-04-02T14:42:00.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.790+0000 7f0da2a00700 1 -- 172.21.15.62:0/526885534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c1a7b00 con 0x7f0d9c06a2a0 2024-04-02T14:42:00.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.791+0000 7f0da2a00700 1 -- 172.21.15.62:0/526885534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d9c062d00 con 0x7f0d9c06a2a0 2024-04-02T14:42:00.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.793+0000 7f0d98c00700 1 -- 172.21.15.62:0/526885534 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0d8c00b860 con 0x7f0d9c06a2a0 2024-04-02T14:42:00.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.795+0000 7f0d98c00700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0d800a3cd0 0x7f0d800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:00.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.795+0000 7f0d98c00700 1 -- 172.21.15.62:0/526885534 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f0d8c0d13e0 con 0x7f0d9c06a2a0 2024-04-02T14:42:00.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.795+0000 7f0d9b400700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0d800a3cd0 0x7f0d800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.795+0000 7f0d98c00700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940] conn(0x7f0d800ad4b0 0x7f0d800af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.795+0000 7f0d98c00700 1 -- 172.21.15.62:0/526885534 --> [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d800aff80 con 0x7f0d800ad4b0 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.795+0000 7f0d98c00700 1 -- 172.21.15.62:0/526885534 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0d8c0d8050 con 0x7f0d9c06a2a0 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.796+0000 7f0da0a00700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940] conn(0x7f0d800ad4b0 0x7f0d800af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.796+0000 7f0d9b400700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0d800a3cd0 0x7f0d800a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c078b70 tx=0x7f0d88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.796+0000 7f0da0a00700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940] conn(0x7f0d800ad4b0 0x7f0d800af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.797+0000 7f0d98c00700 1 -- 172.21.15.62:0/526885534 <== osd.10 v2:172.21.15.113:6824/4261348940 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d800aff80 con 0x7f0d800ad4b0 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.816+0000 7f0da2a00700 1 -- 172.21.15.62:0/526885534 --> [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d9c063180 con 0x7f0d800ad4b0 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.817+0000 7f0d98c00700 1 -- 172.21.15.62:0/526885534 <== osd.10 v2:172.21.15.113:6824/4261348940 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0d9c063180 con 0x7f0d800ad4b0 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.817+0000 7f0da2a00700 1 -- 172.21.15.62:0/526885534 >> [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940] conn(0x7f0d800ad4b0 msgr2=0x7f0d800af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.817+0000 7f0da2a00700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940] conn(0x7f0d800ad4b0 0x7f0d800af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.818+0000 7f0da2a00700 1 -- 172.21.15.62:0/526885534 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0d800a3cd0 msgr2=0x7f0d800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.818+0000 7f0da2a00700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f0d800a3cd0 0x7f0d800a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c078b70 tx=0x7f0d88009260 comp rx=0 tx=0).stop 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.818+0000 7f0da2a00700 1 -- 172.21.15.62:0/526885534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c06a2a0 msgr2=0x7f0d9c1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:00.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.818+0000 7f0da2a00700 1 --2- 172.21.15.62:0/526885534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c06a2a0 0x7f0d9c1015c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009230 tx=0x7f0d8c010f70 comp rx=0 tx=0).stop 2024-04-02T14:42:00.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.818+0000 7f0da0a00700 1 -- 172.21.15.62:0/526885534 reap_dead start 2024-04-02T14:42:00.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.818+0000 7f0da2a00700 1 -- 172.21.15.62:0/526885534 shutdown_connections 2024-04-02T14:42:00.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.818+0000 7f0da2a00700 1 -- 172.21.15.62:0/526885534 >> 172.21.15.62:0/526885534 conn(0x7f0d9c078420 msgr2=0x7f0d9c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:00.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.818+0000 7f0da2a00700 1 -- 172.21.15.62:0/526885534 shutdown_connections 2024-04-02T14:42:00.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:41:59.818+0000 7f0da2a00700 1 -- 172.21.15.62:0/526885534 wait complete. 2024-04-02T14:42:01.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:01 smithi062 ceph-mon[32116]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T14:42:01.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:01 smithi204 ceph-mon[35948]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T14:42:01.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:01 smithi113 ceph-mon[36135]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T14:42:03.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:03 smithi062 ceph-mon[32116]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T14:42:03.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:03 smithi204 ceph-mon[35948]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T14:42:03.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:03 smithi113 ceph-mon[36135]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T14:42:05.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:05 smithi062 ceph-mon[32116]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:05.466 INFO:teuthology.orchestra.run.smithi062.stdout:322122547226 2024-04-02T14:42:05.466 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.10 2024-04-02T14:42:05.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:05 smithi204 ceph-mon[35948]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:05.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:05 smithi113 ceph-mon[36135]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:06.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.696+0000 7f2348200700 1 -- 172.21.15.62:0/3607135035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23400749c0 msgr2=0x7f2340074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:06.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.696+0000 7f2348200700 1 --2- 172.21.15.62:0/3607135035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23400749c0 0x7f2340074da0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2334009990 tx=0x7f2334009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:06.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.696+0000 7f2348200700 1 -- 172.21.15.62:0/3607135035 shutdown_connections 2024-04-02T14:42:06.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.696+0000 7f2348200700 1 --2- 172.21.15.62:0/3607135035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23400749c0 0x7f2340074da0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:06.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.696+0000 7f2348200700 1 --2- 172.21.15.62:0/3607135035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f234010eeb0 0x7f23401112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:06.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.696+0000 7f2348200700 1 --2- 172.21.15.62:0/3607135035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23400752e0 0x7f234010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:06.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.696+0000 7f2348200700 1 -- 172.21.15.62:0/3607135035 >> 172.21.15.62:0/3607135035 conn(0x7f23400fe0c0 msgr2=0x7f23401004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:06.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.696+0000 7f2348200700 1 -- 172.21.15.62:0/3607135035 shutdown_connections 2024-04-02T14:42:06.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.696+0000 7f2348200700 1 -- 172.21.15.62:0/3607135035 wait complete. 2024-04-02T14:42:06.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.697+0000 7f2348200700 1 Processor -- start 2024-04-02T14:42:06.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.697+0000 7f2348200700 1 -- start start 2024-04-02T14:42:06.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.698+0000 7f2348200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23400749c0 0x7f234019b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:06.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.698+0000 7f2348200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23400752e0 0x7f234019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:06.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.698+0000 7f2348200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f234010eeb0 0x7f234019fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:06.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.698+0000 7f2348200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2340114d10 con 0x7f23400752e0 2024-04-02T14:42:06.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.698+0000 7f2348200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2340114b90 con 0x7f23400749c0 2024-04-02T14:42:06.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.698+0000 7f2348200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2340114e90 con 0x7f234010eeb0 2024-04-02T14:42:06.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.698+0000 7f2344e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23400752e0 0x7f234019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:06.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.698+0000 7f2344e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23400752e0 0x7f234019b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47060/0 (socket says 172.21.15.62:47060) 2024-04-02T14:42:06.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.698+0000 7f2344e00700 1 -- 172.21.15.62:0/2743034292 learned_addr learned my addr 172.21.15.62:0/2743034292 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:06.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.698+0000 7f2345800700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23400749c0 0x7f234019b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:06.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.698+0000 7f2346200700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f234010eeb0 0x7f234019fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:06.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.699+0000 7f2344e00700 1 -- 172.21.15.62:0/2743034292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f234010eeb0 msgr2=0x7f234019fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:06.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.699+0000 7f2344e00700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f234010eeb0 0x7f234019fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:06.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.699+0000 7f2344e00700 1 -- 172.21.15.62:0/2743034292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23400749c0 msgr2=0x7f234019b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:06.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.699+0000 7f2344e00700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23400749c0 0x7f234019b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:06.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.699+0000 7f2344e00700 1 -- 172.21.15.62:0/2743034292 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23401a0150 con 0x7f23400752e0 2024-04-02T14:42:06.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.699+0000 7f2345800700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23400749c0 0x7f234019b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:06.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.700+0000 7f2344e00700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23400752e0 0x7f234019b680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f232c00eb30 tx=0x7f232c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:06.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.700+0000 7f233e000700 1 -- 172.21.15.62:0/2743034292 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c00cbd0 con 0x7f23400752e0 2024-04-02T14:42:06.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.700+0000 7f2348200700 1 -- 172.21.15.62:0/2743034292 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23401a0440 con 0x7f23400752e0 2024-04-02T14:42:06.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.700+0000 7f233e000700 1 -- 172.21.15.62:0/2743034292 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f232c004520 con 0x7f23400752e0 2024-04-02T14:42:06.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.700+0000 7f233e000700 1 -- 172.21.15.62:0/2743034292 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c010430 con 0x7f23400752e0 2024-04-02T14:42:06.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.701+0000 7f2348200700 1 -- 172.21.15.62:0/2743034292 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23401a7cc0 con 0x7f23400752e0 2024-04-02T14:42:06.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.702+0000 7f2348200700 1 -- 172.21.15.62:0/2743034292 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2340062d00 con 0x7f23400752e0 2024-04-02T14:42:06.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.704+0000 7f233e000700 1 -- 172.21.15.62:0/2743034292 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f232c004000 con 0x7f23400752e0 2024-04-02T14:42:06.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.705+0000 7f233e000700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f23280a3d50 0x7f23280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:06.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.706+0000 7f233e000700 1 -- 172.21.15.62:0/2743034292 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f232c0d0b60 con 0x7f23400752e0 2024-04-02T14:42:06.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.706+0000 7f2345800700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f23280a3d50 0x7f23280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:06.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.706+0000 7f233e000700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191] conn(0x7f23280ad530 0x7f23280af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:06.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.706+0000 7f233e000700 1 -- 172.21.15.62:0/2743034292 --> [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23280b0000 con 0x7f23280ad530 2024-04-02T14:42:06.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.706+0000 7f233e000700 1 -- 172.21.15.62:0/2743034292 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f232c0d0f10 con 0x7f23400752e0 2024-04-02T14:42:06.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.706+0000 7f2346200700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191] conn(0x7f23280ad530 0x7f23280af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:06.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.707+0000 7f2345800700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f23280a3d50 0x7f23280a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2334005b20 tx=0x7f233401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:06.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.707+0000 7f2346200700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191] conn(0x7f23280ad530 0x7f23280af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:06.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.708+0000 7f233e000700 1 -- 172.21.15.62:0/2743034292 <== osd.0 v2:172.21.15.62:6802/2100120191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23280b0000 con 0x7f23280ad530 2024-04-02T14:42:06.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.726+0000 7f2348200700 1 -- 172.21.15.62:0/2743034292 --> [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2340063130 con 0x7f23280ad530 2024-04-02T14:42:06.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.727+0000 7f233e000700 1 -- 172.21.15.62:0/2743034292 <== osd.0 v2:172.21.15.62:6802/2100120191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2340063130 con 0x7f23280ad530 2024-04-02T14:42:06.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.727+0000 7f2348200700 1 -- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191] conn(0x7f23280ad530 msgr2=0x7f23280af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:06.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.727+0000 7f2348200700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191] conn(0x7f23280ad530 0x7f23280af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:06.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.727+0000 7f2348200700 1 -- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f23280a3d50 msgr2=0x7f23280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:06.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.727+0000 7f2348200700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f23280a3d50 0x7f23280a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2334005b20 tx=0x7f233401a040 comp rx=0 tx=0).stop 2024-04-02T14:42:06.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.727+0000 7f2348200700 1 -- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23400752e0 msgr2=0x7f234019b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:06.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.727+0000 7f2348200700 1 --2- 172.21.15.62:0/2743034292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23400752e0 0x7f234019b680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f232c00eb30 tx=0x7f232c00ee40 comp rx=0 tx=0).stop 2024-04-02T14:42:06.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.727+0000 7f2346200700 1 -- 172.21.15.62:0/2743034292 reap_dead start 2024-04-02T14:42:06.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.728+0000 7f2348200700 1 -- 172.21.15.62:0/2743034292 shutdown_connections 2024-04-02T14:42:06.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.728+0000 7f2348200700 1 -- 172.21.15.62:0/2743034292 >> 172.21.15.62:0/2743034292 conn(0x7f23400fe0c0 msgr2=0x7f23400fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:06.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.728+0000 7f2348200700 1 -- 172.21.15.62:0/2743034292 shutdown_connections 2024-04-02T14:42:06.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:04.728+0000 7f2348200700 1 -- 172.21.15.62:0/2743034292 wait complete. 2024-04-02T14:42:07.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:07 smithi062 ceph-mon[32116]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:42:07.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:07 smithi204 ceph-mon[35948]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:42:07.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:07 smithi113 ceph-mon[36135]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:42:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.538+0000 7f4e56000700 1 -- 172.21.15.62:0/411318082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5006ac60 msgr2=0x7f4e5010f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.538+0000 7f4e56000700 1 --2- 172.21.15.62:0/411318082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5006ac60 0x7f4e5010f450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800b0d0 tx=0x7f4e4800b3e0 comp rx=0 tx=0).stop 2024-04-02T14:42:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.538+0000 7f4e56000700 1 -- 172.21.15.62:0/411318082 shutdown_connections 2024-04-02T14:42:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.538+0000 7f4e56000700 1 --2- 172.21.15.62:0/411318082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5006ac60 0x7f4e5010f450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.538+0000 7f4e56000700 1 --2- 172.21.15.62:0/411318082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e50108650 0x7f4e50108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.538+0000 7f4e56000700 1 --2- 172.21.15.62:0/411318082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5006a2a0 0x7f4e5006a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.538+0000 7f4e56000700 1 -- 172.21.15.62:0/411318082 >> 172.21.15.62:0/411318082 conn(0x7f4e50078420 msgr2=0x7f4e50078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.538+0000 7f4e56000700 1 -- 172.21.15.62:0/411318082 shutdown_connections 2024-04-02T14:42:08.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.538+0000 7f4e56000700 1 -- 172.21.15.62:0/411318082 wait complete. 2024-04-02T14:42:08.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.540+0000 7f4e56000700 1 Processor -- start 2024-04-02T14:42:08.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.540+0000 7f4e56000700 1 -- start start 2024-04-02T14:42:08.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.540+0000 7f4e56000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5006a2a0 0x7f4e501015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:08.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.540+0000 7f4e56000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5006ac60 0x7f4e50101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:08.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.540+0000 7f4e56000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e50108650 0x7f4e50102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:08.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.540+0000 7f4e56000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50112ab0 con 0x7f4e5006a2a0 2024-04-02T14:42:08.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.540+0000 7f4e56000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50112930 con 0x7f4e5006ac60 2024-04-02T14:42:08.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.540+0000 7f4e56000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50112c30 con 0x7f4e50108650 2024-04-02T14:42:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5006a2a0 0x7f4e501015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5006a2a0 0x7f4e501015c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47110/0 (socket says 172.21.15.62:47110) 2024-04-02T14:42:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4f400700 1 -- 172.21.15.62:0/731478385 learned_addr learned my addr 172.21.15.62:0/731478385 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4ea00700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5006ac60 0x7f4e50101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4fe00700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e50108650 0x7f4e50102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4ea00700 1 -- 172.21.15.62:0/731478385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e50108650 msgr2=0x7f4e50102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4ea00700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e50108650 0x7f4e50102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4ea00700 1 -- 172.21.15.62:0/731478385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5006a2a0 msgr2=0x7f4e501015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4ea00700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5006a2a0 0x7f4e501015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4ea00700 1 -- 172.21.15.62:0/731478385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e501a7380 con 0x7f4e5006ac60 2024-04-02T14:42:08.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4f400700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5006a2a0 0x7f4e501015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:08.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.541+0000 7f4e4fe00700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e50108650 0x7f4e50102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:42:08.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.542+0000 7f4e4ea00700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5006ac60 0x7f4e50101b00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00e9f0 tx=0x7f4e3c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:08.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.542+0000 7f4e47e00700 1 -- 172.21.15.62:0/731478385 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c00cbd0 con 0x7f4e5006ac60 2024-04-02T14:42:08.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.542+0000 7f4e47e00700 1 -- 172.21.15.62:0/731478385 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e3c00cd70 con 0x7f4e5006ac60 2024-04-02T14:42:08.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.542+0000 7f4e56000700 1 -- 172.21.15.62:0/731478385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e501a7670 con 0x7f4e5006ac60 2024-04-02T14:42:08.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.542+0000 7f4e56000700 1 -- 172.21.15.62:0/731478385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e501a7c00 con 0x7f4e5006ac60 2024-04-02T14:42:08.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.542+0000 7f4e47e00700 1 -- 172.21.15.62:0/731478385 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0187a0 con 0x7f4e5006ac60 2024-04-02T14:42:08.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.543+0000 7f4e56000700 1 -- 172.21.15.62:0/731478385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e30001030 con 0x7f4e5006ac60 2024-04-02T14:42:08.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.546+0000 7f4e47e00700 1 -- 172.21.15.62:0/731478385 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4e3c018940 con 0x7f4e5006ac60 2024-04-02T14:42:08.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.547+0000 7f4e47e00700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4e340a3d50 0x7f4e340a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:08.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.547+0000 7f4e47e00700 1 -- 172.21.15.62:0/731478385 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f4e3c0d0cf0 con 0x7f4e5006ac60 2024-04-02T14:42:08.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.547+0000 7f4e4f400700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4e340a3d50 0x7f4e340a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:08.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.547+0000 7f4e47e00700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396] conn(0x7f4e340ad530 0x7f4e340af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:08.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.547+0000 7f4e47e00700 1 -- 172.21.15.62:0/731478385 --> [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e3c01de50 con 0x7f4e340ad530 2024-04-02T14:42:08.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.547+0000 7f4e47e00700 1 -- 172.21.15.62:0/731478385 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f4e3c0d10a0 con 0x7f4e5006ac60 2024-04-02T14:42:08.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.548+0000 7f4e4fe00700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396] conn(0x7f4e340ad530 0x7f4e340af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:08.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.548+0000 7f4e4f400700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4e340a3d50 0x7f4e340a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4e40011550 tx=0x7f4e40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:08.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.548+0000 7f4e4fe00700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396] conn(0x7f4e340ad530 0x7f4e340af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:08.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.549+0000 7f4e47e00700 1 -- 172.21.15.62:0/731478385 <== osd.7 v2:172.21.15.113:6816/3862122396 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e3c01de50 con 0x7f4e340ad530 2024-04-02T14:42:08.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.581+0000 7f4e56000700 1 -- 172.21.15.62:0/731478385 --> [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4e30002e60 con 0x7f4e340ad530 2024-04-02T14:42:08.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.582+0000 7f4e47e00700 1 -- 172.21.15.62:0/731478385 <== osd.7 v2:172.21.15.113:6816/3862122396 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4e30002e60 con 0x7f4e340ad530 2024-04-02T14:42:08.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.582+0000 7f4e56000700 1 -- 172.21.15.62:0/731478385 >> [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396] conn(0x7f4e340ad530 msgr2=0x7f4e340af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:08.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.582+0000 7f4e56000700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396] conn(0x7f4e340ad530 0x7f4e340af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:08.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.582+0000 7f4e56000700 1 -- 172.21.15.62:0/731478385 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4e340a3d50 msgr2=0x7f4e340a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:08.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.582+0000 7f4e56000700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4e340a3d50 0x7f4e340a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4e40011550 tx=0x7f4e40009260 comp rx=0 tx=0).stop 2024-04-02T14:42:08.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.582+0000 7f4e56000700 1 -- 172.21.15.62:0/731478385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5006ac60 msgr2=0x7f4e50101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:08.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.582+0000 7f4e56000700 1 --2- 172.21.15.62:0/731478385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5006ac60 0x7f4e50101b00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00e9f0 tx=0x7f4e3c00ed00 comp rx=0 tx=0).stop 2024-04-02T14:42:08.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.582+0000 7f4e4fe00700 1 -- 172.21.15.62:0/731478385 reap_dead start 2024-04-02T14:42:08.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.582+0000 7f4e56000700 1 -- 172.21.15.62:0/731478385 shutdown_connections 2024-04-02T14:42:08.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.582+0000 7f4e56000700 1 -- 172.21.15.62:0/731478385 >> 172.21.15.62:0/731478385 conn(0x7f4e50078420 msgr2=0x7f4e50100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:08.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.583+0000 7f4e56000700 1 -- 172.21.15.62:0/731478385 shutdown_connections 2024-04-02T14:42:08.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.583+0000 7f4e56000700 1 -- 172.21.15.62:0/731478385 wait complete. 2024-04-02T14:42:09.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:09 smithi062 ceph-mon[32116]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:09.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:09 smithi204 ceph-mon[35948]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:09.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.333+0000 7fdaca000700 1 -- 172.21.15.62:0/4276032463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac41046e0 msgr2=0x7fdac4104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.333+0000 7fdaca000700 1 --2- 172.21.15.62:0/4276032463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac41046e0 0x7fdac4104b60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009990 tx=0x7fdab0009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.334+0000 7fdaca000700 1 -- 172.21.15.62:0/4276032463 shutdown_connections 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.334+0000 7fdaca000700 1 --2- 172.21.15.62:0/4276032463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac41050a0 0x7fdac41115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.334+0000 7fdaca000700 1 --2- 172.21.15.62:0/4276032463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac410a6e0 0x7fdac410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.334+0000 7fdaca000700 1 --2- 172.21.15.62:0/4276032463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac41046e0 0x7fdac4104b60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.334+0000 7fdaca000700 1 -- 172.21.15.62:0/4276032463 >> 172.21.15.62:0/4276032463 conn(0x7fdac4100210 msgr2=0x7fdac4102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.334+0000 7fdaca000700 1 -- 172.21.15.62:0/4276032463 shutdown_connections 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.334+0000 7fdaca000700 1 -- 172.21.15.62:0/4276032463 wait complete. 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.335+0000 7fdaca000700 1 Processor -- start 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.335+0000 7fdaca000700 1 -- start start 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.335+0000 7fdaca000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac41046e0 0x7fdac419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.335+0000 7fdaca000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac41050a0 0x7fdac419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.335+0000 7fdaca000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac410a6e0 0x7fdac419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.335+0000 7fdaca000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4114b90 con 0x7fdac410a6e0 2024-04-02T14:42:09.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.335+0000 7fdaca000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4114a10 con 0x7fdac41050a0 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.335+0000 7fdaca000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4114d10 con 0x7fdac41046e0 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.336+0000 7fdac9600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac410a6e0 0x7fdac419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.336+0000 7fdac9600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac410a6e0 0x7fdac419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47116/0 (socket says 172.21.15.62:47116) 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.336+0000 7fdac9600700 1 -- 172.21.15.62:0/1261553581 learned_addr learned my addr 172.21.15.62:0/1261553581 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.336+0000 7fdac8c00700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac41046e0 0x7fdac419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.336+0000 7fdac3e00700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac41050a0 0x7fdac419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.336+0000 7fdac9600700 1 -- 172.21.15.62:0/1261553581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac41046e0 msgr2=0x7fdac419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.336+0000 7fdac9600700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac41046e0 0x7fdac419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.336+0000 7fdac9600700 1 -- 172.21.15.62:0/1261553581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac41050a0 msgr2=0x7fdac419b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.336+0000 7fdac9600700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac41050a0 0x7fdac419b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.336+0000 7fdac9600700 1 -- 172.21.15.62:0/1261553581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac419ff50 con 0x7fdac410a6e0 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.336+0000 7fdac9600700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac410a6e0 0x7fdac419f810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00ef30 tx=0x7fdabc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.337+0000 7fdac1600700 1 -- 172.21.15.62:0/1261553581 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0092e0 con 0x7fdac410a6e0 2024-04-02T14:42:09.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.337+0000 7fdac1600700 1 -- 172.21.15.62:0/1261553581 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdabc004bb0 con 0x7fdac410a6e0 2024-04-02T14:42:09.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.337+0000 7fdaca000700 1 -- 172.21.15.62:0/1261553581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac41a01e0 con 0x7fdac410a6e0 2024-04-02T14:42:09.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.337+0000 7fdac1600700 1 -- 172.21.15.62:0/1261553581 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc004280 con 0x7fdac410a6e0 2024-04-02T14:42:09.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.337+0000 7fdaca000700 1 -- 172.21.15.62:0/1261553581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac41a7ac0 con 0x7fdac410a6e0 2024-04-02T14:42:09.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.338+0000 7fdaca000700 1 -- 172.21.15.62:0/1261553581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdac41010f0 con 0x7fdac410a6e0 2024-04-02T14:42:09.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.341+0000 7fdac1600700 1 -- 172.21.15.62:0/1261553581 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdabc01b030 con 0x7fdac410a6e0 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.342+0000 7fdac1600700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdaac0a3d50 0x7fdaac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.342+0000 7fdac1600700 1 -- 172.21.15.62:0/1261553581 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fdabc0d08b0 con 0x7fdac410a6e0 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.342+0000 7fdac8c00700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdaac0a3d50 0x7fdaac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.343+0000 7fdac1600700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626] conn(0x7fdaac0ad530 0x7fdaac0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.343+0000 7fdac1600700 1 -- 172.21.15.62:0/1261553581 --> [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdaac0b0000 con 0x7fdaac0ad530 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.343+0000 7fdac1600700 1 -- 172.21.15.62:0/1261553581 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fdabc0d0c60 con 0x7fdac410a6e0 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.343+0000 7fdac3e00700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626] conn(0x7fdaac0ad530 0x7fdaac0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.343+0000 7fdac8c00700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdaac0a3d50 0x7fdaac0a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009be0 tx=0x7fdab8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.344+0000 7fdac3e00700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626] conn(0x7fdaac0ad530 0x7fdaac0af950 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.345+0000 7fdac1600700 1 -- 172.21.15.62:0/1261553581 <== osd.8 v2:172.21.15.204:6816/499713626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdaac0b0000 con 0x7fdaac0ad530 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.363+0000 7fdaca000700 1 -- 172.21.15.62:0/1261553581 --> [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdac4105e20 con 0x7fdaac0ad530 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.364+0000 7fdac1600700 1 -- 172.21.15.62:0/1261553581 <== osd.8 v2:172.21.15.204:6816/499713626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdac4105e20 con 0x7fdaac0ad530 2024-04-02T14:42:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.364+0000 7fdaca000700 1 -- 172.21.15.62:0/1261553581 >> [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626] conn(0x7fdaac0ad530 msgr2=0x7fdaac0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.365+0000 7fdaca000700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626] conn(0x7fdaac0ad530 0x7fdaac0af950 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.365+0000 7fdaca000700 1 -- 172.21.15.62:0/1261553581 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdaac0a3d50 msgr2=0x7fdaac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.365+0000 7fdaca000700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdaac0a3d50 0x7fdaac0a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009be0 tx=0x7fdab8009260 comp rx=0 tx=0).stop 2024-04-02T14:42:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.365+0000 7fdaca000700 1 -- 172.21.15.62:0/1261553581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac410a6e0 msgr2=0x7fdac419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.365+0000 7fdaca000700 1 --2- 172.21.15.62:0/1261553581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac410a6e0 0x7fdac419f810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00ef30 tx=0x7fdabc00c3a0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.365+0000 7fdac9600700 1 -- 172.21.15.62:0/1261553581 reap_dead start 2024-04-02T14:42:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.365+0000 7fdaca000700 1 -- 172.21.15.62:0/1261553581 shutdown_connections 2024-04-02T14:42:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.365+0000 7fdaca000700 1 -- 172.21.15.62:0/1261553581 >> 172.21.15.62:0/1261553581 conn(0x7fdac4100210 msgr2=0x7fdac4100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.365+0000 7fdaca000700 1 -- 172.21.15.62:0/1261553581 shutdown_connections 2024-04-02T14:42:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:09.365+0000 7fdaca000700 1 -- 172.21.15.62:0/1261553581 wait complete. 2024-04-02T14:42:09.620 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:09 smithi113 ceph-mon[36135]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:09.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.047+0000 7f9d2c200700 1 -- 172.21.15.62:0/4076044937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d24108630 msgr2=0x7f9d24108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.047+0000 7f9d2c200700 1 --2- 172.21.15.62:0/4076044937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d24108630 0x7f9d24108a10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009990 tx=0x7f9d18009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.048+0000 7f9d2c200700 1 -- 172.21.15.62:0/4076044937 shutdown_connections 2024-04-02T14:42:09.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.048+0000 7f9d2c200700 1 --2- 172.21.15.62:0/4076044937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d24103000 0x7f9d2410f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.048+0000 7f9d2c200700 1 --2- 172.21.15.62:0/4076044937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d24108630 0x7f9d24108a10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.048+0000 7f9d2c200700 1 --2- 172.21.15.62:0/4076044937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d24102640 0x7f9d24102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.048+0000 7f9d2c200700 1 -- 172.21.15.62:0/4076044937 >> 172.21.15.62:0/4076044937 conn(0x7f9d24079750 msgr2=0x7f9d24079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.048+0000 7f9d2c200700 1 -- 172.21.15.62:0/4076044937 shutdown_connections 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.048+0000 7f9d2c200700 1 -- 172.21.15.62:0/4076044937 wait complete. 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.049+0000 7f9d2c200700 1 Processor -- start 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.049+0000 7f9d2c200700 1 -- start start 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.050+0000 7f9d2c200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d24102640 0x7f9d24198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.050+0000 7f9d2c200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d24103000 0x7f9d241993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.050+0000 7f9d2c200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d24108630 0x7f9d2419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.050+0000 7f9d2c200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24112a80 con 0x7f9d24102640 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.050+0000 7f9d2c200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24112900 con 0x7f9d24108630 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.050+0000 7f9d2c200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24112c00 con 0x7f9d24103000 2024-04-02T14:42:09.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.050+0000 7f9d29800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d24102640 0x7f9d24198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.050+0000 7f9d29800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d24102640 0x7f9d24198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47096/0 (socket says 172.21.15.62:47096) 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.050+0000 7f9d29800700 1 -- 172.21.15.62:0/3466213974 learned_addr learned my addr 172.21.15.62:0/3466213974 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.050+0000 7f9d28e00700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d24103000 0x7f9d241993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.050+0000 7f9d2a200700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d24108630 0x7f9d2419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d28e00700 1 -- 172.21.15.62:0/3466213974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d24108630 msgr2=0x7f9d2419d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d28e00700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d24108630 0x7f9d2419d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d28e00700 1 -- 172.21.15.62:0/3466213974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d24102640 msgr2=0x7f9d24198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d28e00700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d24102640 0x7f9d24198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d28e00700 1 -- 172.21.15.62:0/3466213974 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2419dec0 con 0x7f9d24103000 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d2a200700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d24108630 0x7f9d2419d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d28e00700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d24103000 0x7f9d241993f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000e9f0 tx=0x7f9d1000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d22000700 1 -- 172.21.15.62:0/3466213974 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1000cbd0 con 0x7f9d24103000 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d22000700 1 -- 172.21.15.62:0/3466213974 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9d1000cd70 con 0x7f9d24103000 2024-04-02T14:42:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d2c200700 1 -- 172.21.15.62:0/3466213974 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2419e1b0 con 0x7f9d24103000 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d22000700 1 -- 172.21.15.62:0/3466213974 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d100187a0 con 0x7f9d24103000 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.051+0000 7f9d2c200700 1 -- 172.21.15.62:0/3466213974 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d241a5a30 con 0x7f9d24103000 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.052+0000 7f9d2c200700 1 -- 172.21.15.62:0/3466213974 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9d24062d00 con 0x7f9d24103000 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.055+0000 7f9d22000700 1 -- 172.21.15.62:0/3466213974 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9d10010430 con 0x7f9d24103000 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.056+0000 7f9d22000700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f9d0c0a3d50 0x7f9d0c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.056+0000 7f9d22000700 1 -- 172.21.15.62:0/3466213974 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f9d100d0e00 con 0x7f9d24103000 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.056+0000 7f9d29800700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f9d0c0a3d50 0x7f9d0c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.057+0000 7f9d22000700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038] conn(0x7f9d0c0ad530 0x7f9d0c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.057+0000 7f9d22000700 1 -- 172.21.15.62:0/3466213974 --> [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d10004500 con 0x7f9d0c0ad530 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.057+0000 7f9d22000700 1 -- 172.21.15.62:0/3466213974 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f9d100d11b0 con 0x7f9d24103000 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.057+0000 7f9d2a200700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038] conn(0x7f9d0c0ad530 0x7f9d0c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.057+0000 7f9d29800700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f9d0c0a3d50 0x7f9d0c0a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800b2e0 tx=0x7f9d18009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:09.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.058+0000 7f9d2a200700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038] conn(0x7f9d0c0ad530 0x7f9d0c0af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:09.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.059+0000 7f9d22000700 1 -- 172.21.15.62:0/3466213974 <== osd.11 v2:172.21.15.204:6824/1914333038 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9d10004500 con 0x7f9d0c0ad530 2024-04-02T14:42:09.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.077+0000 7f9d2c200700 1 -- 172.21.15.62:0/3466213974 --> [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9d24063130 con 0x7f9d0c0ad530 2024-04-02T14:42:09.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.078+0000 7f9d22000700 1 -- 172.21.15.62:0/3466213974 <== osd.11 v2:172.21.15.204:6824/1914333038 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9d24063130 con 0x7f9d0c0ad530 2024-04-02T14:42:09.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.078+0000 7f9d2c200700 1 -- 172.21.15.62:0/3466213974 >> [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038] conn(0x7f9d0c0ad530 msgr2=0x7f9d0c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.078+0000 7f9d2c200700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038] conn(0x7f9d0c0ad530 0x7f9d0c0af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:09.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.078+0000 7f9d2c200700 1 -- 172.21.15.62:0/3466213974 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f9d0c0a3d50 msgr2=0x7f9d0c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.078+0000 7f9d2c200700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f9d0c0a3d50 0x7f9d0c0a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800b2e0 tx=0x7f9d18009590 comp rx=0 tx=0).stop 2024-04-02T14:42:09.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.078+0000 7f9d2c200700 1 -- 172.21.15.62:0/3466213974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d24103000 msgr2=0x7f9d241993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:09.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.078+0000 7f9d2c200700 1 --2- 172.21.15.62:0/3466213974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d24103000 0x7f9d241993f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000e9f0 tx=0x7f9d1000ed00 comp rx=0 tx=0).stop 2024-04-02T14:42:09.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.078+0000 7f9d2a200700 1 -- 172.21.15.62:0/3466213974 reap_dead start 2024-04-02T14:42:09.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.078+0000 7f9d2c200700 1 -- 172.21.15.62:0/3466213974 shutdown_connections 2024-04-02T14:42:09.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.078+0000 7f9d2c200700 1 -- 172.21.15.62:0/3466213974 >> 172.21.15.62:0/3466213974 conn(0x7f9d24079750 msgr2=0x7f9d24100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:09.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.079+0000 7f9d2c200700 1 -- 172.21.15.62:0/3466213974 shutdown_connections 2024-04-02T14:42:09.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:08.079+0000 7f9d2c200700 1 -- 172.21.15.62:0/3466213974 wait complete. 2024-04-02T14:42:10.253 INFO:teuthology.orchestra.run.smithi062.stdout:34359738452 2024-04-02T14:42:10.254 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-04-02T14:42:10.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.936+0000 7ff000a00700 1 -- 172.21.15.62:0/2662321480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc1085a0 msgr2=0x7feffc108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:10.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.936+0000 7ff000a00700 1 --2- 172.21.15.62:0/2662321480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc1085a0 0x7feffc108980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fefec009920 tx=0x7fefec009c30 comp rx=0 tx=0).stop 2024-04-02T14:42:10.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.936+0000 7ff000a00700 1 -- 172.21.15.62:0/2662321480 shutdown_connections 2024-04-02T14:42:10.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.936+0000 7ff000a00700 1 --2- 172.21.15.62:0/2662321480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc1085a0 0x7feffc108980 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:10.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.936+0000 7ff000a00700 1 --2- 172.21.15.62:0/2662321480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc102eb0 0x7feffc10f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:10.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.936+0000 7ff000a00700 1 --2- 172.21.15.62:0/2662321480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feffc1024f0 0x7feffc102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:10.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.936+0000 7ff000a00700 1 -- 172.21.15.62:0/2662321480 >> 172.21.15.62:0/2662321480 conn(0x7feffc0fdfe0 msgr2=0x7feffc100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:10.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.936+0000 7ff000a00700 1 -- 172.21.15.62:0/2662321480 shutdown_connections 2024-04-02T14:42:10.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.937+0000 7ff000a00700 1 -- 172.21.15.62:0/2662321480 wait complete. 2024-04-02T14:42:10.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7ff000a00700 1 Processor -- start 2024-04-02T14:42:10.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7ff000a00700 1 -- start start 2024-04-02T14:42:10.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7ff000a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feffc1024f0 0x7feffc19b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7ff000a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc102eb0 0x7feffc19b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7ff000a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc1085a0 0x7feffc19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7ff000a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feffc112980 con 0x7feffc1024f0 2024-04-02T14:42:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7ff000a00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feffc112800 con 0x7feffc1085a0 2024-04-02T14:42:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7ff000a00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feffc112b00 con 0x7feffc102eb0 2024-04-02T14:42:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7feffb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feffc1024f0 0x7feffc19b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7feffb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feffc1024f0 0x7feffc19b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47086/0 (socket says 172.21.15.62:47086) 2024-04-02T14:42:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7feffb400700 1 -- 172.21.15.62:0/891939609 learned_addr learned my addr 172.21.15.62:0/891939609 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.938+0000 7feffaa00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc102eb0 0x7feffc19b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.939+0000 7feffbe00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc1085a0 0x7feffc19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:10.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.939+0000 7feffaa00700 1 -- 172.21.15.62:0/891939609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc1085a0 msgr2=0x7feffc19f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:10.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.939+0000 7feffaa00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc1085a0 0x7feffc19f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:10.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.939+0000 7feffaa00700 1 -- 172.21.15.62:0/891939609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feffc1024f0 msgr2=0x7feffc19b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:10.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.939+0000 7feffaa00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feffc1024f0 0x7feffc19b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:10.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.939+0000 7feffaa00700 1 -- 172.21.15.62:0/891939609 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc1a0040 con 0x7feffc102eb0 2024-04-02T14:42:10.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.939+0000 7feffb400700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feffc1024f0 0x7feffc19b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:10.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.939+0000 7feffaa00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc102eb0 0x7feffc19b570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fefe800eb90 tx=0x7fefe800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:10.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.939+0000 7feff3e00700 1 -- 172.21.15.62:0/891939609 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe800cc90 con 0x7feffc102eb0 2024-04-02T14:42:10.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.940+0000 7ff000a00700 1 -- 172.21.15.62:0/891939609 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feffc1a0330 con 0x7feffc102eb0 2024-04-02T14:42:10.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.940+0000 7feff3e00700 1 -- 172.21.15.62:0/891939609 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fefe8004520 con 0x7feffc102eb0 2024-04-02T14:42:10.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.940+0000 7feff3e00700 1 -- 172.21.15.62:0/891939609 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8010450 con 0x7feffc102eb0 2024-04-02T14:42:10.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.940+0000 7ff000a00700 1 -- 172.21.15.62:0/891939609 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feffc1a7bb0 con 0x7feffc102eb0 2024-04-02T14:42:10.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.941+0000 7ff000a00700 1 -- 172.21.15.62:0/891939609 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feffc0ffe10 con 0x7feffc102eb0 2024-04-02T14:42:10.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.943+0000 7feff3e00700 1 -- 172.21.15.62:0/891939609 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fefe8004000 con 0x7feffc102eb0 2024-04-02T14:42:10.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.944+0000 7feff3e00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fefe00a3cd0 0x7fefe00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:10.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.944+0000 7feff3e00700 1 -- 172.21.15.62:0/891939609 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fefe80d0b30 con 0x7feffc102eb0 2024-04-02T14:42:10.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.945+0000 7feffb400700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fefe00a3cd0 0x7fefe00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:10.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.945+0000 7feff3e00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144] conn(0x7fefe00ad4b0 0x7fefe00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:10.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.945+0000 7feff3e00700 1 -- 172.21.15.62:0/891939609 --> [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fefe8019e50 con 0x7fefe00ad4b0 2024-04-02T14:42:10.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.945+0000 7feff3e00700 1 -- 172.21.15.62:0/891939609 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fefe80d0ee0 con 0x7feffc102eb0 2024-04-02T14:42:10.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.945+0000 7feffbe00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144] conn(0x7fefe00ad4b0 0x7fefe00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:10.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.945+0000 7feffb400700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fefe00a3cd0 0x7fefe00a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fefec009680 tx=0x7fefec019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:10.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.946+0000 7feffbe00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144] conn(0x7fefe00ad4b0 0x7fefe00af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:10.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.947+0000 7feff3e00700 1 -- 172.21.15.62:0/891939609 <== osd.2 v2:172.21.15.204:6800/3439625144 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefe8019e50 con 0x7fefe00ad4b0 2024-04-02T14:42:10.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.965+0000 7ff000a00700 1 -- 172.21.15.62:0/891939609 --> [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feffc103c10 con 0x7fefe00ad4b0 2024-04-02T14:42:10.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.966+0000 7feff3e00700 1 -- 172.21.15.62:0/891939609 <== osd.2 v2:172.21.15.204:6800/3439625144 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feffc103c10 con 0x7fefe00ad4b0 2024-04-02T14:42:10.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.966+0000 7ff000a00700 1 -- 172.21.15.62:0/891939609 >> [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144] conn(0x7fefe00ad4b0 msgr2=0x7fefe00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:10.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.966+0000 7ff000a00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144] conn(0x7fefe00ad4b0 0x7fefe00af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:10.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.967+0000 7ff000a00700 1 -- 172.21.15.62:0/891939609 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fefe00a3cd0 msgr2=0x7fefe00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:10.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.967+0000 7ff000a00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fefe00a3cd0 0x7fefe00a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fefec009680 tx=0x7fefec019040 comp rx=0 tx=0).stop 2024-04-02T14:42:10.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.967+0000 7ff000a00700 1 -- 172.21.15.62:0/891939609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc102eb0 msgr2=0x7feffc19b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:10.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.967+0000 7ff000a00700 1 --2- 172.21.15.62:0/891939609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc102eb0 0x7feffc19b570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fefe800eb90 tx=0x7fefe800eea0 comp rx=0 tx=0).stop 2024-04-02T14:42:10.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.967+0000 7feffbe00700 1 -- 172.21.15.62:0/891939609 reap_dead start 2024-04-02T14:42:10.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.967+0000 7ff000a00700 1 -- 172.21.15.62:0/891939609 shutdown_connections 2024-04-02T14:42:10.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.967+0000 7ff000a00700 1 -- 172.21.15.62:0/891939609 >> 172.21.15.62:0/891939609 conn(0x7feffc0fdfe0 msgr2=0x7feffc0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:10.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.967+0000 7ff000a00700 1 -- 172.21.15.62:0/891939609 shutdown_connections 2024-04-02T14:42:10.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:05.967+0000 7ff000a00700 1 -- 172.21.15.62:0/891939609 wait complete. 2024-04-02T14:42:11.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:11 smithi204 ceph-mon[35948]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:42:11.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:11 smithi062 ceph-mon[32116]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:42:11.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:11 smithi113 ceph-mon[36135]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:42:12.657 INFO:teuthology.orchestra.run.smithi062.stdout:356482285590 2024-04-02T14:42:12.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.11 2024-04-02T14:42:13.433 INFO:teuthology.orchestra.run.smithi062.stdout:227633266733 2024-04-02T14:42:13.433 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.7 2024-04-02T14:42:13.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:13 smithi204 ceph-mon[35948]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:42:13.516 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:13 smithi062 ceph-mon[32116]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:42:13.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:13 smithi113 ceph-mon[36135]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:42:13.671 INFO:teuthology.orchestra.run.smithi062.stdout:266287972392 2024-04-02T14:42:13.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.8 2024-04-02T14:42:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.077+0000 7f798c600700 1 -- 172.21.15.62:0/4162802815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984075140 msgr2=0x7f798410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.077+0000 7f798c600700 1 --2- 172.21.15.62:0/4162802815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984075140 0x7f798410eab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7970009990 tx=0x7f7970009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.077+0000 7f798c600700 1 -- 172.21.15.62:0/4162802815 shutdown_connections 2024-04-02T14:42:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.077+0000 7f798c600700 1 --2- 172.21.15.62:0/4162802815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7984074820 0x7f7984074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.077+0000 7f798c600700 1 --2- 172.21.15.62:0/4162802815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f798410eff0 0x7f79841113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.077+0000 7f798c600700 1 --2- 172.21.15.62:0/4162802815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984075140 0x7f798410eab0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.077+0000 7f798c600700 1 -- 172.21.15.62:0/4162802815 >> 172.21.15.62:0/4162802815 conn(0x7f7984079720 msgr2=0x7f7984079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.077+0000 7f798c600700 1 -- 172.21.15.62:0/4162802815 shutdown_connections 2024-04-02T14:42:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.077+0000 7f798c600700 1 -- 172.21.15.62:0/4162802815 wait complete. 2024-04-02T14:42:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.078+0000 7f798c600700 1 Processor -- start 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.078+0000 7f798c600700 1 -- start start 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f798c600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7984074820 0x7f798419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f798c600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984075140 0x7f798419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f798c600700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798410eff0 0x7f798419f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f798c600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7984114c30 con 0x7f7984075140 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f798c600700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7984114ab0 con 0x7f798410eff0 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f798c600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7984114db0 con 0x7f7984074820 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f7989200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984075140 0x7f798419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f7989c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7984074820 0x7f798419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f7989200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984075140 0x7f798419b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60608/0 (socket says 172.21.15.62:60608) 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f7989200700 1 -- 172.21.15.62:0/2041844108 learned_addr learned my addr 172.21.15.62:0/2041844108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f798a600700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798410eff0 0x7f798419f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f7989c00700 1 -- 172.21.15.62:0/2041844108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798410eff0 msgr2=0x7f798419f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f7989c00700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798410eff0 0x7f798419f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f7989c00700 1 -- 172.21.15.62:0/2041844108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984075140 msgr2=0x7f798419b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f7989c00700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984075140 0x7f798419b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.079+0000 7f7989c00700 1 -- 172.21.15.62:0/2041844108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79841a0080 con 0x7f7984074820 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.080+0000 7f7989c00700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7984074820 0x7f798419b0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f797800e9f0 tx=0x7f797800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.080+0000 7f7982000700 1 -- 172.21.15.62:0/2041844108 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797800cb70 con 0x7f7984074820 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.080+0000 7f798c600700 1 -- 172.21.15.62:0/2041844108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79841a0370 con 0x7f7984074820 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.080+0000 7f7982000700 1 -- 172.21.15.62:0/2041844108 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f797800ee90 con 0x7f7984074820 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.080+0000 7f7982000700 1 -- 172.21.15.62:0/2041844108 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978010430 con 0x7f7984074820 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.080+0000 7f798c600700 1 -- 172.21.15.62:0/2041844108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79841a7bf0 con 0x7f7984074820 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.081+0000 7f798c600700 1 -- 172.21.15.62:0/2041844108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f79841001c0 con 0x7f7984074820 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.083+0000 7f7982000700 1 -- 172.21.15.62:0/2041844108 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7978003e90 con 0x7f7984074820 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.085+0000 7f7982000700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f796c0a3d50 0x7f796c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.085+0000 7f7982000700 1 -- 172.21.15.62:0/2041844108 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f79780d08a0 con 0x7f7984074820 2024-04-02T14:42:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.085+0000 7f7989200700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f796c0a3d50 0x7f796c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.085+0000 7f7982000700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927] conn(0x7f796c0ad530 0x7f796c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.085+0000 7f7982000700 1 -- 172.21.15.62:0/2041844108 --> [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7978012ec0 con 0x7f796c0ad530 2024-04-02T14:42:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.085+0000 7f7982000700 1 -- 172.21.15.62:0/2041844108 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f79780d0c50 con 0x7f7984074820 2024-04-02T14:42:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.085+0000 7f798a600700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927] conn(0x7f796c0ad530 0x7f796c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.086+0000 7f7989200700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f796c0a3d50 0x7f796c0a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7970009750 tx=0x7f797000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.086+0000 7f798a600700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927] conn(0x7f796c0ad530 0x7f796c0af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.087+0000 7f7982000700 1 -- 172.21.15.62:0/2041844108 <== osd.4 v2:172.21.15.113:6808/3019628927 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7978012ec0 con 0x7f796c0ad530 2024-04-02T14:42:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.105+0000 7f798c600700 1 -- 172.21.15.62:0/2041844108 --> [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7984075ec0 con 0x7f796c0ad530 2024-04-02T14:42:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.105+0000 7f7982000700 1 -- 172.21.15.62:0/2041844108 <== osd.4 v2:172.21.15.113:6808/3019628927 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7984075ec0 con 0x7f796c0ad530 2024-04-02T14:42:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.106+0000 7f798c600700 1 -- 172.21.15.62:0/2041844108 >> [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927] conn(0x7f796c0ad530 msgr2=0x7f796c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.106+0000 7f798c600700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927] conn(0x7f796c0ad530 0x7f796c0af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.106+0000 7f798c600700 1 -- 172.21.15.62:0/2041844108 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f796c0a3d50 msgr2=0x7f796c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.106+0000 7f798c600700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f796c0a3d50 0x7f796c0a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7970009750 tx=0x7f797000b280 comp rx=0 tx=0).stop 2024-04-02T14:42:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.106+0000 7f798c600700 1 -- 172.21.15.62:0/2041844108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7984074820 msgr2=0x7f798419b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.106+0000 7f798c600700 1 --2- 172.21.15.62:0/2041844108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7984074820 0x7f798419b0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f797800e9f0 tx=0x7f797800ed00 comp rx=0 tx=0).stop 2024-04-02T14:42:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.106+0000 7f798a600700 1 -- 172.21.15.62:0/2041844108 reap_dead start 2024-04-02T14:42:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.106+0000 7f798c600700 1 -- 172.21.15.62:0/2041844108 shutdown_connections 2024-04-02T14:42:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.106+0000 7f798c600700 1 -- 172.21.15.62:0/2041844108 >> 172.21.15.62:0/2041844108 conn(0x7f7984079720 msgr2=0x7f79840ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.107+0000 7f798c600700 1 -- 172.21.15.62:0/2041844108 shutdown_connections 2024-04-02T14:42:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:13.107+0000 7f798c600700 1 -- 172.21.15.62:0/2041844108 wait complete. 2024-04-02T14:42:15.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.082+0000 7f6746c00700 1 -- 172.21.15.62:0/351730079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6740108630 msgr2=0x7f6740108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:15.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.082+0000 7f6746c00700 1 --2- 172.21.15.62:0/351730079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6740108630 0x7f6740108a10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6730009990 tx=0x7f6730009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:15.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.082+0000 7f6746c00700 1 -- 172.21.15.62:0/351730079 shutdown_connections 2024-04-02T14:42:15.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.082+0000 7f6746c00700 1 --2- 172.21.15.62:0/351730079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6740103000 0x7f674010f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:15.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.082+0000 7f6746c00700 1 --2- 172.21.15.62:0/351730079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6740108630 0x7f6740108a10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:15.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.082+0000 7f6746c00700 1 --2- 172.21.15.62:0/351730079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6740102640 0x7f6740102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:15.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.082+0000 7f6746c00700 1 -- 172.21.15.62:0/351730079 >> 172.21.15.62:0/351730079 conn(0x7f6740079750 msgr2=0x7f6740079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:15.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.082+0000 7f6746c00700 1 -- 172.21.15.62:0/351730079 shutdown_connections 2024-04-02T14:42:15.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.083+0000 7f6746c00700 1 -- 172.21.15.62:0/351730079 wait complete. 2024-04-02T14:42:15.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.084+0000 7f6746c00700 1 Processor -- start 2024-04-02T14:42:15.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.084+0000 7f6746c00700 1 -- start start 2024-04-02T14:42:15.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.084+0000 7f6746c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6740102640 0x7f674019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.084+0000 7f6746c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6740103000 0x7f674019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.084+0000 7f6746c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6740108630 0x7f674019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.084+0000 7f6746c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6740112af0 con 0x7f6740103000 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.084+0000 7f6746c00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6740112970 con 0x7f6740108630 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.084+0000 7f6746c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6740112c70 con 0x7f6740102640 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.085+0000 7f6737e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6740103000 0x7f674019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.085+0000 7f6737e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6740103000 0x7f674019b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60640/0 (socket says 172.21.15.62:60640) 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.085+0000 7f6737e00700 1 -- 172.21.15.62:0/2485215925 learned_addr learned my addr 172.21.15.62:0/2485215925 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.085+0000 7f673fe00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6740102640 0x7f674019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.085+0000 7f6744c00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6740108630 0x7f674019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.085+0000 7f6737e00700 1 -- 172.21.15.62:0/2485215925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6740102640 msgr2=0x7f674019b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.085+0000 7f6737e00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6740102640 0x7f674019b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.085+0000 7f6737e00700 1 -- 172.21.15.62:0/2485215925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6740108630 msgr2=0x7f674019fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.085+0000 7f6737e00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6740108630 0x7f674019fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.085+0000 7f6737e00700 1 -- 172.21.15.62:0/2485215925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67401a0140 con 0x7f6740103000 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.085+0000 7f6737e00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6740103000 0x7f674019b670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f672800e9c0 tx=0x7f672800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.086+0000 7f673d600700 1 -- 172.21.15.62:0/2485215925 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672800cbd0 con 0x7f6740103000 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.086+0000 7f673d600700 1 -- 172.21.15.62:0/2485215925 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f672800ee60 con 0x7f6740103000 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.086+0000 7f673d600700 1 -- 172.21.15.62:0/2485215925 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728010430 con 0x7f6740103000 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.086+0000 7f6746c00700 1 -- 172.21.15.62:0/2485215925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67401a0430 con 0x7f6740103000 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.086+0000 7f6746c00700 1 -- 172.21.15.62:0/2485215925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67401a7cb0 con 0x7f6740103000 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.087+0000 7f6746c00700 1 -- 172.21.15.62:0/2485215925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6740100e90 con 0x7f6740103000 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.089+0000 7f673d600700 1 -- 172.21.15.62:0/2485215925 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6728004750 con 0x7f6740103000 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.090+0000 7f673d600700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f67240a3d50 0x7f67240a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.091+0000 7f673d600700 1 -- 172.21.15.62:0/2485215925 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f67280d0b20 con 0x7f6740103000 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.091+0000 7f673fe00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f67240a3d50 0x7f67240a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.091+0000 7f673d600700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574] conn(0x7f67240ad530 0x7f67240af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.091+0000 7f673d600700 1 -- 172.21.15.62:0/2485215925 --> [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f672801ee50 con 0x7f67240ad530 2024-04-02T14:42:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.091+0000 7f673d600700 1 -- 172.21.15.62:0/2485215925 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f67280d0ed0 con 0x7f6740103000 2024-04-02T14:42:15.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.091+0000 7f6744c00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574] conn(0x7f67240ad530 0x7f67240af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:15.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.091+0000 7f673fe00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f67240a3d50 0x7f67240a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6730006010 tx=0x7f673000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:15.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.092+0000 7f6744c00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574] conn(0x7f67240ad530 0x7f67240af950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:15.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.093+0000 7f673d600700 1 -- 172.21.15.62:0/2485215925 <== osd.5 v2:172.21.15.204:6808/730785574 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f672801ee50 con 0x7f67240ad530 2024-04-02T14:42:15.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.112+0000 7f6746c00700 1 -- 172.21.15.62:0/2485215925 --> [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6740103d70 con 0x7f67240ad530 2024-04-02T14:42:15.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.112+0000 7f673d600700 1 -- 172.21.15.62:0/2485215925 <== osd.5 v2:172.21.15.204:6808/730785574 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6740103d70 con 0x7f67240ad530 2024-04-02T14:42:15.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.113+0000 7f6746c00700 1 -- 172.21.15.62:0/2485215925 >> [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574] conn(0x7f67240ad530 msgr2=0x7f67240af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:15.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.113+0000 7f6746c00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574] conn(0x7f67240ad530 0x7f67240af950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:15.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.113+0000 7f6746c00700 1 -- 172.21.15.62:0/2485215925 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f67240a3d50 msgr2=0x7f67240a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:15.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.113+0000 7f6746c00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f67240a3d50 0x7f67240a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6730006010 tx=0x7f673000b260 comp rx=0 tx=0).stop 2024-04-02T14:42:15.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.113+0000 7f6746c00700 1 -- 172.21.15.62:0/2485215925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6740103000 msgr2=0x7f674019b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:15.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.113+0000 7f6746c00700 1 --2- 172.21.15.62:0/2485215925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6740103000 0x7f674019b670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f672800e9c0 tx=0x7f672800ecd0 comp rx=0 tx=0).stop 2024-04-02T14:42:15.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.113+0000 7f6744c00700 1 -- 172.21.15.62:0/2485215925 reap_dead start 2024-04-02T14:42:15.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.113+0000 7f6746c00700 1 -- 172.21.15.62:0/2485215925 shutdown_connections 2024-04-02T14:42:15.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.113+0000 7f6746c00700 1 -- 172.21.15.62:0/2485215925 >> 172.21.15.62:0/2485215925 conn(0x7f6740079750 msgr2=0x7f67401004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:15.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.114+0000 7f6746c00700 1 -- 172.21.15.62:0/2485215925 shutdown_connections 2024-04-02T14:42:15.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:15.114+0000 7f6746c00700 1 -- 172.21.15.62:0/2485215925 wait complete. 2024-04-02T14:42:15.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:15 smithi204 ceph-mon[35948]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:15.548 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:15 smithi062 ceph-mon[32116]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:15.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:15 smithi113 ceph-mon[36135]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:16.233 INFO:teuthology.orchestra.run.smithi062.stdout:150323855424 2024-04-02T14:42:16.233 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-04-02T14:42:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:17 smithi062 ceph-mon[32116]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:42:17.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:17 smithi204 ceph-mon[35948]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:42:17.542 INFO:teuthology.orchestra.run.smithi062.stdout:77309411403 2024-04-02T14:42:17.542 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-04-02T14:42:17.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:17 smithi113 ceph-mon[36135]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:42:17.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.043+0000 7fdafee00700 1 -- 172.21.15.62:0/489357268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf806ab30 msgr2=0x7fdaf806af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:17.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.043+0000 7fdafee00700 1 --2- 172.21.15.62:0/489357268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf806ab30 0x7fdaf806af10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009990 tx=0x7fdae8009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:17.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.044+0000 7fdafee00700 1 -- 172.21.15.62:0/489357268 shutdown_connections 2024-04-02T14:42:17.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.044+0000 7fdafee00700 1 --2- 172.21.15.62:0/489357268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf810f0d0 0x7fdaf81114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:17.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.044+0000 7fdafee00700 1 --2- 172.21.15.62:0/489357268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf806ab30 0x7fdaf806af10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:17.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.044+0000 7fdafee00700 1 --2- 172.21.15.62:0/489357268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaf806b450 0x7fdaf810eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:17.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.044+0000 7fdafee00700 1 -- 172.21.15.62:0/489357268 >> 172.21.15.62:0/489357268 conn(0x7fdaf8077540 msgr2=0x7fdaf8077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:17.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.044+0000 7fdafee00700 1 -- 172.21.15.62:0/489357268 shutdown_connections 2024-04-02T14:42:17.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.044+0000 7fdafee00700 1 -- 172.21.15.62:0/489357268 wait complete. 2024-04-02T14:42:17.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.045+0000 7fdafee00700 1 Processor -- start 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.045+0000 7fdafee00700 1 -- start start 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdafee00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaf806ab30 0x7fdaf819b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdafee00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf806b450 0x7fdaf819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdafee00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf810f0d0 0x7fdaf819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdafee00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8114bf0 con 0x7fdaf810f0d0 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdafee00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8114a70 con 0x7fdaf806ab30 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdafee00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8114d70 con 0x7fdaf806b450 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdafce00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf810f0d0 0x7fdaf819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdaf7400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf806b450 0x7fdaf819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdafce00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf810f0d0 0x7fdaf819f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60656/0 (socket says 172.21.15.62:60656) 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdafce00700 1 -- 172.21.15.62:0/3483195488 learned_addr learned my addr 172.21.15.62:0/3483195488 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdaf7400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf806b450 0x7fdaf819b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:44036/0 (socket says 172.21.15.62:44036) 2024-04-02T14:42:17.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.046+0000 7fdaf7e00700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaf806ab30 0x7fdaf819b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:17.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.047+0000 7fdaf7400700 1 -- 172.21.15.62:0/3483195488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaf806ab30 msgr2=0x7fdaf819b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:17.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.047+0000 7fdaf7400700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaf806ab30 0x7fdaf819b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:17.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.047+0000 7fdaf7400700 1 -- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf810f0d0 msgr2=0x7fdaf819f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:17.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.047+0000 7fdaf7400700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf810f0d0 0x7fdaf819f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:17.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.047+0000 7fdaf7400700 1 -- 172.21.15.62:0/3483195488 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf81a00a0 con 0x7fdaf806b450 2024-04-02T14:42:17.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.047+0000 7fdaf7400700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf806b450 0x7fdaf819b620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdae400d750 tx=0x7fdae400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:17.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.047+0000 7fdaf4c00700 1 -- 172.21.15.62:0/3483195488 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4014030 con 0x7fdaf806b450 2024-04-02T14:42:17.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.047+0000 7fdafee00700 1 -- 172.21.15.62:0/3483195488 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf81a0390 con 0x7fdaf806b450 2024-04-02T14:42:17.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.047+0000 7fdaf4c00700 1 -- 172.21.15.62:0/3483195488 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdae4004520 con 0x7fdaf806b450 2024-04-02T14:42:17.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.047+0000 7fdaf4c00700 1 -- 172.21.15.62:0/3483195488 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae40054a0 con 0x7fdaf806b450 2024-04-02T14:42:17.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.047+0000 7fdafee00700 1 -- 172.21.15.62:0/3483195488 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf81a7c10 con 0x7fdaf806b450 2024-04-02T14:42:17.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.048+0000 7fdafee00700 1 -- 172.21.15.62:0/3483195488 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdad8001030 con 0x7fdaf806b450 2024-04-02T14:42:17.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.051+0000 7fdaf4c00700 1 -- 172.21.15.62:0/3483195488 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdae4004020 con 0x7fdaf806b450 2024-04-02T14:42:17.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.052+0000 7fdaf4c00700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdadc0a3cd0 0x7fdadc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:17.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.052+0000 7fdaf4c00700 1 -- 172.21.15.62:0/3483195488 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fdae40cfff0 con 0x7fdaf806b450 2024-04-02T14:42:17.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.052+0000 7fdaf7e00700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdadc0a3cd0 0x7fdadc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:17.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.052+0000 7fdaf4c00700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320] conn(0x7fdadc0ad4b0 0x7fdadc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:17.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.052+0000 7fdaf4c00700 1 -- 172.21.15.62:0/3483195488 --> [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdae4021e50 con 0x7fdadc0ad4b0 2024-04-02T14:42:17.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.053+0000 7fdaf4c00700 1 -- 172.21.15.62:0/3483195488 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fdae40d03a0 con 0x7fdaf806b450 2024-04-02T14:42:17.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.053+0000 7fdafce00700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320] conn(0x7fdadc0ad4b0 0x7fdadc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:17.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.053+0000 7fdaf7e00700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdadc0a3cd0 0x7fdadc0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdae8006010 tx=0x7fdae800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:17.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.053+0000 7fdafce00700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320] conn(0x7fdadc0ad4b0 0x7fdadc0af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:17.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.054+0000 7fdaf4c00700 1 -- 172.21.15.62:0/3483195488 <== osd.9 v2:172.21.15.62:6826/465866320 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdae4021e50 con 0x7fdadc0ad4b0 2024-04-02T14:42:17.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.073+0000 7fdafee00700 1 -- 172.21.15.62:0/3483195488 --> [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdad8002e60 con 0x7fdadc0ad4b0 2024-04-02T14:42:17.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.074+0000 7fdaf4c00700 1 -- 172.21.15.62:0/3483195488 <== osd.9 v2:172.21.15.62:6826/465866320 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdad8002e60 con 0x7fdadc0ad4b0 2024-04-02T14:42:17.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.074+0000 7fdafee00700 1 -- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320] conn(0x7fdadc0ad4b0 msgr2=0x7fdadc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:17.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.074+0000 7fdafee00700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320] conn(0x7fdadc0ad4b0 0x7fdadc0af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:17.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.074+0000 7fdafee00700 1 -- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdadc0a3cd0 msgr2=0x7fdadc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:17.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.074+0000 7fdafee00700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdadc0a3cd0 0x7fdadc0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdae8006010 tx=0x7fdae800b540 comp rx=0 tx=0).stop 2024-04-02T14:42:17.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.074+0000 7fdafee00700 1 -- 172.21.15.62:0/3483195488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf806b450 msgr2=0x7fdaf819b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:17.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.074+0000 7fdafee00700 1 --2- 172.21.15.62:0/3483195488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf806b450 0x7fdaf819b620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdae400d750 tx=0x7fdae400da60 comp rx=0 tx=0).stop 2024-04-02T14:42:17.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.074+0000 7fdafce00700 1 -- 172.21.15.62:0/3483195488 reap_dead start 2024-04-02T14:42:17.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.074+0000 7fdafee00700 1 -- 172.21.15.62:0/3483195488 shutdown_connections 2024-04-02T14:42:17.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.074+0000 7fdafee00700 1 -- 172.21.15.62:0/3483195488 >> 172.21.15.62:0/3483195488 conn(0x7fdaf8077540 msgr2=0x7fdaf8110520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:17.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.075+0000 7fdafee00700 1 -- 172.21.15.62:0/3483195488 shutdown_connections 2024-04-02T14:42:17.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:17.075+0000 7fdafee00700 1 -- 172.21.15.62:0/3483195488 wait complete. 2024-04-02T14:42:18.978 INFO:teuthology.orchestra.run.smithi062.stdout:171798691898 2024-04-02T14:42:18.979 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-04-02T14:42:19.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:19 smithi204 ceph-mon[35948]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:42:19.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:19 smithi062 ceph-mon[32116]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:42:19.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:19 smithi113 ceph-mon[36135]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:42:19.802 INFO:teuthology.orchestra.run.smithi062.stdout:300647710755 2024-04-02T14:42:19.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.9 2024-04-02T14:42:19.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.190+0000 7f8a7b800700 1 -- 172.21.15.62:0/206601509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a740749c0 msgr2=0x7f8a74074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:19.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.190+0000 7f8a7b800700 1 --2- 172.21.15.62:0/206601509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a740749c0 0x7f8a74074da0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009990 tx=0x7f8a68009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:19.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.190+0000 7f8a7b800700 1 -- 172.21.15.62:0/206601509 shutdown_connections 2024-04-02T14:42:19.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.190+0000 7f8a7b800700 1 --2- 172.21.15.62:0/206601509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7410eeb0 0x7f8a741112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:19.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.190+0000 7f8a7b800700 1 --2- 172.21.15.62:0/206601509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a740749c0 0x7f8a74074da0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:19.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.190+0000 7f8a7b800700 1 --2- 172.21.15.62:0/206601509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a740752e0 0x7f8a7410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:19.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.190+0000 7f8a7b800700 1 -- 172.21.15.62:0/206601509 >> 172.21.15.62:0/206601509 conn(0x7f8a740fe0c0 msgr2=0x7f8a741004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.190+0000 7f8a7b800700 1 -- 172.21.15.62:0/206601509 shutdown_connections 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.190+0000 7f8a7b800700 1 -- 172.21.15.62:0/206601509 wait complete. 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.191+0000 7f8a7b800700 1 Processor -- start 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.191+0000 7f8a7b800700 1 -- start start 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.191+0000 7f8a7b800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a740749c0 0x7f8a7419b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.191+0000 7f8a7b800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a740752e0 0x7f8a7419b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a7b800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7410eeb0 0x7f8a7419f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a7b800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a74114cc0 con 0x7f8a740749c0 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a7b800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8a74114b40 con 0x7f8a7410eeb0 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a7b800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a74114e40 con 0x7f8a740752e0 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a78e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a740749c0 0x7f8a7419b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a78e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a740749c0 0x7f8a7419b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60674/0 (socket says 172.21.15.62:60674) 2024-04-02T14:42:19.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a78e00700 1 -- 172.21.15.62:0/419726857 learned_addr learned my addr 172.21.15.62:0/419726857 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:19.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a73e00700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a740752e0 0x7f8a7419b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:19.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a79800700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7410eeb0 0x7f8a7419f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:19.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a78e00700 1 -- 172.21.15.62:0/419726857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a740752e0 msgr2=0x7f8a7419b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:19.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a78e00700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a740752e0 0x7f8a7419b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:19.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a78e00700 1 -- 172.21.15.62:0/419726857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7410eeb0 msgr2=0x7f8a7419f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:19.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a78e00700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7410eeb0 0x7f8a7419f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:19.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.192+0000 7f8a78e00700 1 -- 172.21.15.62:0/419726857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a741a00a0 con 0x7f8a740749c0 2024-04-02T14:42:19.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.193+0000 7f8a78e00700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a740749c0 0x7f8a7419b090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a680095b0 tx=0x7f8a68005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:19.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.193+0000 7f8a71600700 1 -- 172.21.15.62:0/419726857 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68026070 con 0x7f8a740749c0 2024-04-02T14:42:19.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.193+0000 7f8a71600700 1 -- 172.21.15.62:0/419726857 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a6800fe30 con 0x7f8a740749c0 2024-04-02T14:42:19.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.193+0000 7f8a7b800700 1 -- 172.21.15.62:0/419726857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a741a0330 con 0x7f8a740749c0 2024-04-02T14:42:19.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.193+0000 7f8a71600700 1 -- 172.21.15.62:0/419726857 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a680173f0 con 0x7f8a740749c0 2024-04-02T14:42:19.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.193+0000 7f8a7b800700 1 -- 172.21.15.62:0/419726857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a741a7c10 con 0x7f8a740749c0 2024-04-02T14:42:19.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.194+0000 7f8a7b800700 1 -- 172.21.15.62:0/419726857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a74062d80 con 0x7f8a740749c0 2024-04-02T14:42:19.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.197+0000 7f8a71600700 1 -- 172.21.15.62:0/419726857 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8a68029070 con 0x7f8a740749c0 2024-04-02T14:42:19.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.198+0000 7f8a71600700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8a5c0a3cd0 0x7f8a5c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:19.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.198+0000 7f8a71600700 1 -- 172.21.15.62:0/419726857 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f8a680d1280 con 0x7f8a740749c0 2024-04-02T14:42:19.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.198+0000 7f8a73e00700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8a5c0a3cd0 0x7f8a5c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:19.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.198+0000 7f8a71600700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868] conn(0x7f8a5c0ad4b0 0x7f8a5c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:19.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.198+0000 7f8a71600700 1 -- 172.21.15.62:0/419726857 --> [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a5c0aff80 con 0x7f8a5c0ad4b0 2024-04-02T14:42:19.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.198+0000 7f8a71600700 1 -- 172.21.15.62:0/419726857 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f8a680d1760 con 0x7f8a740749c0 2024-04-02T14:42:19.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.199+0000 7f8a79800700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868] conn(0x7f8a5c0ad4b0 0x7f8a5c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:19.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.199+0000 7f8a73e00700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8a5c0a3cd0 0x7f8a5c0a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8a74111710 tx=0x7f8a60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:19.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.199+0000 7f8a79800700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868] conn(0x7f8a5c0ad4b0 0x7f8a5c0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:19.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.200+0000 7f8a71600700 1 -- 172.21.15.62:0/419726857 <== osd.3 v2:172.21.15.62:6810/263141868 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a5c0aff80 con 0x7f8a5c0ad4b0 2024-04-02T14:42:19.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.220+0000 7f8a7b800700 1 -- 172.21.15.62:0/419726857 --> [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a74063200 con 0x7f8a5c0ad4b0 2024-04-02T14:42:19.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.221+0000 7f8a71600700 1 -- 172.21.15.62:0/419726857 <== osd.3 v2:172.21.15.62:6810/263141868 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a74063200 con 0x7f8a5c0ad4b0 2024-04-02T14:42:19.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.221+0000 7f8a7b800700 1 -- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868] conn(0x7f8a5c0ad4b0 msgr2=0x7f8a5c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:19.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.221+0000 7f8a7b800700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868] conn(0x7f8a5c0ad4b0 0x7f8a5c0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:19.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.221+0000 7f8a7b800700 1 -- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8a5c0a3cd0 msgr2=0x7f8a5c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:19.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.221+0000 7f8a7b800700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8a5c0a3cd0 0x7f8a5c0a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8a74111710 tx=0x7f8a60009260 comp rx=0 tx=0).stop 2024-04-02T14:42:19.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.221+0000 7f8a7b800700 1 -- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a740749c0 msgr2=0x7f8a7419b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:19.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.221+0000 7f8a7b800700 1 --2- 172.21.15.62:0/419726857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a740749c0 0x7f8a7419b090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a680095b0 tx=0x7f8a68005590 comp rx=0 tx=0).stop 2024-04-02T14:42:19.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.221+0000 7f8a79800700 1 -- 172.21.15.62:0/419726857 reap_dead start 2024-04-02T14:42:19.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.222+0000 7f8a7b800700 1 -- 172.21.15.62:0/419726857 shutdown_connections 2024-04-02T14:42:19.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.222+0000 7f8a7b800700 1 -- 172.21.15.62:0/419726857 >> 172.21.15.62:0/419726857 conn(0x7f8a740fe0c0 msgr2=0x7f8a740fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:19.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.222+0000 7f8a7b800700 1 -- 172.21.15.62:0/419726857 shutdown_connections 2024-04-02T14:42:19.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.222+0000 7f8a7b800700 1 -- 172.21.15.62:0/419726857 wait complete. 2024-04-02T14:42:20.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.767+0000 7f39f9400700 1 -- 172.21.15.62:0/991334672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f4103fb0 msgr2=0x7f39f410c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.767+0000 7f39f9400700 1 --2- 172.21.15.62:0/991334672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f4103fb0 0x7f39f410c4a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f39e00098f0 tx=0x7f39e0009c00 comp rx=0 tx=0).stop 2024-04-02T14:42:20.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.768+0000 7f39f9400700 1 -- 172.21.15.62:0/991334672 shutdown_connections 2024-04-02T14:42:20.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.768+0000 7f39f9400700 1 --2- 172.21.15.62:0/991334672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f410cac0 0x7f39f410ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.768+0000 7f39f9400700 1 --2- 172.21.15.62:0/991334672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f4103690 0x7f39f4103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.768+0000 7f39f9400700 1 --2- 172.21.15.62:0/991334672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f4103fb0 0x7f39f410c4a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.768+0000 7f39f9400700 1 -- 172.21.15.62:0/991334672 >> 172.21.15.62:0/991334672 conn(0x7f39f40fcec0 msgr2=0x7f39f40ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:20.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.768+0000 7f39f9400700 1 -- 172.21.15.62:0/991334672 shutdown_connections 2024-04-02T14:42:20.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.768+0000 7f39f9400700 1 -- 172.21.15.62:0/991334672 wait complete. 2024-04-02T14:42:20.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.769+0000 7f39f9400700 1 Processor -- start 2024-04-02T14:42:20.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.770+0000 7f39f9400700 1 -- start start 2024-04-02T14:42:20.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.770+0000 7f39f9400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f4103690 0x7f39f41012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:20.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.770+0000 7f39f9400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f4103fb0 0x7f39f4101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:20.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.770+0000 7f39f9400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f410cac0 0x7f39f4101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:20.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.770+0000 7f39f9400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f39f4112990 con 0x7f39f4103690 2024-04-02T14:42:20.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.770+0000 7f39f9400700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f39f4112810 con 0x7f39f4103fb0 2024-04-02T14:42:20.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.770+0000 7f39f9400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39f4112b10 con 0x7f39f410cac0 2024-04-02T14:42:20.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.771+0000 7f39f3400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f4103fb0 0x7f39f4101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:20.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.771+0000 7f39f3400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f4103fb0 0x7f39f4101830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41118/0 (socket says 172.21.15.62:41118) 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.771+0000 7f39f3400700 1 -- 172.21.15.62:0/3186999446 learned_addr learned my addr 172.21.15.62:0/3186999446 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.771+0000 7f39f8a00700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f410cac0 0x7f39f4101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.771+0000 7f39f3e00700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f4103690 0x7f39f41012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.771+0000 7f39f3400700 1 -- 172.21.15.62:0/3186999446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f410cac0 msgr2=0x7f39f4101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.771+0000 7f39f3400700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f410cac0 0x7f39f4101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.771+0000 7f39f3400700 1 -- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f4103690 msgr2=0x7f39f41012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.771+0000 7f39f3400700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f4103690 0x7f39f41012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.771+0000 7f39f3400700 1 -- 172.21.15.62:0/3186999446 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f41a7380 con 0x7f39f4103fb0 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.771+0000 7f39f3400700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f4103fb0 0x7f39f4101830 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c010 tx=0x7f39e0010f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.772+0000 7f39f0c00700 1 -- 172.21.15.62:0/3186999446 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e000f560 con 0x7f39f4103fb0 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.772+0000 7f39f9400700 1 -- 172.21.15.62:0/3186999446 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f41a7610 con 0x7f39f4103fb0 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.772+0000 7f39f0c00700 1 -- 172.21.15.62:0/3186999446 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f39e000b560 con 0x7f39f4103fb0 2024-04-02T14:42:20.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.772+0000 7f39f0c00700 1 -- 172.21.15.62:0/3186999446 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0003a90 con 0x7f39f4103fb0 2024-04-02T14:42:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.772+0000 7f39f9400700 1 -- 172.21.15.62:0/3186999446 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f41a7b40 con 0x7f39f4103fb0 2024-04-02T14:42:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.773+0000 7f39f9400700 1 -- 172.21.15.62:0/3186999446 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39f4062d00 con 0x7f39f4103fb0 2024-04-02T14:42:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.775+0000 7f39f0c00700 1 -- 172.21.15.62:0/3186999446 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f39e0003c70 con 0x7f39f4103fb0 2024-04-02T14:42:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.777+0000 7f39f0c00700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f39d80a3cd0 0x7f39d80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.777+0000 7f39f0c00700 1 -- 172.21.15.62:0/3186999446 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f39e00d1840 con 0x7f39f4103fb0 2024-04-02T14:42:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.777+0000 7f39f3e00700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f39d80a3cd0 0x7f39d80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.777+0000 7f39f0c00700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892] conn(0x7f39d80ad4b0 0x7f39d80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.777+0000 7f39f0c00700 1 -- 172.21.15.62:0/3186999446 --> [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39d80aff80 con 0x7f39d80ad4b0 2024-04-02T14:42:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.777+0000 7f39f0c00700 1 -- 172.21.15.62:0/3186999446 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f39e00d1bf0 con 0x7f39f4103fb0 2024-04-02T14:42:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.778+0000 7f39f8a00700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892] conn(0x7f39d80ad4b0 0x7f39d80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.778+0000 7f39f3e00700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f39d80a3cd0 0x7f39d80a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009750 tx=0x7f39e4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:20.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.778+0000 7f39f8a00700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892] conn(0x7f39d80ad4b0 0x7f39d80af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:20.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.780+0000 7f39f0c00700 1 -- 172.21.15.62:0/3186999446 <== osd.6 v2:172.21.15.62:6818/2905631892 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f39d80aff80 con 0x7f39d80ad4b0 2024-04-02T14:42:20.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.811+0000 7f39f9400700 1 -- 172.21.15.62:0/3186999446 --> [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f39f4063100 con 0x7f39d80ad4b0 2024-04-02T14:42:20.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.812+0000 7f39f0c00700 1 -- 172.21.15.62:0/3186999446 <== osd.6 v2:172.21.15.62:6818/2905631892 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f39f4063100 con 0x7f39d80ad4b0 2024-04-02T14:42:20.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.813+0000 7f39f9400700 1 -- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892] conn(0x7f39d80ad4b0 msgr2=0x7f39d80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.813+0000 7f39f9400700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892] conn(0x7f39d80ad4b0 0x7f39d80af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.813+0000 7f39f9400700 1 -- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f39d80a3cd0 msgr2=0x7f39d80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.813+0000 7f39f9400700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f39d80a3cd0 0x7f39d80a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009750 tx=0x7f39e4009310 comp rx=0 tx=0).stop 2024-04-02T14:42:20.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.813+0000 7f39f9400700 1 -- 172.21.15.62:0/3186999446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f4103fb0 msgr2=0x7f39f4101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.813+0000 7f39f9400700 1 --2- 172.21.15.62:0/3186999446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f4103fb0 0x7f39f4101830 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c010 tx=0x7f39e0010f70 comp rx=0 tx=0).stop 2024-04-02T14:42:20.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.813+0000 7f39f8a00700 1 -- 172.21.15.62:0/3186999446 reap_dead start 2024-04-02T14:42:20.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.813+0000 7f39f9400700 1 -- 172.21.15.62:0/3186999446 shutdown_connections 2024-04-02T14:42:20.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.813+0000 7f39f9400700 1 -- 172.21.15.62:0/3186999446 >> 172.21.15.62:0/3186999446 conn(0x7f39f40fcec0 msgr2=0x7f39f40fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:20.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.813+0000 7f39f9400700 1 -- 172.21.15.62:0/3186999446 shutdown_connections 2024-04-02T14:42:20.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:19.813+0000 7f39f9400700 1 -- 172.21.15.62:0/3186999446 wait complete. 2024-04-02T14:42:20.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:20 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-02T14:42:20.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:20 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-02T14:42:20.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:20 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-02T14:42:20.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:20 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-02T14:42:20.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:20 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-02T14:42:20.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:20 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-02T14:42:20.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:20 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-02T14:42:20.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:20 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-02T14:42:20.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:20 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-02T14:42:20.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:20 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-02T14:42:20.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:20 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-02T14:42:20.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:20 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-02T14:42:20.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:20 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-02T14:42:20.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:20 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-02T14:42:20.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:20 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-02T14:42:20.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:20 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-02T14:42:20.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:20 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-02T14:42:20.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:20 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-02T14:42:20.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:20 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-02T14:42:20.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:20 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-02T14:42:20.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:20 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-02T14:42:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.761+0000 7fb20b200700 1 -- 172.21.15.62:0/607195818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb204075140 msgr2=0x7fb20410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.761+0000 7fb20b200700 1 --2- 172.21.15.62:0/607195818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb204075140 0x7fb20410eab0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009990 tx=0x7fb1f0009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.762+0000 7fb20b200700 1 -- 172.21.15.62:0/607195818 shutdown_connections 2024-04-02T14:42:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.762+0000 7fb20b200700 1 --2- 172.21.15.62:0/607195818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb204074820 0x7fb204074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.762+0000 7fb20b200700 1 --2- 172.21.15.62:0/607195818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb20410eff0 0x7fb2041113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.762+0000 7fb20b200700 1 --2- 172.21.15.62:0/607195818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb204075140 0x7fb20410eab0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.762+0000 7fb20b200700 1 -- 172.21.15.62:0/607195818 >> 172.21.15.62:0/607195818 conn(0x7fb204079750 msgr2=0x7fb204079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.762+0000 7fb20b200700 1 -- 172.21.15.62:0/607195818 shutdown_connections 2024-04-02T14:42:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.762+0000 7fb20b200700 1 -- 172.21.15.62:0/607195818 wait complete. 2024-04-02T14:42:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.763+0000 7fb20b200700 1 Processor -- start 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.763+0000 7fb20b200700 1 -- start start 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.763+0000 7fb20b200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb204074820 0x7fb20419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.763+0000 7fb20b200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb204075140 0x7fb20419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.763+0000 7fb20b200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb20410eff0 0x7fb20419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb20b200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb204114d40 con 0x7fb204074820 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb20b200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb204114bc0 con 0x7fb204075140 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb20b200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb204114ec0 con 0x7fb20410eff0 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb203e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb204074820 0x7fb20419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb203e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb204074820 0x7fb20419b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60722/0 (socket says 172.21.15.62:60722) 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb203e00700 1 -- 172.21.15.62:0/299173549 learned_addr learned my addr 172.21.15.62:0/299173549 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb203400700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb204075140 0x7fb20419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb209200700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb20410eff0 0x7fb20419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb203e00700 1 -- 172.21.15.62:0/299173549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb20410eff0 msgr2=0x7fb20419f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb203e00700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb20410eff0 0x7fb20419f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb203e00700 1 -- 172.21.15.62:0/299173549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb204075140 msgr2=0x7fb20419b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb203e00700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb204075140 0x7fb20419b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb203e00700 1 -- 172.21.15.62:0/299173549 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2041a0110 con 0x7fb204074820 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.764+0000 7fb203400700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb204075140 0x7fb20419b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.765+0000 7fb203e00700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb204074820 0x7fb20419b150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400e9a0 tx=0x7fb1f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.765+0000 7fb200c00700 1 -- 172.21.15.62:0/299173549 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f400ca10 con 0x7fb204074820 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.765+0000 7fb20b200700 1 -- 172.21.15.62:0/299173549 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2041a0400 con 0x7fb204074820 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.765+0000 7fb200c00700 1 -- 172.21.15.62:0/299173549 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb1f400cbb0 con 0x7fb204074820 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.765+0000 7fb200c00700 1 -- 172.21.15.62:0/299173549 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4018710 con 0x7fb204074820 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.765+0000 7fb20b200700 1 -- 172.21.15.62:0/299173549 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2041a7c80 con 0x7fb204074820 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.766+0000 7fb20b200700 1 -- 172.21.15.62:0/299173549 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb204100100 con 0x7fb204074820 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.769+0000 7fb200c00700 1 -- 172.21.15.62:0/299173549 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb1f4010430 con 0x7fb204074820 2024-04-02T14:42:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.770+0000 7fb200c00700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb1e80a3cd0 0x7fb1e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:20.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.770+0000 7fb200c00700 1 -- 172.21.15.62:0/299173549 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fb1f40d0f00 con 0x7fb204074820 2024-04-02T14:42:20.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.770+0000 7fb203400700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb1e80a3cd0 0x7fb1e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:20.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.771+0000 7fb200c00700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295] conn(0x7fb1e80ad790 0x7fb1e80afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:20.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.771+0000 7fb200c00700 1 -- 172.21.15.62:0/299173549 --> [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1f4027e50 con 0x7fb1e80ad790 2024-04-02T14:42:20.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.771+0000 7fb200c00700 1 -- 172.21.15.62:0/299173549 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb1f40d1350 con 0x7fb204074820 2024-04-02T14:42:20.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.771+0000 7fb203400700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb1e80a3cd0 0x7fb1e80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb204102090 tx=0x7fb1f000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:20.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.771+0000 7fb209200700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295] conn(0x7fb1e80ad790 0x7fb1e80afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:20.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.772+0000 7fb209200700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295] conn(0x7fb1e80ad790 0x7fb1e80afc30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:20.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.773+0000 7fb200c00700 1 -- 172.21.15.62:0/299173549 <== osd.1 v2:172.21.15.113:6800/124870295 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1f4027e50 con 0x7fb1e80ad790 2024-04-02T14:42:20.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.797+0000 7fb20b200700 1 -- 172.21.15.62:0/299173549 --> [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb204075f50 con 0x7fb1e80ad790 2024-04-02T14:42:20.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.798+0000 7fb200c00700 1 -- 172.21.15.62:0/299173549 <== osd.1 v2:172.21.15.113:6800/124870295 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb204075f50 con 0x7fb1e80ad790 2024-04-02T14:42:20.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.798+0000 7fb20b200700 1 -- 172.21.15.62:0/299173549 >> [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295] conn(0x7fb1e80ad790 msgr2=0x7fb1e80afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.798+0000 7fb20b200700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295] conn(0x7fb1e80ad790 0x7fb1e80afc30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.798+0000 7fb20b200700 1 -- 172.21.15.62:0/299173549 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb1e80a3cd0 msgr2=0x7fb1e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.798+0000 7fb20b200700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb1e80a3cd0 0x7fb1e80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb204102090 tx=0x7fb1f000b260 comp rx=0 tx=0).stop 2024-04-02T14:42:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.798+0000 7fb20b200700 1 -- 172.21.15.62:0/299173549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb204074820 msgr2=0x7fb20419b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.798+0000 7fb20b200700 1 --2- 172.21.15.62:0/299173549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb204074820 0x7fb20419b150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400e9a0 tx=0x7fb1f400ecb0 comp rx=0 tx=0).stop 2024-04-02T14:42:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.798+0000 7fb209200700 1 -- 172.21.15.62:0/299173549 reap_dead start 2024-04-02T14:42:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.798+0000 7fb20b200700 1 -- 172.21.15.62:0/299173549 shutdown_connections 2024-04-02T14:42:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.798+0000 7fb20b200700 1 -- 172.21.15.62:0/299173549 >> 172.21.15.62:0/299173549 conn(0x7fb204079750 msgr2=0x7fb2040ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.799+0000 7fb20b200700 1 -- 172.21.15.62:0/299173549 shutdown_connections 2024-04-02T14:42:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:20.799+0000 7fb20b200700 1 -- 172.21.15.62:0/299173549 wait complete. 2024-04-02T14:42:21.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:21 smithi204 ceph-mon[35948]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-02T14:42:21.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-02T14:42:21.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-02T14:42:21.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-02T14:42:21.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-02T14:42:21.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-02T14:42:21.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-02T14:42:21.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:21 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-02T14:42:21.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:21 smithi204 ceph-mon[35948]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T14:42:21.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:21 smithi062 ceph-mon[32116]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-02T14:42:21.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-02T14:42:21.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-02T14:42:21.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-02T14:42:21.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-02T14:42:21.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-02T14:42:21.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-02T14:42:21.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:21 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-02T14:42:21.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:21 smithi062 ceph-mon[32116]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T14:42:21.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:21 smithi113 ceph-mon[36135]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-02T14:42:21.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-02T14:42:21.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-02T14:42:21.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-02T14:42:21.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-02T14:42:21.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-02T14:42:21.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-02T14:42:21.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:21 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-02T14:42:21.619 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:21 smithi113 ceph-mon[36135]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T14:42:22.238 INFO:teuthology.orchestra.run.smithi062.stdout:206158430261 2024-04-02T14:42:22.239 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.6 2024-04-02T14:42:22.368 INFO:teuthology.orchestra.run.smithi062.stdout:55834574929 2024-04-02T14:42:22.368 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-04-02T14:42:22.436 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:22 smithi062 ceph-mon[32116]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T14:42:22.436 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:22 smithi062 ceph-mon[32116]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:42:22.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:22 smithi204 ceph-mon[35948]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T14:42:22.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:22 smithi204 ceph-mon[35948]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:42:22.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:22 smithi113 ceph-mon[36135]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T14:42:22.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:22 smithi113 ceph-mon[36135]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:42:22.740 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:22.743 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:22.765 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:22.791 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:22.860 INFO:teuthology.orchestra.run.smithi062.stdout:115964117061 2024-04-02T14:42:22.861 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-04-02T14:42:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:23.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:23 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:42:23.522 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:23 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:42:23.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:23 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:42:25.020 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:25.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:25 smithi062 ceph-mon[32116]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-02T14:42:25.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:25 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:25.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:25 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:25.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:25 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:25.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:25 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:25.257 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:25 smithi204 ceph-mon[35948]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-02T14:42:25.257 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:25 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:25.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:25 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:25.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:25 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:25.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:25 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:25.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:25 smithi113 ceph-mon[36135]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-02T14:42:25.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:25 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:25.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:25 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:25.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:25 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:25.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:25 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:26.992 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:27.257 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:27 smithi204 ceph-mon[35948]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 126 B/s, 0 objects/s recovering 2024-04-02T14:42:27.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:27 smithi062 ceph-mon[32116]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 126 B/s, 0 objects/s recovering 2024-04-02T14:42:27.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:27 smithi113 ceph-mon[36135]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 126 B/s, 0 objects/s recovering 2024-04-02T14:42:29.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:29 smithi062 ceph-mon[32116]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 126 B/s, 0 objects/s recovering 2024-04-02T14:42:29.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:29 smithi113 ceph-mon[36135]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 126 B/s, 0 objects/s recovering 2024-04-02T14:42:29.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:29 smithi204 ceph-mon[35948]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 126 B/s, 0 objects/s recovering 2024-04-02T14:42:29.857 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:31.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:31 smithi062 ceph-mon[32116]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s; 110 B/s, 0 objects/s recovering 2024-04-02T14:42:31.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:31 smithi113 ceph-mon[36135]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s; 110 B/s, 0 objects/s recovering 2024-04-02T14:42:31.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:31 smithi204 ceph-mon[35948]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s; 110 B/s, 0 objects/s recovering 2024-04-02T14:42:32.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.844+0000 7f4ed1000700 1 -- 172.21.15.62:0/2763714497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ecc1047f0 msgr2=0x7f4ecc104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:32.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.844+0000 7f4ed1000700 1 --2- 172.21.15.62:0/2763714497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ecc1047f0 0x7f4ecc104c70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009990 tx=0x7f4eb8009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:32.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.844+0000 7f4ed1000700 1 -- 172.21.15.62:0/2763714497 shutdown_connections 2024-04-02T14:42:32.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.844+0000 7f4ed1000700 1 --2- 172.21.15.62:0/2763714497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ecc10a7f0 0x7f4ecc10abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.844+0000 7f4ed1000700 1 --2- 172.21.15.62:0/2763714497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ecc1051b0 0x7f4ecc1116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.844+0000 7f4ed1000700 1 --2- 172.21.15.62:0/2763714497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ecc1047f0 0x7f4ecc104c70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.844+0000 7f4ed1000700 1 -- 172.21.15.62:0/2763714497 >> 172.21.15.62:0/2763714497 conn(0x7f4ecc1002c0 msgr2=0x7f4ecc1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.844+0000 7f4ed1000700 1 -- 172.21.15.62:0/2763714497 shutdown_connections 2024-04-02T14:42:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.844+0000 7f4ed1000700 1 -- 172.21.15.62:0/2763714497 wait complete. 2024-04-02T14:42:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.845+0000 7f4ed1000700 1 Processor -- start 2024-04-02T14:42:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.845+0000 7f4ed1000700 1 -- start start 2024-04-02T14:42:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4ed1000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ecc1047f0 0x7f4ecc19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4ed1000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ecc1051b0 0x7f4ecc19b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4ed1000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ecc10a7f0 0x7f4ecc19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4ed1000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc114c90 con 0x7f4ecc1051b0 2024-04-02T14:42:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4ed1000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc114b10 con 0x7f4ecc1047f0 2024-04-02T14:42:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4ed1000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc114e10 con 0x7f4ecc10a7f0 2024-04-02T14:42:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4ec9400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ecc1051b0 0x7f4ecc19b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4eca800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ecc10a7f0 0x7f4ecc19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4ec9e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ecc1047f0 0x7f4ecc19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4eca800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ecc10a7f0 0x7f4ecc19f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:59552/0 (socket says 172.21.15.62:59552) 2024-04-02T14:42:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4ec9400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ecc1051b0 0x7f4ecc19b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38754/0 (socket says 172.21.15.62:38754) 2024-04-02T14:42:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.846+0000 7f4eca800700 1 -- 172.21.15.62:0/1226523010 learned_addr learned my addr 172.21.15.62:0/1226523010 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.847+0000 7f4ec9400700 1 -- 172.21.15.62:0/1226523010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ecc10a7f0 msgr2=0x7f4ecc19f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.847+0000 7f4ec9400700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ecc10a7f0 0x7f4ecc19f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.847+0000 7f4ec9400700 1 -- 172.21.15.62:0/1226523010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ecc1047f0 msgr2=0x7f4ecc19b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.847+0000 7f4ec9400700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ecc1047f0 0x7f4ecc19b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.847+0000 7f4ec9400700 1 -- 172.21.15.62:0/1226523010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc1a0060 con 0x7f4ecc1051b0 2024-04-02T14:42:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.847+0000 7f4eca800700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ecc10a7f0 0x7f4ecc19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.847+0000 7f4ec9400700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ecc1051b0 0x7f4ecc19b5e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800b2e0 tx=0x7f4eb80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.847+0000 7f4ec2a00700 1 -- 172.21.15.62:0/1226523010 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8026070 con 0x7f4ecc1051b0 2024-04-02T14:42:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.848+0000 7f4ec2a00700 1 -- 172.21.15.62:0/1226523010 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4eb800fe00 con 0x7f4ecc1051b0 2024-04-02T14:42:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.848+0000 7f4ed1000700 1 -- 172.21.15.62:0/1226523010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc1a02f0 con 0x7f4ecc1051b0 2024-04-02T14:42:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.848+0000 7f4ec2a00700 1 -- 172.21.15.62:0/1226523010 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80173f0 con 0x7f4ecc1051b0 2024-04-02T14:42:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.848+0000 7f4ed1000700 1 -- 172.21.15.62:0/1226523010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc1a7bd0 con 0x7f4ecc1051b0 2024-04-02T14:42:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.849+0000 7f4ed1000700 1 -- 172.21.15.62:0/1226523010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc0692b0 con 0x7f4ecc1051b0 2024-04-02T14:42:32.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.851+0000 7f4ec2a00700 1 -- 172.21.15.62:0/1226523010 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4eb8029070 con 0x7f4ecc1051b0 2024-04-02T14:42:32.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.852+0000 7f4ec2a00700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4eb00a3cd0 0x7f4eb00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:32.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.853+0000 7f4ec2a00700 1 -- 172.21.15.62:0/1226523010 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f4eb80d1220 con 0x7f4ecc1051b0 2024-04-02T14:42:32.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.853+0000 7f4ec9e00700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4eb00a3cd0 0x7f4eb00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:32.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.853+0000 7f4ec9e00700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4eb00a3cd0 0x7f4eb00a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009be0 tx=0x7f4ebc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:32.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:32.857+0000 7f4ec2a00700 1 -- 172.21.15.62:0/1226523010 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4eb805df20 con 0x7f4ecc1051b0 2024-04-02T14:42:33.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.092+0000 7f4ed1000700 1 -- 172.21.15.62:0/1226523010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f4ecc02dd30 con 0x7f4ecc1051b0 2024-04-02T14:42:33.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:33 smithi062 ceph-mon[32116]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 99 B/s, 0 objects/s recovering 2024-04-02T14:42:33.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.093+0000 7f4ec2a00700 1 -- 172.21.15.62:0/1226523010 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4eb8013070 con 0x7f4ecc1051b0 2024-04-02T14:42:33.091 INFO:teuthology.orchestra.run.smithi062.stdout:322122547232 2024-04-02T14:42:33.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.096+0000 7f4ed1000700 1 -- 172.21.15.62:0/1226523010 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4eb00a3cd0 msgr2=0x7f4eb00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:33.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.096+0000 7f4ed1000700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4eb00a3cd0 0x7f4eb00a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009be0 tx=0x7f4ebc009260 comp rx=0 tx=0).stop 2024-04-02T14:42:33.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.096+0000 7f4ed1000700 1 -- 172.21.15.62:0/1226523010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ecc1051b0 msgr2=0x7f4ecc19b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:33.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.096+0000 7f4ed1000700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ecc1051b0 0x7f4ecc19b5e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800b2e0 tx=0x7f4eb80047b0 comp rx=0 tx=0).stop 2024-04-02T14:42:33.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.097+0000 7f4ed1000700 1 -- 172.21.15.62:0/1226523010 shutdown_connections 2024-04-02T14:42:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.097+0000 7f4ed1000700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4eb00a3cd0 0x7f4eb00a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.097+0000 7f4ed1000700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ecc1047f0 0x7f4ecc19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.097+0000 7f4ed1000700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ecc10a7f0 0x7f4ecc19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.097+0000 7f4ed1000700 1 --2- 172.21.15.62:0/1226523010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ecc1051b0 0x7f4ecc19b5e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.097+0000 7f4ed1000700 1 -- 172.21.15.62:0/1226523010 >> 172.21.15.62:0/1226523010 conn(0x7f4ecc1002c0 msgr2=0x7f4ecc1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.098+0000 7f4ed1000700 1 -- 172.21.15.62:0/1226523010 shutdown_connections 2024-04-02T14:42:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:33.098+0000 7f4ed1000700 1 -- 172.21.15.62:0/1226523010 wait complete. 2024-04-02T14:42:33.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:33 smithi204 ceph-mon[35948]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 99 B/s, 0 objects/s recovering 2024-04-02T14:42:33.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:33 smithi113 ceph-mon[36135]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 99 B/s, 0 objects/s recovering 2024-04-02T14:42:34.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:34 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1226523010' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T14:42:34.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:34 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1226523010' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T14:42:34.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:34 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1226523010' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T14:42:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.784+0000 7fbc09800700 1 -- 172.21.15.62:0/1098913732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc04109810 msgr2=0x7fbc0410d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.784+0000 7fbc09800700 1 --2- 172.21.15.62:0/1098913732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc04109810 0x7fbc0410d860 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00099c0 tx=0x7fbbf0009cd0 comp rx=0 tx=0).stop 2024-04-02T14:42:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.784+0000 7fbc09800700 1 -- 172.21.15.62:0/1098913732 shutdown_connections 2024-04-02T14:42:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.784+0000 7fbc09800700 1 --2- 172.21.15.62:0/1098913732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc0410df40 0x7fbc04115b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.784+0000 7fbc09800700 1 --2- 172.21.15.62:0/1098913732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc04108e60 0x7fbc04109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.784+0000 7fbc09800700 1 --2- 172.21.15.62:0/1098913732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc04109810 0x7fbc0410d860 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.784+0000 7fbc09800700 1 -- 172.21.15.62:0/1098913732 >> 172.21.15.62:0/1098913732 conn(0x7fbc04075dc0 msgr2=0x7fbc040781e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.785+0000 7fbc09800700 1 -- 172.21.15.62:0/1098913732 shutdown_connections 2024-04-02T14:42:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.785+0000 7fbc09800700 1 -- 172.21.15.62:0/1098913732 wait complete. 2024-04-02T14:42:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.786+0000 7fbc09800700 1 Processor -- start 2024-04-02T14:42:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.786+0000 7fbc09800700 1 -- start start 2024-04-02T14:42:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.786+0000 7fbc09800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc04108e60 0x7fbc0419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.786+0000 7fbc09800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc04109810 0x7fbc0419fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.786+0000 7fbc09800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0410df40 0x7fbc041a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.786+0000 7fbc09800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc04119060 con 0x7fbc04109810 2024-04-02T14:42:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.786+0000 7fbc09800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbc04118ee0 con 0x7fbc04108e60 2024-04-02T14:42:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.786+0000 7fbc09800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbc041191e0 con 0x7fbc0410df40 2024-04-02T14:42:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.786+0000 7fbc02000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc04109810 0x7fbc0419fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.787+0000 7fbc02a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc04108e60 0x7fbc0419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.787+0000 7fbc02000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc04109810 0x7fbc0419fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38770/0 (socket says 172.21.15.62:38770) 2024-04-02T14:42:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.787+0000 7fbc02000700 1 -- 172.21.15.62:0/2333031446 learned_addr learned my addr 172.21.15.62:0/2333031446 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.787+0000 7fbc03400700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0410df40 0x7fbc041a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.787+0000 7fbc02000700 1 -- 172.21.15.62:0/2333031446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0410df40 msgr2=0x7fbc041a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.787+0000 7fbc02000700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0410df40 0x7fbc041a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.787+0000 7fbc02000700 1 -- 172.21.15.62:0/2333031446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc04108e60 msgr2=0x7fbc0419f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.787+0000 7fbc02000700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc04108e60 0x7fbc0419f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.787+0000 7fbc02000700 1 -- 172.21.15.62:0/2333031446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc041a4510 con 0x7fbc04109810 2024-04-02T14:42:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.787+0000 7fbc02a00700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc04108e60 0x7fbc0419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.787+0000 7fbc02000700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc04109810 0x7fbc0419fa90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000bc60 tx=0x7fbbf0003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.788+0000 7fbbfb400700 1 -- 172.21.15.62:0/2333031446 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0009e60 con 0x7fbc04109810 2024-04-02T14:42:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.788+0000 7fbbfb400700 1 -- 172.21.15.62:0/2333031446 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbbf000be40 con 0x7fbc04109810 2024-04-02T14:42:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.788+0000 7fbc09800700 1 -- 172.21.15.62:0/2333031446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc041a47a0 con 0x7fbc04109810 2024-04-02T14:42:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.788+0000 7fbbfb400700 1 -- 172.21.15.62:0/2333031446 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000fd40 con 0x7fbc04109810 2024-04-02T14:42:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.788+0000 7fbc09800700 1 -- 172.21.15.62:0/2333031446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc041ac080 con 0x7fbc04109810 2024-04-02T14:42:34.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.789+0000 7fbc09800700 1 -- 172.21.15.62:0/2333031446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc040692b0 con 0x7fbc04109810 2024-04-02T14:42:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.791+0000 7fbbfb400700 1 -- 172.21.15.62:0/2333031446 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbbf000f460 con 0x7fbc04109810 2024-04-02T14:42:34.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.792+0000 7fbbfb400700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbbe80a3cd0 0x7fbbe80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:34.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.793+0000 7fbbfb400700 1 -- 172.21.15.62:0/2333031446 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fbbf00d1270 con 0x7fbc04109810 2024-04-02T14:42:34.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.793+0000 7fbc02a00700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbbe80a3cd0 0x7fbbe80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:34.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.794+0000 7fbc02a00700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbbe80a3cd0 0x7fbbe80a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4011490 tx=0x7fbbf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:34.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:34.797+0000 7fbbfb400700 1 -- 172.21.15.62:0/2333031446 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbbf0013070 con 0x7fbc04109810 2024-04-02T14:42:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.053+0000 7fbc09800700 1 -- 172.21.15.62:0/2333031446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fbc0402dd30 con 0x7fbc04109810 2024-04-02T14:42:35.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.054+0000 7fbbfb400700 1 -- 172.21.15.62:0/2333031446 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbbf009db50 con 0x7fbc04109810 2024-04-02T14:42:35.052 INFO:teuthology.orchestra.run.smithi062.stdout:266287972397 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.056+0000 7fbc09800700 1 -- 172.21.15.62:0/2333031446 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbbe80a3cd0 msgr2=0x7fbbe80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.057+0000 7fbc09800700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbbe80a3cd0 0x7fbbe80a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4011490 tx=0x7fbbf4009260 comp rx=0 tx=0).stop 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.057+0000 7fbc09800700 1 -- 172.21.15.62:0/2333031446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc04109810 msgr2=0x7fbc0419fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.057+0000 7fbc09800700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc04109810 0x7fbc0419fa90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000bc60 tx=0x7fbbf0003c30 comp rx=0 tx=0).stop 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.057+0000 7fbc09800700 1 -- 172.21.15.62:0/2333031446 shutdown_connections 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.057+0000 7fbc09800700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbbe80a3cd0 0x7fbbe80a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.057+0000 7fbc09800700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc04108e60 0x7fbc0419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.057+0000 7fbc09800700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0410df40 0x7fbc041a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.057+0000 7fbc09800700 1 --2- 172.21.15.62:0/2333031446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc04109810 0x7fbc0419fa90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.057+0000 7fbc09800700 1 -- 172.21.15.62:0/2333031446 >> 172.21.15.62:0/2333031446 conn(0x7fbc04075dc0 msgr2=0x7fbc04078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.057+0000 7fbc09800700 1 -- 172.21.15.62:0/2333031446 shutdown_connections 2024-04-02T14:42:35.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.057+0000 7fbc09800700 1 -- 172.21.15.62:0/2333031446 wait complete. 2024-04-02T14:42:35.319 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:35 smithi062 ceph-mon[32116]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s; 84 B/s, 0 objects/s recovering 2024-04-02T14:42:35.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:35 smithi113 ceph-mon[36135]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s; 84 B/s, 0 objects/s recovering 2024-04-02T14:42:35.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.378+0000 7f8a1d600700 1 -- 172.21.15.62:0/3783248846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a18106c80 msgr2=0x7f8a18111790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:35.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.378+0000 7f8a1d600700 1 --2- 172.21.15.62:0/3783248846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a18106c80 0x7f8a18111790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000b0a0 tx=0x7f8a1000b3b0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.378+0000 7f8a1d600700 1 -- 172.21.15.62:0/3783248846 shutdown_connections 2024-04-02T14:42:35.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.378+0000 7f8a1d600700 1 --2- 172.21.15.62:0/3783248846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a18106c80 0x7f8a18111790 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.378+0000 7f8a1d600700 1 --2- 172.21.15.62:0/3783248846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a18104b70 0x7f8a18104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.378+0000 7f8a1d600700 1 --2- 172.21.15.62:0/3783248846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181062c0 0x7f8a18106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.379+0000 7f8a1d600700 1 -- 172.21.15.62:0/3783248846 >> 172.21.15.62:0/3783248846 conn(0x7f8a18100290 msgr2=0x7f8a181026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:35.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.379+0000 7f8a1d600700 1 -- 172.21.15.62:0/3783248846 shutdown_connections 2024-04-02T14:42:35.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.379+0000 7f8a1d600700 1 -- 172.21.15.62:0/3783248846 wait complete. 2024-04-02T14:42:35.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.380+0000 7f8a1d600700 1 Processor -- start 2024-04-02T14:42:35.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.380+0000 7f8a1d600700 1 -- start start 2024-04-02T14:42:35.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.380+0000 7f8a1d600700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a18104b70 0x7f8a1819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:35.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.380+0000 7f8a1d600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181062c0 0x7f8a1819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:35.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.380+0000 7f8a1d600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a18106c80 0x7f8a1819fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:35.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.380+0000 7f8a1d600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18114d90 con 0x7f8a18106c80 2024-04-02T14:42:35.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.380+0000 7f8a1d600700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18114c10 con 0x7f8a18104b70 2024-04-02T14:42:35.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.380+0000 7f8a1d600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18114f10 con 0x7f8a181062c0 2024-04-02T14:42:35.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.380+0000 7f8a16a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a18104b70 0x7f8a1819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:35.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.381+0000 7f8a16a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a18104b70 0x7f8a1819b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:32838/0 (socket says 172.21.15.62:32838) 2024-04-02T14:42:35.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.381+0000 7f8a16a00700 1 -- 172.21.15.62:0/1534606859 learned_addr learned my addr 172.21.15.62:0/1534606859 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:35.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.381+0000 7f8a17400700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a18106c80 0x7f8a1819fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:35.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.381+0000 7f8a16000700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181062c0 0x7f8a1819b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:35.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.381+0000 7f8a16a00700 1 -- 172.21.15.62:0/1534606859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181062c0 msgr2=0x7f8a1819b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:35.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.381+0000 7f8a16a00700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181062c0 0x7f8a1819b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.381+0000 7f8a16a00700 1 -- 172.21.15.62:0/1534606859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a18106c80 msgr2=0x7f8a1819fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:35.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.381+0000 7f8a16a00700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a18106c80 0x7f8a1819fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.381+0000 7f8a16a00700 1 -- 172.21.15.62:0/1534606859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a181a0160 con 0x7f8a18104b70 2024-04-02T14:42:35.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.381+0000 7f8a16a00700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a18104b70 0x7f8a1819b1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800e9a0 tx=0x7f8a0800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:35.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.382+0000 7f8a0f400700 1 -- 172.21.15.62:0/1534606859 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800ca10 con 0x7f8a18104b70 2024-04-02T14:42:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.382+0000 7f8a0f400700 1 -- 172.21.15.62:0/1534606859 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a0800cbb0 con 0x7f8a18104b70 2024-04-02T14:42:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.382+0000 7f8a0f400700 1 -- 172.21.15.62:0/1534606859 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08018710 con 0x7f8a18104b70 2024-04-02T14:42:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.382+0000 7f8a1d600700 1 -- 172.21.15.62:0/1534606859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a181a0450 con 0x7f8a18104b70 2024-04-02T14:42:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.382+0000 7f8a1d600700 1 -- 172.21.15.62:0/1534606859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a181a7cd0 con 0x7f8a18104b70 2024-04-02T14:42:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.383+0000 7f8a1d600700 1 -- 172.21.15.62:0/1534606859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a180692b0 con 0x7f8a18104b70 2024-04-02T14:42:35.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.385+0000 7f8a0f400700 1 -- 172.21.15.62:0/1534606859 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8a08010430 con 0x7f8a18104b70 2024-04-02T14:42:35.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.387+0000 7f8a0f400700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f89fc0a3d50 0x7f89fc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:35.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.387+0000 7f8a0f400700 1 -- 172.21.15.62:0/1534606859 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f8a080d0b50 con 0x7f8a18104b70 2024-04-02T14:42:35.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.387+0000 7f8a16000700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f89fc0a3d50 0x7f89fc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:35.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.388+0000 7f8a16000700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f89fc0a3d50 0x7f89fc0a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a18102810 tx=0x7f8a0400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:35.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.389+0000 7f8a0f400700 1 -- 172.21.15.62:0/1534606859 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8a0809d3a0 con 0x7f8a18104b70 2024-04-02T14:42:35.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:35 smithi204 ceph-mon[35948]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s; 84 B/s, 0 objects/s recovering 2024-04-02T14:42:35.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.623+0000 7f8a1d600700 1 -- 172.21.15.62:0/1534606859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8a1802dd30 con 0x7f8a18104b70 2024-04-02T14:42:35.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.623+0000 7f8a0f400700 1 -- 172.21.15.62:0/1534606859 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a0809cca0 con 0x7f8a18104b70 2024-04-02T14:42:35.622 INFO:teuthology.orchestra.run.smithi062.stdout:34359738458 2024-04-02T14:42:35.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.626+0000 7f8a1d600700 1 -- 172.21.15.62:0/1534606859 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f89fc0a3d50 msgr2=0x7f89fc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:35.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.626+0000 7f8a1d600700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f89fc0a3d50 0x7f89fc0a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a18102810 tx=0x7f8a0400b3c0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.626+0000 7f8a1d600700 1 -- 172.21.15.62:0/1534606859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a18104b70 msgr2=0x7f8a1819b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:35.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.626+0000 7f8a1d600700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a18104b70 0x7f8a1819b1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800e9a0 tx=0x7f8a0800ecb0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.626+0000 7f8a1d600700 1 -- 172.21.15.62:0/1534606859 shutdown_connections 2024-04-02T14:42:35.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.626+0000 7f8a1d600700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f89fc0a3d50 0x7f89fc0a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.626+0000 7f8a1d600700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a18104b70 0x7f8a1819b1a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.626+0000 7f8a1d600700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a18106c80 0x7f8a1819fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.626+0000 7f8a1d600700 1 --2- 172.21.15.62:0/1534606859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181062c0 0x7f8a1819b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:35.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.626+0000 7f8a1d600700 1 -- 172.21.15.62:0/1534606859 >> 172.21.15.62:0/1534606859 conn(0x7f8a18100290 msgr2=0x7f8a1810a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:35.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.627+0000 7f8a1d600700 1 -- 172.21.15.62:0/1534606859 shutdown_connections 2024-04-02T14:42:35.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:35.627+0000 7f8a1d600700 1 -- 172.21.15.62:0/1534606859 wait complete. 2024-04-02T14:42:36.299 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:36 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2333031446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T14:42:36.299 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:36 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1534606859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T14:42:36.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:36 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2333031446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T14:42:36.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:36 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1534606859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T14:42:36.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:36 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2333031446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T14:42:36.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:36 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1534606859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T14:42:37.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:37 smithi062 ceph-mon[32116]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 84 B/s, 0 objects/s recovering 2024-04-02T14:42:37.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:37 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2242347560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T14:42:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.244+0000 7f39b9600700 1 -- 172.21.15.62:0/529604097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b410f020 msgr2=0x7f39b4111410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.244+0000 7f39b9600700 1 --2- 172.21.15.62:0/529604097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b410f020 0x7f39b4111410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009f90 tx=0x7f39ac00b330 comp rx=0 tx=0).stop 2024-04-02T14:42:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.244+0000 7f39b9600700 1 -- 172.21.15.62:0/529604097 shutdown_connections 2024-04-02T14:42:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.244+0000 7f39b9600700 1 --2- 172.21.15.62:0/529604097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b410f020 0x7f39b4111410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.244+0000 7f39b9600700 1 --2- 172.21.15.62:0/529604097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4074820 0x7f39b4074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.244+0000 7f39b9600700 1 --2- 172.21.15.62:0/529604097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39b4075140 0x7f39b410eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.244+0000 7f39b9600700 1 -- 172.21.15.62:0/529604097 >> 172.21.15.62:0/529604097 conn(0x7f39b4079740 msgr2=0x7f39b4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.244+0000 7f39b9600700 1 -- 172.21.15.62:0/529604097 shutdown_connections 2024-04-02T14:42:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.244+0000 7f39b9600700 1 -- 172.21.15.62:0/529604097 wait complete. 2024-04-02T14:42:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.246+0000 7f39b9600700 1 Processor -- start 2024-04-02T14:42:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.246+0000 7f39b9600700 1 -- start start 2024-04-02T14:42:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b9600700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b4074820 0x7f39b419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b9600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4075140 0x7f39b419b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b9600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39b410f020 0x7f39b419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b9600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f39b4114d40 con 0x7f39b4075140 2024-04-02T14:42:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b9600700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f39b4114bc0 con 0x7f39b4074820 2024-04-02T14:42:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b9600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39b4114ec0 con 0x7f39b410f020 2024-04-02T14:42:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b2000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4075140 0x7f39b419b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b2000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4075140 0x7f39b419b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38834/0 (socket says 172.21.15.62:38834) 2024-04-02T14:42:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b2000700 1 -- 172.21.15.62:0/3851386099 learned_addr learned my addr 172.21.15.62:0/3851386099 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b2a00700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b4074820 0x7f39b419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b3400700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39b410f020 0x7f39b419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b2000700 1 -- 172.21.15.62:0/3851386099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39b410f020 msgr2=0x7f39b419f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b2000700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39b410f020 0x7f39b419f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b2000700 1 -- 172.21.15.62:0/3851386099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b4074820 msgr2=0x7f39b419b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b2000700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b4074820 0x7f39b419b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.247+0000 7f39b2000700 1 -- 172.21.15.62:0/3851386099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b41a0070 con 0x7f39b4075140 2024-04-02T14:42:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.248+0000 7f39b2000700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4075140 0x7f39b419b5f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f39a000e9c0 tx=0x7f39a000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.248+0000 7f39ab400700 1 -- 172.21.15.62:0/3851386099 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a000cbd0 con 0x7f39b4075140 2024-04-02T14:42:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.248+0000 7f39b9600700 1 -- 172.21.15.62:0/3851386099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b41a0360 con 0x7f39b4075140 2024-04-02T14:42:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.248+0000 7f39ab400700 1 -- 172.21.15.62:0/3851386099 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f39a000ee50 con 0x7f39b4075140 2024-04-02T14:42:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.248+0000 7f39ab400700 1 -- 172.21.15.62:0/3851386099 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a00056d0 con 0x7f39b4075140 2024-04-02T14:42:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.248+0000 7f39b9600700 1 -- 172.21.15.62:0/3851386099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b41a7be0 con 0x7f39b4075140 2024-04-02T14:42:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.250+0000 7f39b9600700 1 -- 172.21.15.62:0/3851386099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b40692b0 con 0x7f39b4075140 2024-04-02T14:42:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.252+0000 7f39ab400700 1 -- 172.21.15.62:0/3851386099 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f39a0004750 con 0x7f39b4075140 2024-04-02T14:42:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.253+0000 7f39ab400700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f39980a3e70 0x7f39980a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.253+0000 7f39ab400700 1 -- 172.21.15.62:0/3851386099 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f39a00d07e0 con 0x7f39b4075140 2024-04-02T14:42:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.253+0000 7f39b2a00700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f39980a3e70 0x7f39980a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.254+0000 7f39b2a00700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f39980a3e70 0x7f39980a6330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f39a4005fd0 tx=0x7f39a4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.258+0000 7f39ab400700 1 -- 172.21.15.62:0/3851386099 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f39a009d090 con 0x7f39b4075140 2024-04-02T14:42:37.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:37 smithi113 ceph-mon[36135]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 84 B/s, 0 objects/s recovering 2024-04-02T14:42:37.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:37 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2242347560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T14:42:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.491+0000 7f39b9600700 1 -- 172.21.15.62:0/3851386099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f39b41a7e10 con 0x7f39b4075140 2024-04-02T14:42:37.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.492+0000 7f39ab400700 1 -- 172.21.15.62:0/3851386099 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f39a009c990 con 0x7f39b4075140 2024-04-02T14:42:37.490 INFO:teuthology.orchestra.run.smithi062.stdout:356482285596 2024-04-02T14:42:37.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.495+0000 7f39b9600700 1 -- 172.21.15.62:0/3851386099 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f39980a3e70 msgr2=0x7f39980a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:37.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.495+0000 7f39b9600700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f39980a3e70 0x7f39980a6330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f39a4005fd0 tx=0x7f39a4009310 comp rx=0 tx=0).stop 2024-04-02T14:42:37.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.495+0000 7f39b9600700 1 -- 172.21.15.62:0/3851386099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4075140 msgr2=0x7f39b419b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.495+0000 7f39b9600700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4075140 0x7f39b419b5f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f39a000e9c0 tx=0x7f39a000ecd0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.495+0000 7f39b9600700 1 -- 172.21.15.62:0/3851386099 shutdown_connections 2024-04-02T14:42:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.495+0000 7f39b9600700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f39980a3e70 0x7f39980a6330 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.495+0000 7f39b9600700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b4074820 0x7f39b419b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.495+0000 7f39b9600700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39b410f020 0x7f39b419f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.495+0000 7f39b9600700 1 --2- 172.21.15.62:0/3851386099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4075140 0x7f39b419b5f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.495+0000 7f39b9600700 1 -- 172.21.15.62:0/3851386099 >> 172.21.15.62:0/3851386099 conn(0x7f39b4079740 msgr2=0x7f39b40ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.496+0000 7f39b9600700 1 -- 172.21.15.62:0/3851386099 shutdown_connections 2024-04-02T14:42:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:37.496+0000 7f39b9600700 1 -- 172.21.15.62:0/3851386099 wait complete. 2024-04-02T14:42:37.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:37 smithi204 ceph-mon[35948]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 84 B/s, 0 objects/s recovering 2024-04-02T14:42:37.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:37 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2242347560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T14:42:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.679+0000 7fba25a00700 1 -- 172.21.15.62:0/3999666845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba200749c0 msgr2=0x7fba20074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.679+0000 7fba25a00700 1 --2- 172.21.15.62:0/3999666845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba200749c0 0x7fba20074da0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fba1000b0a0 tx=0x7fba1000b3b0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.679+0000 7fba25a00700 1 -- 172.21.15.62:0/3999666845 shutdown_connections 2024-04-02T14:42:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.679+0000 7fba25a00700 1 --2- 172.21.15.62:0/3999666845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba200749c0 0x7fba20074da0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.679+0000 7fba25a00700 1 --2- 172.21.15.62:0/3999666845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba2010eeb0 0x7fba201112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.679+0000 7fba25a00700 1 --2- 172.21.15.62:0/3999666845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba200752e0 0x7fba2010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.679+0000 7fba25a00700 1 -- 172.21.15.62:0/3999666845 >> 172.21.15.62:0/3999666845 conn(0x7fba200fe0c0 msgr2=0x7fba201004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.679+0000 7fba25a00700 1 -- 172.21.15.62:0/3999666845 shutdown_connections 2024-04-02T14:42:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.679+0000 7fba25a00700 1 -- 172.21.15.62:0/3999666845 wait complete. 2024-04-02T14:42:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.680+0000 7fba25a00700 1 Processor -- start 2024-04-02T14:42:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.680+0000 7fba25a00700 1 -- start start 2024-04-02T14:42:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba25a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba200749c0 0x7fba2019b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba25a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba200752e0 0x7fba2019b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba25a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba2010eeb0 0x7fba2019f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba25a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fba20114cd0 con 0x7fba200749c0 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba25a00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fba20114b50 con 0x7fba2010eeb0 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba25a00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fba20114e50 con 0x7fba200752e0 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba1ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba200749c0 0x7fba2019b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba1ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba200749c0 0x7fba2019b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38810/0 (socket says 172.21.15.62:38810) 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba1ea00700 1 -- 172.21.15.62:0/2242347560 learned_addr learned my addr 172.21.15.62:0/2242347560 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba1f400700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba2010eeb0 0x7fba2019f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba16000700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba200752e0 0x7fba2019b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba1ea00700 1 -- 172.21.15.62:0/2242347560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba200752e0 msgr2=0x7fba2019b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba1ea00700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba200752e0 0x7fba2019b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba1ea00700 1 -- 172.21.15.62:0/2242347560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba2010eeb0 msgr2=0x7fba2019f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba1ea00700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba2010eeb0 0x7fba2019f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba1ea00700 1 -- 172.21.15.62:0/2242347560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba201a0120 con 0x7fba200749c0 2024-04-02T14:42:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.681+0000 7fba16000700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba200752e0 0x7fba2019b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.682+0000 7fba1ea00700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba200749c0 0x7fba2019b160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fba10006070 tx=0x7fba10004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.682+0000 7fba17e00700 1 -- 172.21.15.62:0/2242347560 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10021070 con 0x7fba200749c0 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.682+0000 7fba25a00700 1 -- 172.21.15.62:0/2242347560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba201a03b0 con 0x7fba200749c0 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.682+0000 7fba17e00700 1 -- 172.21.15.62:0/2242347560 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fba10003c00 con 0x7fba200749c0 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.682+0000 7fba17e00700 1 -- 172.21.15.62:0/2242347560 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10004840 con 0x7fba200749c0 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.682+0000 7fba25a00700 1 -- 172.21.15.62:0/2242347560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba201a7c90 con 0x7fba200749c0 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.683+0000 7fba25a00700 1 -- 172.21.15.62:0/2242347560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba200692b0 con 0x7fba200749c0 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.688+0000 7fba17e00700 1 -- 172.21.15.62:0/2242347560 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fba10003db0 con 0x7fba200749c0 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.689+0000 7fba17e00700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fba040a3cd0 0x7fba040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.689+0000 7fba17e00700 1 -- 172.21.15.62:0/2242347560 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fba1000f020 con 0x7fba200749c0 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.690+0000 7fba16000700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fba040a3cd0 0x7fba040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.690+0000 7fba16000700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fba040a3cd0 0x7fba040a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fba20111760 tx=0x7fba08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.692+0000 7fba17e00700 1 -- 172.21.15.62:0/2242347560 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fba1002a020 con 0x7fba200749c0 2024-04-02T14:42:37.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.941+0000 7fba25a00700 1 -- 172.21.15.62:0/2242347560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fba2002dd30 con 0x7fba200749c0 2024-04-02T14:42:37.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.941+0000 7fba17e00700 1 -- 172.21.15.62:0/2242347560 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fba1002e3e0 con 0x7fba200749c0 2024-04-02T14:42:37.797 INFO:teuthology.orchestra.run.smithi062.stdout:300647710759 2024-04-02T14:42:37.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 -- 172.21.15.62:0/2242347560 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fba040a3cd0 msgr2=0x7fba040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:37.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fba040a3cd0 0x7fba040a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fba20111760 tx=0x7fba08009260 comp rx=0 tx=0).stop 2024-04-02T14:42:37.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 -- 172.21.15.62:0/2242347560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba200749c0 msgr2=0x7fba2019b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:37.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba200749c0 0x7fba2019b160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fba10006070 tx=0x7fba10004c60 comp rx=0 tx=0).stop 2024-04-02T14:42:37.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 -- 172.21.15.62:0/2242347560 shutdown_connections 2024-04-02T14:42:37.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fba040a3cd0 0x7fba040a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba2010eeb0 0x7fba2019f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba200749c0 0x7fba2019b160 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 --2- 172.21.15.62:0/2242347560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba200752e0 0x7fba2019b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:37.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 -- 172.21.15.62:0/2242347560 >> 172.21.15.62:0/2242347560 conn(0x7fba200fe0c0 msgr2=0x7fba200fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:37.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 -- 172.21.15.62:0/2242347560 shutdown_connections 2024-04-02T14:42:37.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:36.945+0000 7fba25a00700 1 -- 172.21.15.62:0/2242347560 wait complete. 2024-04-02T14:42:38.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:38 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3851386099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T14:42:38.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:38 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3851386099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T14:42:38.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:38 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3851386099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T14:42:38.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.549+0000 7f74c2200700 1 -- 172.21.15.62:0/1036817120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc1046e0 msgr2=0x7f74bc104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:38.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.549+0000 7f74c2200700 1 --2- 172.21.15.62:0/1036817120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc1046e0 0x7f74bc104b60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009990 tx=0x7f74a8009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:38.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.550+0000 7f74c2200700 1 -- 172.21.15.62:0/1036817120 shutdown_connections 2024-04-02T14:42:38.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.550+0000 7f74c2200700 1 --2- 172.21.15.62:0/1036817120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74bc1050a0 0x7f74bc1115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:38.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.550+0000 7f74c2200700 1 --2- 172.21.15.62:0/1036817120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc10a6e0 0x7f74bc10aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:38.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.550+0000 7f74c2200700 1 --2- 172.21.15.62:0/1036817120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc1046e0 0x7f74bc104b60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:38.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.550+0000 7f74c2200700 1 -- 172.21.15.62:0/1036817120 >> 172.21.15.62:0/1036817120 conn(0x7f74bc100210 msgr2=0x7f74bc102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:38.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.550+0000 7f74c2200700 1 -- 172.21.15.62:0/1036817120 shutdown_connections 2024-04-02T14:42:38.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.550+0000 7f74c2200700 1 -- 172.21.15.62:0/1036817120 wait complete. 2024-04-02T14:42:38.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.551+0000 7f74c2200700 1 Processor -- start 2024-04-02T14:42:38.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.551+0000 7f74c2200700 1 -- start start 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.551+0000 7f74c2200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc1046e0 0x7f74bc19b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.551+0000 7f74c2200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc1050a0 0x7f74bc19b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.551+0000 7f74c2200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74bc10a6e0 0x7f74bc19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.551+0000 7f74c2200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc114ba0 con 0x7f74bc1046e0 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.551+0000 7f74c2200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc114a20 con 0x7f74bc10a6e0 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.551+0000 7f74c2200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc114d20 con 0x7f74bc1050a0 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.552+0000 7f74c0e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc1046e0 0x7f74bc19b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.552+0000 7f74c0e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc1046e0 0x7f74bc19b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38842/0 (socket says 172.21.15.62:38842) 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.552+0000 7f74c0e00700 1 -- 172.21.15.62:0/1081721425 learned_addr learned my addr 172.21.15.62:0/1081721425 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.552+0000 7f74bbe00700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc1050a0 0x7f74bc19b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.552+0000 7f74c1800700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74bc10a6e0 0x7f74bc19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.552+0000 7f74c0e00700 1 -- 172.21.15.62:0/1081721425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc1050a0 msgr2=0x7f74bc19b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.552+0000 7f74c0e00700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc1050a0 0x7f74bc19b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:38.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.552+0000 7f74c0e00700 1 -- 172.21.15.62:0/1081721425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74bc10a6e0 msgr2=0x7f74bc19f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:38.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.552+0000 7f74c0e00700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74bc10a6e0 0x7f74bc19f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:38.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.552+0000 7f74c0e00700 1 -- 172.21.15.62:0/1081721425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc1a0080 con 0x7f74bc1046e0 2024-04-02T14:42:38.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.552+0000 7f74bbe00700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc1050a0 0x7f74bc19b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:38.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.553+0000 7f74c0e00700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc1046e0 0x7f74bc19b070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74b000e9c0 tx=0x7f74b000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:38.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.553+0000 7f74b9600700 1 -- 172.21.15.62:0/1081721425 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b000cbd0 con 0x7f74bc1046e0 2024-04-02T14:42:38.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.553+0000 7f74c2200700 1 -- 172.21.15.62:0/1081721425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc1a0370 con 0x7f74bc1046e0 2024-04-02T14:42:38.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.553+0000 7f74b9600700 1 -- 172.21.15.62:0/1081721425 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f74b000ee60 con 0x7f74bc1046e0 2024-04-02T14:42:38.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.553+0000 7f74b9600700 1 -- 172.21.15.62:0/1081721425 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0010430 con 0x7f74bc1046e0 2024-04-02T14:42:38.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.553+0000 7f74c2200700 1 -- 172.21.15.62:0/1081721425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc1a7bf0 con 0x7f74bc1046e0 2024-04-02T14:42:38.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.554+0000 7f74c2200700 1 -- 172.21.15.62:0/1081721425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749c005330 con 0x7f74bc1046e0 2024-04-02T14:42:38.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.556+0000 7f74b9600700 1 -- 172.21.15.62:0/1081721425 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f74b00105d0 con 0x7f74bc1046e0 2024-04-02T14:42:38.911 INFO:teuthology.orchestra.run.smithi062.stdout:227633266739 2024-04-02T14:42:38.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.558+0000 7f74b9600700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f74a40a3cd0 0x7f74a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:38.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.558+0000 7f74b9600700 1 -- 172.21.15.62:0/1081721425 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f74b00d0900 con 0x7f74bc1046e0 2024-04-02T14:42:38.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.558+0000 7f74bbe00700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f74a40a3cd0 0x7f74a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:38.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.559+0000 7f74bbe00700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f74a40a3cd0 0x7f74a40a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f74bc109c30 tx=0x7f74a801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:38.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.560+0000 7f74b9600700 1 -- 172.21.15.62:0/1081721425 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f74b0053f20 con 0x7f74bc1046e0 2024-04-02T14:42:38.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.801+0000 7f74c2200700 1 -- 172.21.15.62:0/1081721425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f749c0058f0 con 0x7f74bc1046e0 2024-04-02T14:42:38.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.801+0000 7f74b9600700 1 -- 172.21.15.62:0/1081721425 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f74b009d150 con 0x7f74bc1046e0 2024-04-02T14:42:38.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.804+0000 7f74c2200700 1 -- 172.21.15.62:0/1081721425 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f74a40a3cd0 msgr2=0x7f74a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:38.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.804+0000 7f74c2200700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f74a40a3cd0 0x7f74a40a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f74bc109c30 tx=0x7f74a801a040 comp rx=0 tx=0).stop 2024-04-02T14:42:38.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.804+0000 7f74c2200700 1 -- 172.21.15.62:0/1081721425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc1046e0 msgr2=0x7f74bc19b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:38.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.804+0000 7f74c2200700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc1046e0 0x7f74bc19b070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74b000e9c0 tx=0x7f74b000ecd0 comp rx=0 tx=0).stop 2024-04-02T14:42:38.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.805+0000 7f74c2200700 1 -- 172.21.15.62:0/1081721425 shutdown_connections 2024-04-02T14:42:38.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.805+0000 7f74c2200700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f74a40a3cd0 0x7f74a40a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:38.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.805+0000 7f74c2200700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74bc10a6e0 0x7f74bc19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:38.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.805+0000 7f74c2200700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc1046e0 0x7f74bc19b070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:38.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.805+0000 7f74c2200700 1 --2- 172.21.15.62:0/1081721425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc1050a0 0x7f74bc19b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:38.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.805+0000 7f74c2200700 1 -- 172.21.15.62:0/1081721425 >> 172.21.15.62:0/1081721425 conn(0x7f74bc100210 msgr2=0x7f74bc100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:38.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.805+0000 7f74c2200700 1 -- 172.21.15.62:0/1081721425 shutdown_connections 2024-04-02T14:42:38.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:38.805+0000 7f74c2200700 1 -- 172.21.15.62:0/1081721425 wait complete. 2024-04-02T14:42:39.175 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:39.180 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:39.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:39 smithi062 ceph-mon[32116]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:39.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:39 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1081721425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T14:42:39.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:39 smithi113 ceph-mon[36135]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:39.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:39 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1081721425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T14:42:39.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:39 smithi204 ceph-mon[35948]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:39.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:39 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1081721425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T14:42:41.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:41 smithi062 ceph-mon[32116]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:42:41.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:41 smithi113 ceph-mon[36135]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:42:41.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:41 smithi204 ceph-mon[35948]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:42:41.651 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738458 for osd.0 2024-04-02T14:42:41.652 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:41.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.927+0000 7fe5c0c00700 1 -- 172.21.15.62:0/2721761006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5bc1094e0 msgr2=0x7fe5bc1159a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:41.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.927+0000 7fe5c0c00700 1 --2- 172.21.15.62:0/2721761006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5bc1094e0 0x7fe5bc1159a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400b0a0 tx=0x7fe5b400b3b0 comp rx=0 tx=0).stop 2024-04-02T14:42:41.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.927+0000 7fe5c0c00700 1 -- 172.21.15.62:0/2721761006 shutdown_connections 2024-04-02T14:42:41.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.927+0000 7fe5c0c00700 1 --2- 172.21.15.62:0/2721761006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5bc10eb60 0x7fe5bc10ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:41.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.927+0000 7fe5c0c00700 1 --2- 172.21.15.62:0/2721761006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5bc1094e0 0x7fe5bc1159a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:41.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.927+0000 7fe5c0c00700 1 --2- 172.21.15.62:0/2721761006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5bc108b20 0x7fe5bc108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:41.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.927+0000 7fe5c0c00700 1 -- 172.21.15.62:0/2721761006 >> 172.21.15.62:0/2721761006 conn(0x7fe5bc078290 msgr2=0x7fe5bc07a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:41.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.927+0000 7fe5c0c00700 1 -- 172.21.15.62:0/2721761006 shutdown_connections 2024-04-02T14:42:41.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.927+0000 7fe5c0c00700 1 -- 172.21.15.62:0/2721761006 wait complete. 2024-04-02T14:42:41.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.928+0000 7fe5c0c00700 1 Processor -- start 2024-04-02T14:42:41.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.928+0000 7fe5c0c00700 1 -- start start 2024-04-02T14:42:41.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.929+0000 7fe5c0c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5bc108b20 0x7fe5bc19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:41.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.929+0000 7fe5c0c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5bc1094e0 0x7fe5bc19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:41.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.929+0000 7fe5c0c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5bc10eb60 0x7fe5bc1a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:41.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.929+0000 7fe5c0c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc118f40 con 0x7fe5bc108b20 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.929+0000 7fe5c0c00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc118dc0 con 0x7fe5bc1094e0 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.929+0000 7fe5c0c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1190c0 con 0x7fe5bc10eb60 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.929+0000 7fe5bb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5bc108b20 0x7fe5bc19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.929+0000 7fe5bb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5bc108b20 0x7fe5bc19f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38856/0 (socket says 172.21.15.62:38856) 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.929+0000 7fe5bb400700 1 -- 172.21.15.62:0/736320251 learned_addr learned my addr 172.21.15.62:0/736320251 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.929+0000 7fe5bbe00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5bc10eb60 0x7fe5bc1a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.929+0000 7fe5baa00700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5bc1094e0 0x7fe5bc19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.930+0000 7fe5bbe00700 1 -- 172.21.15.62:0/736320251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5bc1094e0 msgr2=0x7fe5bc19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.930+0000 7fe5bbe00700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5bc1094e0 0x7fe5bc19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.930+0000 7fe5bbe00700 1 -- 172.21.15.62:0/736320251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5bc108b20 msgr2=0x7fe5bc19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.930+0000 7fe5bbe00700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5bc108b20 0x7fe5bc19f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.930+0000 7fe5bbe00700 1 -- 172.21.15.62:0/736320251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc1a42b0 con 0x7fe5bc10eb60 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.930+0000 7fe5bb400700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5bc108b20 0x7fe5bc19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:41.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.930+0000 7fe5bbe00700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5bc10eb60 0x7fe5bc1a3b70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400b0a0 tx=0x7fe5b4004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:41.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.930+0000 7fe5b3e00700 1 -- 172.21.15.62:0/736320251 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4021070 con 0x7fe5bc10eb60 2024-04-02T14:42:41.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.930+0000 7fe5c0c00700 1 -- 172.21.15.62:0/736320251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc1a4540 con 0x7fe5bc10eb60 2024-04-02T14:42:41.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.930+0000 7fe5b3e00700 1 -- 172.21.15.62:0/736320251 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe5b4007b80 con 0x7fe5bc10eb60 2024-04-02T14:42:41.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.930+0000 7fe5b3e00700 1 -- 172.21.15.62:0/736320251 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40052a0 con 0x7fe5bc10eb60 2024-04-02T14:42:41.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.931+0000 7fe5c0c00700 1 -- 172.21.15.62:0/736320251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5bc1abe20 con 0x7fe5bc10eb60 2024-04-02T14:42:41.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.931+0000 7fe5c0c00700 1 -- 172.21.15.62:0/736320251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5bc0692b0 con 0x7fe5bc10eb60 2024-04-02T14:42:41.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.934+0000 7fe5b3e00700 1 -- 172.21.15.62:0/736320251 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe5b4024070 con 0x7fe5bc10eb60 2024-04-02T14:42:41.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.935+0000 7fe5b3e00700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe5a00a3cd0 0x7fe5a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:41.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.935+0000 7fe5b3e00700 1 -- 172.21.15.62:0/736320251 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fe5b400f020 con 0x7fe5bc10eb60 2024-04-02T14:42:41.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.935+0000 7fe5bb400700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe5a00a3cd0 0x7fe5a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:41.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.936+0000 7fe5bb400700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe5a00a3cd0 0x7fe5a00a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0103d0 tx=0x7fe5ac007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:41.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:41.938+0000 7fe5b3e00700 1 -- 172.21.15.62:0/736320251 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe5b402b020 con 0x7fe5bc10eb60 2024-04-02T14:42:42.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.185+0000 7fe5c0c00700 1 -- 172.21.15.62:0/736320251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe5bc1abff0 con 0x7fe5bc10eb60 2024-04-02T14:42:42.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.186+0000 7fe5b3e00700 1 -- 172.21.15.62:0/736320251 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe5b409e950 con 0x7fe5bc10eb60 2024-04-02T14:42:42.184 INFO:teuthology.orchestra.run.smithi062.stdout:171798691903 2024-04-02T14:42:42.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.189+0000 7fe5c0c00700 1 -- 172.21.15.62:0/736320251 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe5a00a3cd0 msgr2=0x7fe5a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:42.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.189+0000 7fe5c0c00700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe5a00a3cd0 0x7fe5a00a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0103d0 tx=0x7fe5ac007480 comp rx=0 tx=0).stop 2024-04-02T14:42:42.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.189+0000 7fe5c0c00700 1 -- 172.21.15.62:0/736320251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5bc10eb60 msgr2=0x7fe5bc1a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:42.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.189+0000 7fe5c0c00700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5bc10eb60 0x7fe5bc1a3b70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400b0a0 tx=0x7fe5b4004d00 comp rx=0 tx=0).stop 2024-04-02T14:42:42.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.189+0000 7fe5c0c00700 1 -- 172.21.15.62:0/736320251 shutdown_connections 2024-04-02T14:42:42.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.189+0000 7fe5c0c00700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fe5a00a3cd0 0x7fe5a00a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:42.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.189+0000 7fe5c0c00700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5bc10eb60 0x7fe5bc1a3b70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:42.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.189+0000 7fe5c0c00700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5bc108b20 0x7fe5bc19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:42.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.189+0000 7fe5c0c00700 1 --2- 172.21.15.62:0/736320251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5bc1094e0 0x7fe5bc19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:42.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.189+0000 7fe5c0c00700 1 -- 172.21.15.62:0/736320251 >> 172.21.15.62:0/736320251 conn(0x7fe5bc078290 msgr2=0x7fe5bc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:42.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.189+0000 7fe5c0c00700 1 -- 172.21.15.62:0/736320251 shutdown_connections 2024-04-02T14:42:42.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:42.190+0000 7fe5c0c00700 1 -- 172.21.15.62:0/736320251 wait complete. 2024-04-02T14:42:42.985 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285590 got 356482285596 for osd.11 2024-04-02T14:42:42.985 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:43.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:43 smithi062 ceph-mon[32116]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:42:43.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:43 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/736320251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T14:42:43.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:43 smithi113 ceph-mon[36135]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:42:43.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:43 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/736320251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T14:42:43.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:43 smithi204 ceph-mon[35948]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:42:43.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:43 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/736320251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T14:42:44.194 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972392 got 266287972397 for osd.8 2024-04-02T14:42:44.194 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:44.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.830+0000 7f3715e00700 1 -- 172.21.15.62:0/2914224811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37101024f0 msgr2=0x7f3710102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:44.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.830+0000 7f3715e00700 1 --2- 172.21.15.62:0/2914224811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37101024f0 0x7f3710102970 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3700009920 tx=0x7f3700009c30 comp rx=0 tx=0).stop 2024-04-02T14:42:44.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.830+0000 7f3715e00700 1 -- 172.21.15.62:0/2914224811 shutdown_connections 2024-04-02T14:42:44.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.830+0000 7f3715e00700 1 --2- 172.21.15.62:0/2914224811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3710102eb0 0x7f371010f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:44.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.830+0000 7f3715e00700 1 --2- 172.21.15.62:0/2914224811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37101085a0 0x7f3710108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:44.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.830+0000 7f3715e00700 1 --2- 172.21.15.62:0/2914224811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37101024f0 0x7f3710102970 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:44.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.830+0000 7f3715e00700 1 -- 172.21.15.62:0/2914224811 >> 172.21.15.62:0/2914224811 conn(0x7f37100fdfc0 msgr2=0x7f37101003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:44.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.830+0000 7f3715e00700 1 -- 172.21.15.62:0/2914224811 shutdown_connections 2024-04-02T14:42:44.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.830+0000 7f3715e00700 1 -- 172.21.15.62:0/2914224811 wait complete. 2024-04-02T14:42:44.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.832+0000 7f3715e00700 1 Processor -- start 2024-04-02T14:42:44.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.832+0000 7f3715e00700 1 -- start start 2024-04-02T14:42:44.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.832+0000 7f3715e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37101024f0 0x7f371019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:44.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.832+0000 7f3715e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3710102eb0 0x7f371019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:44.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.832+0000 7f3715e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37101085a0 0x7f371019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:44.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3715e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3710112920 con 0x7f37101085a0 2024-04-02T14:42:44.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3715e00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f37101127a0 con 0x7f3710102eb0 2024-04-02T14:42:44.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3715e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3710112aa0 con 0x7f37101024f0 2024-04-02T14:42:44.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3707e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3710102eb0 0x7f371019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:44.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3714a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37101024f0 0x7f371019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:44.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3715400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37101085a0 0x7f371019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:44.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3714a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37101024f0 0x7f371019af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50368/0 (socket says 172.21.15.62:50368) 2024-04-02T14:42:44.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3714a00700 1 -- 172.21.15.62:0/2619011435 learned_addr learned my addr 172.21.15.62:0/2619011435 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:44.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3707e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3710102eb0 0x7f371019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37032/0 (socket says 172.21.15.62:37032) 2024-04-02T14:42:44.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3714a00700 1 -- 172.21.15.62:0/2619011435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3710102eb0 msgr2=0x7f371019b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:44.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3714a00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3710102eb0 0x7f371019b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:44.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3714a00700 1 -- 172.21.15.62:0/2619011435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37101085a0 msgr2=0x7f371019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:44.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3714a00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37101085a0 0x7f371019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:44.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3714a00700 1 -- 172.21.15.62:0/2619011435 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371019ff30 con 0x7f37101024f0 2024-04-02T14:42:44.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.833+0000 7f3707e00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3710102eb0 0x7f371019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:44.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.834+0000 7f3714a00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37101024f0 0x7f371019af20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36f800e9a0 tx=0x7f36f800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:44.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.834+0000 7f370e000700 1 -- 172.21.15.62:0/2619011435 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f800ca10 con 0x7f37101024f0 2024-04-02T14:42:44.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.834+0000 7f3715e00700 1 -- 172.21.15.62:0/2619011435 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37101a0220 con 0x7f37101024f0 2024-04-02T14:42:44.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.834+0000 7f370e000700 1 -- 172.21.15.62:0/2619011435 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f36f800cbb0 con 0x7f37101024f0 2024-04-02T14:42:44.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.834+0000 7f370e000700 1 -- 172.21.15.62:0/2619011435 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8018710 con 0x7f37101024f0 2024-04-02T14:42:44.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.834+0000 7f3715e00700 1 -- 172.21.15.62:0/2619011435 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37101a7aa0 con 0x7f37101024f0 2024-04-02T14:42:44.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.835+0000 7f3715e00700 1 -- 172.21.15.62:0/2619011435 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37100692b0 con 0x7f37101024f0 2024-04-02T14:42:44.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.837+0000 7f370e000700 1 -- 172.21.15.62:0/2619011435 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f36f8010430 con 0x7f37101024f0 2024-04-02T14:42:44.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.839+0000 7f370e000700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f36f00a3d50 0x7f36f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:44.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.839+0000 7f370e000700 1 -- 172.21.15.62:0/2619011435 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f36f80d06f0 con 0x7f37101024f0 2024-04-02T14:42:44.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.839+0000 7f3707e00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f36f00a3d50 0x7f36f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:44.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.840+0000 7f3707e00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f36f00a3d50 0x7f36f00a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3700005e90 tx=0x7f3700019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:44.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:44.843+0000 7f370e000700 1 -- 172.21.15.62:0/2619011435 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f36f809cf40 con 0x7f37101024f0 2024-04-02T14:42:45.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.083+0000 7f3715e00700 1 -- 172.21.15.62:0/2619011435 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f37101a7cd0 con 0x7f37101024f0 2024-04-02T14:42:45.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:45 smithi062 ceph-mon[32116]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:45.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.084+0000 7f370e000700 1 -- 172.21.15.62:0/2619011435 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f36f809c840 con 0x7f37101024f0 2024-04-02T14:42:45.082 INFO:teuthology.orchestra.run.smithi062.stdout:77309411410 2024-04-02T14:42:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.087+0000 7f3715e00700 1 -- 172.21.15.62:0/2619011435 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f36f00a3d50 msgr2=0x7f36f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.087+0000 7f3715e00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f36f00a3d50 0x7f36f00a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3700005e90 tx=0x7f3700019040 comp rx=0 tx=0).stop 2024-04-02T14:42:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.087+0000 7f3715e00700 1 -- 172.21.15.62:0/2619011435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37101024f0 msgr2=0x7f371019af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.087+0000 7f3715e00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37101024f0 0x7f371019af20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36f800e9a0 tx=0x7f36f800ecb0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.087+0000 7f3715e00700 1 -- 172.21.15.62:0/2619011435 shutdown_connections 2024-04-02T14:42:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.087+0000 7f3715e00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f36f00a3d50 0x7f36f00a6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.087+0000 7f3715e00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37101085a0 0x7f371019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.087+0000 7f3715e00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37101024f0 0x7f371019af20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.087+0000 7f3715e00700 1 --2- 172.21.15.62:0/2619011435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3710102eb0 0x7f371019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.087+0000 7f3715e00700 1 -- 172.21.15.62:0/2619011435 >> 172.21.15.62:0/2619011435 conn(0x7f37100fdfc0 msgr2=0x7f37100ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:45.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.087+0000 7f3715e00700 1 -- 172.21.15.62:0/2619011435 shutdown_connections 2024-04-02T14:42:45.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.088+0000 7f3715e00700 1 -- 172.21.15.62:0/2619011435 wait complete. 2024-04-02T14:42:45.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:45 smithi113 ceph-mon[36135]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:45.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:45 smithi204 ceph-mon[35948]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:45.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.212+0000 7ff066c00700 1 -- 172.21.15.62:0/3202972843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff060102670 msgr2=0x7ff060102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:45.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.212+0000 7ff066c00700 1 --2- 172.21.15.62:0/3202972843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff060102670 0x7ff060102af0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009990 tx=0x7ff04c009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.212+0000 7ff066c00700 1 -- 172.21.15.62:0/3202972843 shutdown_connections 2024-04-02T14:42:45.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.213+0000 7ff066c00700 1 --2- 172.21.15.62:0/3202972843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff060103030 0x7ff06010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.213+0000 7ff066c00700 1 --2- 172.21.15.62:0/3202972843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060108680 0x7ff060108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.213+0000 7ff066c00700 1 --2- 172.21.15.62:0/3202972843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff060102670 0x7ff060102af0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.213+0000 7ff066c00700 1 -- 172.21.15.62:0/3202972843 >> 172.21.15.62:0/3202972843 conn(0x7ff060079740 msgr2=0x7ff060079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.213+0000 7ff066c00700 1 -- 172.21.15.62:0/3202972843 shutdown_connections 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.213+0000 7ff066c00700 1 -- 172.21.15.62:0/3202972843 wait complete. 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.214+0000 7ff066c00700 1 Processor -- start 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.214+0000 7ff066c00700 1 -- start start 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.214+0000 7ff066c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060102670 0x7ff060198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.214+0000 7ff066c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff060103030 0x7ff0601993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.214+0000 7ff066c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff060108680 0x7ff06019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.214+0000 7ff066c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff060112a60 con 0x7ff060108680 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.214+0000 7ff066c00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff0601128e0 con 0x7ff060103030 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.214+0000 7ff066c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff060112be0 con 0x7ff060102670 2024-04-02T14:42:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.215+0000 7ff05fe00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060102670 0x7ff060198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.215+0000 7ff064c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff060108680 0x7ff06019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.215+0000 7ff05fe00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060102670 0x7ff060198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50398/0 (socket says 172.21.15.62:50398) 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.215+0000 7ff05fe00700 1 -- 172.21.15.62:0/3089116723 learned_addr learned my addr 172.21.15.62:0/3089116723 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.215+0000 7ff05f400700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff060103030 0x7ff0601993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.215+0000 7ff05fe00700 1 -- 172.21.15.62:0/3089116723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff060103030 msgr2=0x7ff0601993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.215+0000 7ff05fe00700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff060103030 0x7ff0601993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.215+0000 7ff05fe00700 1 -- 172.21.15.62:0/3089116723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff060108680 msgr2=0x7ff06019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.215+0000 7ff05fe00700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff060108680 0x7ff06019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.215+0000 7ff05fe00700 1 -- 172.21.15.62:0/3089116723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06019de50 con 0x7ff060102670 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.215+0000 7ff064c00700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff060108680 0x7ff06019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.216+0000 7ff05fe00700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060102670 0x7ff060198e90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff05000e950 tx=0x7ff05000ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.216+0000 7ff05cc00700 1 -- 172.21.15.62:0/3089116723 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050004230 con 0x7ff060102670 2024-04-02T14:42:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.216+0000 7ff066c00700 1 -- 172.21.15.62:0/3089116723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06019e140 con 0x7ff060102670 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.216+0000 7ff05cc00700 1 -- 172.21.15.62:0/3089116723 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0500043d0 con 0x7ff060102670 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.216+0000 7ff05cc00700 1 -- 172.21.15.62:0/3089116723 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050010430 con 0x7ff060102670 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.216+0000 7ff066c00700 1 -- 172.21.15.62:0/3089116723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0601a59c0 con 0x7ff060102670 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.217+0000 7ff066c00700 1 -- 172.21.15.62:0/3089116723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0600692b0 con 0x7ff060102670 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.221+0000 7ff05cc00700 1 -- 172.21.15.62:0/3089116723 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff05000c7e0 con 0x7ff060102670 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.222+0000 7ff05cc00700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0440a3cd0 0x7ff0440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.223+0000 7ff05cc00700 1 -- 172.21.15.62:0/3089116723 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7ff0500d0730 con 0x7ff060102670 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.223+0000 7ff05f400700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0440a3cd0 0x7ff0440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.223+0000 7ff05f400700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0440a3cd0 0x7ff0440a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00b2e0 tx=0x7ff04c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.226+0000 7ff05cc00700 1 -- 172.21.15.62:0/3089116723 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff05009cf80 con 0x7ff060102670 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.495+0000 7ff066c00700 1 -- 172.21.15.62:0/3089116723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff06019a1a0 con 0x7ff060102670 2024-04-02T14:42:45.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.495+0000 7ff05cc00700 1 -- 172.21.15.62:0/3089116723 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff05009c880 con 0x7ff060102670 2024-04-02T14:42:45.780 INFO:teuthology.orchestra.run.smithi062.stdout:150323855430 2024-04-02T14:42:45.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.498+0000 7ff066c00700 1 -- 172.21.15.62:0/3089116723 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0440a3cd0 msgr2=0x7ff0440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:45.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.498+0000 7ff066c00700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0440a3cd0 0x7ff0440a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00b2e0 tx=0x7ff04c005960 comp rx=0 tx=0).stop 2024-04-02T14:42:45.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.498+0000 7ff066c00700 1 -- 172.21.15.62:0/3089116723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060102670 msgr2=0x7ff060198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:45.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.498+0000 7ff066c00700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060102670 0x7ff060198e90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff05000e950 tx=0x7ff05000ec60 comp rx=0 tx=0).stop 2024-04-02T14:42:45.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.499+0000 7ff066c00700 1 -- 172.21.15.62:0/3089116723 shutdown_connections 2024-04-02T14:42:45.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.499+0000 7ff066c00700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0440a3cd0 0x7ff0440a6190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.499+0000 7ff066c00700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff060108680 0x7ff06019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.499+0000 7ff066c00700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060102670 0x7ff060198e90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.499+0000 7ff066c00700 1 --2- 172.21.15.62:0/3089116723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff060103030 0x7ff0601993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:45.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.499+0000 7ff066c00700 1 -- 172.21.15.62:0/3089116723 >> 172.21.15.62:0/3089116723 conn(0x7ff060079740 msgr2=0x7ff0601010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:45.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.499+0000 7ff066c00700 1 -- 172.21.15.62:0/3089116723 shutdown_connections 2024-04-02T14:42:45.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:45.499+0000 7ff066c00700 1 -- 172.21.15.62:0/3089116723 wait complete. 2024-04-02T14:42:45.996 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691898 got 171798691903 for osd.5 2024-04-02T14:42:45.996 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:46.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:46 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2619011435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T14:42:46.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:46 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3089116723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T14:42:46.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:46 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2619011435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T14:42:46.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:46 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3089116723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T14:42:46.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:46 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2619011435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T14:42:46.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:46 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3089116723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T14:42:47.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:47 smithi062 ceph-mon[32116]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:42:47.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.166+0000 7f8e2fa00700 1 -- 172.21.15.62:0/1517653336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e280752e0 msgr2=0x7f8e2810e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.166+0000 7f8e2fa00700 1 --2- 172.21.15.62:0/1517653336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e280752e0 0x7f8e2810e970 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009990 tx=0x7f8e14009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.166+0000 7f8e2fa00700 1 -- 172.21.15.62:0/1517653336 shutdown_connections 2024-04-02T14:42:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.166+0000 7f8e2fa00700 1 --2- 172.21.15.62:0/1517653336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2810eeb0 0x7f8e281112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.166+0000 7f8e2fa00700 1 --2- 172.21.15.62:0/1517653336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e280749c0 0x7f8e28074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.166+0000 7f8e2fa00700 1 --2- 172.21.15.62:0/1517653336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e280752e0 0x7f8e2810e970 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.166+0000 7f8e2fa00700 1 -- 172.21.15.62:0/1517653336 >> 172.21.15.62:0/1517653336 conn(0x7f8e280fe0c0 msgr2=0x7f8e281004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.167+0000 7f8e2fa00700 1 -- 172.21.15.62:0/1517653336 shutdown_connections 2024-04-02T14:42:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.167+0000 7f8e2fa00700 1 -- 172.21.15.62:0/1517653336 wait complete. 2024-04-02T14:42:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.168+0000 7f8e2fa00700 1 Processor -- start 2024-04-02T14:42:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.168+0000 7f8e2fa00700 1 -- start start 2024-04-02T14:42:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.168+0000 7f8e2fa00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e280749c0 0x7f8e2819b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.168+0000 7f8e2fa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e280752e0 0x7f8e2819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.168+0000 7f8e2fa00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2810eeb0 0x7f8e2819fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.168+0000 7f8e2fa00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28114d10 con 0x7f8e280752e0 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.168+0000 7f8e2fa00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28114b90 con 0x7f8e2810eeb0 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.168+0000 7f8e2fa00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28114e90 con 0x7f8e280749c0 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.169+0000 7f8e27e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e280752e0 0x7f8e2819b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.169+0000 7f8e27e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e280752e0 0x7f8e2819b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37074/0 (socket says 172.21.15.62:37074) 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.169+0000 7f8e27e00700 1 -- 172.21.15.62:0/3472456313 learned_addr learned my addr 172.21.15.62:0/3472456313 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.169+0000 7f8e2da00700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2810eeb0 0x7f8e2819fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.169+0000 7f8e2d000700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e280749c0 0x7f8e2819b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.169+0000 7f8e27e00700 1 -- 172.21.15.62:0/3472456313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e280749c0 msgr2=0x7f8e2819b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.169+0000 7f8e27e00700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e280749c0 0x7f8e2819b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.169+0000 7f8e27e00700 1 -- 172.21.15.62:0/3472456313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2810eeb0 msgr2=0x7f8e2819fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:47.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.169+0000 7f8e27e00700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2810eeb0 0x7f8e2819fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:47.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.169+0000 7f8e27e00700 1 -- 172.21.15.62:0/3472456313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e281a0140 con 0x7f8e280752e0 2024-04-02T14:42:47.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.169+0000 7f8e27e00700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e280752e0 0x7f8e2819b6c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e14005910 tx=0x7f8e140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:47.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.170+0000 7f8e25600700 1 -- 172.21.15.62:0/3472456313 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14027070 con 0x7f8e280752e0 2024-04-02T14:42:47.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.170+0000 7f8e2fa00700 1 -- 172.21.15.62:0/3472456313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e281a03d0 con 0x7f8e280752e0 2024-04-02T14:42:47.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.170+0000 7f8e25600700 1 -- 172.21.15.62:0/3472456313 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8e1400fe00 con 0x7f8e280752e0 2024-04-02T14:42:47.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.170+0000 7f8e25600700 1 -- 172.21.15.62:0/3472456313 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14017650 con 0x7f8e280752e0 2024-04-02T14:42:47.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.170+0000 7f8e2fa00700 1 -- 172.21.15.62:0/3472456313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e281a7cb0 con 0x7f8e280752e0 2024-04-02T14:42:47.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.171+0000 7f8e2fa00700 1 -- 172.21.15.62:0/3472456313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e280692b0 con 0x7f8e280752e0 2024-04-02T14:42:47.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.173+0000 7f8e25600700 1 -- 172.21.15.62:0/3472456313 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8e1402a070 con 0x7f8e280752e0 2024-04-02T14:42:47.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.174+0000 7f8e25600700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8e100a3cd0 0x7f8e100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:47.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.175+0000 7f8e25600700 1 -- 172.21.15.62:0/3472456313 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f8e140d1930 con 0x7f8e280752e0 2024-04-02T14:42:47.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.175+0000 7f8e2d000700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8e100a3cd0 0x7f8e100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:47.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.176+0000 7f8e2d000700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8e100a3cd0 0x7f8e100a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009c60 tx=0x7f8e1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:47.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.177+0000 7f8e25600700 1 -- 172.21.15.62:0/3472456313 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8e14031070 con 0x7f8e280752e0 2024-04-02T14:42:47.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:47 smithi113 ceph-mon[36135]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:42:47.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.413+0000 7f8e2fa00700 1 -- 172.21.15.62:0/3472456313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8e2802dd30 con 0x7f8e280752e0 2024-04-02T14:42:47.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.414+0000 7f8e25600700 1 -- 172.21.15.62:0/3472456313 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8e14013070 con 0x7f8e280752e0 2024-04-02T14:42:47.412 INFO:teuthology.orchestra.run.smithi062.stdout:206158430266 2024-04-02T14:42:47.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.417+0000 7f8e2fa00700 1 -- 172.21.15.62:0/3472456313 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8e100a3cd0 msgr2=0x7f8e100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:47.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.417+0000 7f8e2fa00700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8e100a3cd0 0x7f8e100a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009c60 tx=0x7f8e1c009260 comp rx=0 tx=0).stop 2024-04-02T14:42:47.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.417+0000 7f8e2fa00700 1 -- 172.21.15.62:0/3472456313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e280752e0 msgr2=0x7f8e2819b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:47.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.417+0000 7f8e2fa00700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e280752e0 0x7f8e2819b6c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e14005910 tx=0x7f8e140047b0 comp rx=0 tx=0).stop 2024-04-02T14:42:47.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.418+0000 7f8e2fa00700 1 -- 172.21.15.62:0/3472456313 shutdown_connections 2024-04-02T14:42:47.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.418+0000 7f8e2fa00700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8e100a3cd0 0x7f8e100a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:47.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.418+0000 7f8e2fa00700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2810eeb0 0x7f8e2819fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:47.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.418+0000 7f8e2fa00700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e280749c0 0x7f8e2819b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:47.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.418+0000 7f8e2fa00700 1 --2- 172.21.15.62:0/3472456313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e280752e0 0x7f8e2819b6c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:47.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.418+0000 7f8e2fa00700 1 -- 172.21.15.62:0/3472456313 >> 172.21.15.62:0/3472456313 conn(0x7f8e280fe0c0 msgr2=0x7f8e280fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:47.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.418+0000 7f8e2fa00700 1 -- 172.21.15.62:0/3472456313 shutdown_connections 2024-04-02T14:42:47.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:47.418+0000 7f8e2fa00700 1 -- 172.21.15.62:0/3472456313 wait complete. 2024-04-02T14:42:47.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:47 smithi204 ceph-mon[35948]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:42:48.001 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710755 got 300647710759 for osd.9 2024-04-02T14:42:48.002 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:48.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:48 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3472456313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T14:42:48.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:48 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3472456313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T14:42:48.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:48 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3472456313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T14:42:49.003 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855424 got 150323855430 for osd.4 2024-04-02T14:42:49.003 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:49.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:49 smithi062 ceph-mon[32116]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:49.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:49 smithi113 ceph-mon[36135]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:49.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:49 smithi204 ceph-mon[35948]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:42:49.563 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266733 got 227633266739 for osd.7 2024-04-02T14:42:49.563 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:49.712 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547226 got 322122547232 for osd.10 2024-04-02T14:42:49.713 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:50.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.647+0000 7f148a800700 1 -- 172.21.15.62:0/1711013277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14841046c0 msgr2=0x7f1484104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:50.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.647+0000 7f148a800700 1 --2- 172.21.15.62:0/1711013277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14841046c0 0x7f1484104b40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1470009990 tx=0x7f1470009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:50.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.647+0000 7f148a800700 1 -- 172.21.15.62:0/1711013277 shutdown_connections 2024-04-02T14:42:50.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.647+0000 7f148a800700 1 --2- 172.21.15.62:0/1711013277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1484105080 0x7f1484111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:50.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.647+0000 7f148a800700 1 --2- 172.21.15.62:0/1711013277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f148410a6c0 0x7f148410aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:50.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.647+0000 7f148a800700 1 --2- 172.21.15.62:0/1711013277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14841046c0 0x7f1484104b40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:50.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.647+0000 7f148a800700 1 -- 172.21.15.62:0/1711013277 >> 172.21.15.62:0/1711013277 conn(0x7f14841001d0 msgr2=0x7f14841025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:50.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.647+0000 7f148a800700 1 -- 172.21.15.62:0/1711013277 shutdown_connections 2024-04-02T14:42:50.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.647+0000 7f148a800700 1 -- 172.21.15.62:0/1711013277 wait complete. 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.648+0000 7f148a800700 1 Processor -- start 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.648+0000 7f148a800700 1 -- start start 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f148a800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14841046c0 0x7f148419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f148a800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1484105080 0x7f148419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f148a800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f148410a6c0 0x7f148419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f148a800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1484114b70 con 0x7f14841046c0 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f148a800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f14841149f0 con 0x7f148410a6c0 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f148a800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1484114cf0 con 0x7f1484105080 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f1489400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14841046c0 0x7f148419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f1489400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14841046c0 0x7f148419af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37096/0 (socket says 172.21.15.62:37096) 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f1489400700 1 -- 172.21.15.62:0/4110139194 learned_addr learned my addr 172.21.15.62:0/4110139194 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f1488a00700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1484105080 0x7f148419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f1489e00700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f148410a6c0 0x7f148419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f1488a00700 1 -- 172.21.15.62:0/4110139194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f148410a6c0 msgr2=0x7f148419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f1488a00700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f148410a6c0 0x7f148419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f1488a00700 1 -- 172.21.15.62:0/4110139194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14841046c0 msgr2=0x7f148419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f1488a00700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14841046c0 0x7f148419af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.649+0000 7f1488a00700 1 -- 172.21.15.62:0/4110139194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148419ff20 con 0x7f1484105080 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.650+0000 7f1488a00700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1484105080 0x7f148419b4a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f14700059d0 tx=0x7f147000b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.650+0000 7f1482000700 1 -- 172.21.15.62:0/4110139194 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470027070 con 0x7f1484105080 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.650+0000 7f148a800700 1 -- 172.21.15.62:0/4110139194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14841a01b0 con 0x7f1484105080 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.650+0000 7f1482000700 1 -- 172.21.15.62:0/4110139194 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f147000fe00 con 0x7f1484105080 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.650+0000 7f1482000700 1 -- 172.21.15.62:0/4110139194 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700173f0 con 0x7f1484105080 2024-04-02T14:42:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.650+0000 7f148a800700 1 -- 172.21.15.62:0/4110139194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14841a7a90 con 0x7f1484105080 2024-04-02T14:42:50.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.651+0000 7f148a800700 1 -- 172.21.15.62:0/4110139194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f148410cc50 con 0x7f1484105080 2024-04-02T14:42:50.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.654+0000 7f1482000700 1 -- 172.21.15.62:0/4110139194 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f147002a070 con 0x7f1484105080 2024-04-02T14:42:50.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.655+0000 7f1482000700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f146c0a3cc0 0x7f146c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:50.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.655+0000 7f1482000700 1 -- 172.21.15.62:0/4110139194 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f14700d1570 con 0x7f1484105080 2024-04-02T14:42:50.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.655+0000 7f1489400700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f146c0a3cc0 0x7f146c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:50.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.656+0000 7f1489400700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f146c0a3cc0 0x7f146c0a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1478009c10 tx=0x7f147800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:50.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.657+0000 7f1482000700 1 -- 172.21.15.62:0/4110139194 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1470013070 con 0x7f1484105080 2024-04-02T14:42:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:50.814 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411403 got 77309411410 for osd.2 2024-04-02T14:42:50.814 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:50.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.916+0000 7f148a800700 1 -- 172.21.15.62:0/4110139194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f148419c270 con 0x7f1484105080 2024-04-02T14:42:50.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.917+0000 7f1482000700 1 -- 172.21.15.62:0/4110139194 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f147009dfb0 con 0x7f1484105080 2024-04-02T14:42:50.915 INFO:teuthology.orchestra.run.smithi062.stdout:115964117067 2024-04-02T14:42:50.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.919+0000 7f148a800700 1 -- 172.21.15.62:0/4110139194 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f146c0a3cc0 msgr2=0x7f146c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:50.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.919+0000 7f148a800700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f146c0a3cc0 0x7f146c0a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1478009c10 tx=0x7f147800b040 comp rx=0 tx=0).stop 2024-04-02T14:42:50.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.919+0000 7f148a800700 1 -- 172.21.15.62:0/4110139194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1484105080 msgr2=0x7f148419b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:50.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.919+0000 7f148a800700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1484105080 0x7f148419b4a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f14700059d0 tx=0x7f147000b610 comp rx=0 tx=0).stop 2024-04-02T14:42:50.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.920+0000 7f148a800700 1 -- 172.21.15.62:0/4110139194 shutdown_connections 2024-04-02T14:42:50.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.920+0000 7f148a800700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f146c0a3cc0 0x7f146c0a6180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:50.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.920+0000 7f148a800700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f148410a6c0 0x7f148419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:50.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.920+0000 7f148a800700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14841046c0 0x7f148419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:50.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.920+0000 7f148a800700 1 --2- 172.21.15.62:0/4110139194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1484105080 0x7f148419b4a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:50.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.920+0000 7f148a800700 1 -- 172.21.15.62:0/4110139194 >> 172.21.15.62:0/4110139194 conn(0x7f14841001d0 msgr2=0x7f1484100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:50.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.920+0000 7f148a800700 1 -- 172.21.15.62:0/4110139194 shutdown_connections 2024-04-02T14:42:50.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:50.920+0000 7f148a800700 1 -- 172.21.15.62:0/4110139194 wait complete. 2024-04-02T14:42:51.318 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:51 smithi062 ceph-mon[32116]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:42:51.318 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:51 smithi062 ceph-mon[32116]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T14:42:51.318 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:51 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/4110139194' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T14:42:51.321 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430261 got 206158430266 for osd.6 2024-04-02T14:42:51.321 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:51.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:51 smithi204 ceph-mon[35948]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:42:51.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:51 smithi204 ceph-mon[35948]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T14:42:51.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:51 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/4110139194' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T14:42:51.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:51 smithi113 ceph-mon[36135]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:42:51.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:51 smithi113 ceph-mon[36135]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T14:42:51.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:51 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/4110139194' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T14:42:53.107 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117061 got 115964117067 for osd.3 2024-04-02T14:42:53.107 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:53.418 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:53 smithi062 ceph-mon[32116]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-02T14:42:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:53 smithi204 ceph-mon[35948]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-02T14:42:53.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:53 smithi113 ceph-mon[36135]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-02T14:42:54.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.830+0000 7fc05d000700 1 -- 172.21.15.62:0/3387833501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc05806a2a0 msgr2=0x7fc05806a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:54.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.830+0000 7fc05d000700 1 --2- 172.21.15.62:0/3387833501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc05806a2a0 0x7fc05806a720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc044009990 tx=0x7fc044009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:54.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.831+0000 7fc05d000700 1 -- 172.21.15.62:0/3387833501 shutdown_connections 2024-04-02T14:42:54.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.831+0000 7fc05d000700 1 --2- 172.21.15.62:0/3387833501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc058108650 0x7fc058108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:54.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.831+0000 7fc05d000700 1 --2- 172.21.15.62:0/3387833501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc05806ac60 0x7fc05810f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:54.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.831+0000 7fc05d000700 1 --2- 172.21.15.62:0/3387833501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc05806a2a0 0x7fc05806a720 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:54.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.831+0000 7fc05d000700 1 -- 172.21.15.62:0/3387833501 >> 172.21.15.62:0/3387833501 conn(0x7fc058078420 msgr2=0x7fc058078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:54.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.831+0000 7fc05d000700 1 -- 172.21.15.62:0/3387833501 shutdown_connections 2024-04-02T14:42:54.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.831+0000 7fc05d000700 1 -- 172.21.15.62:0/3387833501 wait complete. 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.832+0000 7fc05d000700 1 Processor -- start 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.832+0000 7fc05d000700 1 -- start start 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.833+0000 7fc05d000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc05806a2a0 0x7fc05819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.833+0000 7fc05d000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc05806ac60 0x7fc05819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.833+0000 7fc05d000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc058108650 0x7fc05819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.833+0000 7fc05d000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc058112a20 con 0x7fc05806a2a0 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.833+0000 7fc05d000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581128a0 con 0x7fc05806ac60 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.833+0000 7fc05d000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc058112ba0 con 0x7fc058108650 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.833+0000 7fc055e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc05806a2a0 0x7fc05819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.833+0000 7fc056800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc058108650 0x7fc05819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.833+0000 7fc055400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc05806ac60 0x7fc05819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.833+0000 7fc056800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc058108650 0x7fc05819f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:59564/0 (socket says 172.21.15.62:59564) 2024-04-02T14:42:54.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.833+0000 7fc056800700 1 -- 172.21.15.62:0/1047452145 learned_addr learned my addr 172.21.15.62:0/1047452145 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:54.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.834+0000 7fc056800700 1 -- 172.21.15.62:0/1047452145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc05806ac60 msgr2=0x7fc05819b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:54.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.834+0000 7fc056800700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc05806ac60 0x7fc05819b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:54.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.834+0000 7fc056800700 1 -- 172.21.15.62:0/1047452145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc05806a2a0 msgr2=0x7fc05819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:54.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.834+0000 7fc056800700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc05806a2a0 0x7fc05819aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:54.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.834+0000 7fc056800700 1 -- 172.21.15.62:0/1047452145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0581a0000 con 0x7fc058108650 2024-04-02T14:42:54.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.834+0000 7fc056800700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc058108650 0x7fc05819f8c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc05000ef30 tx=0x7fc05000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:54.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.834+0000 7fc04ea00700 1 -- 172.21.15.62:0/1047452145 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0500092e0 con 0x7fc058108650 2024-04-02T14:42:54.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.834+0000 7fc05d000700 1 -- 172.21.15.62:0/1047452145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0581a0290 con 0x7fc058108650 2024-04-02T14:42:54.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.834+0000 7fc04ea00700 1 -- 172.21.15.62:0/1047452145 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc05000ce40 con 0x7fc058108650 2024-04-02T14:42:54.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.835+0000 7fc04ea00700 1 -- 172.21.15.62:0/1047452145 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050007bf0 con 0x7fc058108650 2024-04-02T14:42:54.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.835+0000 7fc05d000700 1 -- 172.21.15.62:0/1047452145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0581a7b70 con 0x7fc058108650 2024-04-02T14:42:54.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.836+0000 7fc05d000700 1 -- 172.21.15.62:0/1047452145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0580692b0 con 0x7fc058108650 2024-04-02T14:42:54.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.838+0000 7fc04ea00700 1 -- 172.21.15.62:0/1047452145 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc05001b020 con 0x7fc058108650 2024-04-02T14:42:54.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.840+0000 7fc04ea00700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc03c0a3cc0 0x7fc03c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:54.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.840+0000 7fc04ea00700 1 -- 172.21.15.62:0/1047452145 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fc0500d09f0 con 0x7fc058108650 2024-04-02T14:42:54.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.840+0000 7fc055e00700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc03c0a3cc0 0x7fc03c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:54.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.841+0000 7fc055e00700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc03c0a3cc0 0x7fc03c0a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc048009be0 tx=0x7fc048009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:54.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:54.842+0000 7fc04ea00700 1 -- 172.21.15.62:0/1047452145 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc050010040 con 0x7fc058108650 2024-04-02T14:42:55.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.076+0000 7fc05d000700 1 -- 172.21.15.62:0/1047452145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc0581a7d40 con 0x7fc058108650 2024-04-02T14:42:55.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.077+0000 7fc04ea00700 1 -- 172.21.15.62:0/1047452145 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc05009d3a0 con 0x7fc058108650 2024-04-02T14:42:55.075 INFO:teuthology.orchestra.run.smithi062.stdout:55834574935 2024-04-02T14:42:55.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.079+0000 7fc05d000700 1 -- 172.21.15.62:0/1047452145 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc03c0a3cc0 msgr2=0x7fc03c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:55.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.079+0000 7fc05d000700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc03c0a3cc0 0x7fc03c0a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc048009be0 tx=0x7fc048009260 comp rx=0 tx=0).stop 2024-04-02T14:42:55.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.079+0000 7fc05d000700 1 -- 172.21.15.62:0/1047452145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc058108650 msgr2=0x7fc05819f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:55.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.079+0000 7fc05d000700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc058108650 0x7fc05819f8c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc05000ef30 tx=0x7fc05000c3a0 comp rx=0 tx=0).stop 2024-04-02T14:42:55.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.080+0000 7fc05d000700 1 -- 172.21.15.62:0/1047452145 shutdown_connections 2024-04-02T14:42:55.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.080+0000 7fc05d000700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc03c0a3cc0 0x7fc03c0a6180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:55.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.080+0000 7fc05d000700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc058108650 0x7fc05819f8c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:55.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.080+0000 7fc05d000700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc05806a2a0 0x7fc05819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:55.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.080+0000 7fc05d000700 1 --2- 172.21.15.62:0/1047452145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc05806ac60 0x7fc05819b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:55.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.080+0000 7fc05d000700 1 -- 172.21.15.62:0/1047452145 >> 172.21.15.62:0/1047452145 conn(0x7fc058078420 msgr2=0x7fc05810d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:55.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.080+0000 7fc05d000700 1 -- 172.21.15.62:0/1047452145 shutdown_connections 2024-04-02T14:42:55.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:55.080+0000 7fc05d000700 1 -- 172.21.15.62:0/1047452145 wait complete. 2024-04-02T14:42:55.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:55 smithi062 ceph-mon[32116]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-02T14:42:55.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:55 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1047452145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T14:42:55.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:55 smithi204 ceph-mon[35948]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-02T14:42:55.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:55 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1047452145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T14:42:55.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:55 smithi113 ceph-mon[36135]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-02T14:42:55.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:55 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1047452145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T14:42:55.991 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574929 got 55834574935 for osd.1 2024-04-02T14:42:55.992 DEBUG:teuthology.parallel:result is None 2024-04-02T14:42:55.992 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-02T14:42:55.993 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T14:42:56.456 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:42:57.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:57 smithi062 ceph-mon[32116]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T14:42:57.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:57 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:57.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:57 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:57.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:57 smithi113 ceph-mon[36135]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T14:42:57.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:57 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:57.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:57 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:57.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:57 smithi204 ceph-mon[35948]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T14:42:57.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:57 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:57.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:57 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:58.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:58 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:42:58.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:58 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:42:58.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:58 smithi062 ceph-mon[32116]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:58.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:58 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:42:58.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:58 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:42:58.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:58 smithi113 ceph-mon[36135]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:58.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:58 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:42:58.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:58 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:42:58.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:58 smithi204 ceph-mon[35948]: from='mgr.14152 172.21.15.62:0/1603320989' entity='mgr.x' 2024-04-02T14:42:58.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.687+0000 7f6b8e600700 1 -- 172.21.15.62:0/3011639562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b88108630 msgr2=0x7f6b88108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:58.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.687+0000 7f6b8e600700 1 --2- 172.21.15.62:0/3011639562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b88108630 0x7f6b88108a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009990 tx=0x7f6b78009ca0 comp rx=0 tx=0).stop 2024-04-02T14:42:58.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.687+0000 7f6b8e600700 1 -- 172.21.15.62:0/3011639562 shutdown_connections 2024-04-02T14:42:58.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.687+0000 7f6b8e600700 1 --2- 172.21.15.62:0/3011639562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8806b4f0 0x7f6b8810f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:58.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.687+0000 7f6b8e600700 1 --2- 172.21.15.62:0/3011639562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b88108630 0x7f6b88108a10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:58.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.687+0000 7f6b8e600700 1 --2- 172.21.15.62:0/3011639562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b8806ab30 0x7f6b8806afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:58.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.687+0000 7f6b8e600700 1 -- 172.21.15.62:0/3011639562 >> 172.21.15.62:0/3011639562 conn(0x7f6b88077540 msgr2=0x7f6b88077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:58.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.687+0000 7f6b8e600700 1 -- 172.21.15.62:0/3011639562 shutdown_connections 2024-04-02T14:42:58.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.687+0000 7f6b8e600700 1 -- 172.21.15.62:0/3011639562 wait complete. 2024-04-02T14:42:58.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.689+0000 7f6b8e600700 1 Processor -- start 2024-04-02T14:42:58.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.689+0000 7f6b8e600700 1 -- start start 2024-04-02T14:42:58.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b8e600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b8806ab30 0x7f6b88198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b8e600700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8806b4f0 0x7f6b881993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b8e600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b88108630 0x7f6b8819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b8e600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88112a40 con 0x7f6b88108630 2024-04-02T14:42:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b8e600700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6b881128c0 con 0x7f6b8806b4f0 2024-04-02T14:42:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b8e600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88112bc0 con 0x7f6b8806ab30 2024-04-02T14:42:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b87e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b88108630 0x7f6b8819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b86a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8806b4f0 0x7f6b881993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b87400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b8806ab30 0x7f6b88198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b87e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b88108630 0x7f6b8819d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55560/0 (socket says 172.21.15.62:55560) 2024-04-02T14:42:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b87e00700 1 -- 172.21.15.62:0/2199593472 learned_addr learned my addr 172.21.15.62:0/2199593472 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:42:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.690+0000 7f6b86a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8806b4f0 0x7f6b881993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:50878/0 (socket says 172.21.15.62:50878) 2024-04-02T14:42:58.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.691+0000 7f6b86a00700 1 -- 172.21.15.62:0/2199593472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b8806ab30 msgr2=0x7f6b88198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:58.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.691+0000 7f6b86a00700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b8806ab30 0x7f6b88198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:58.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.691+0000 7f6b86a00700 1 -- 172.21.15.62:0/2199593472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b88108630 msgr2=0x7f6b8819d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:58.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.691+0000 7f6b86a00700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b88108630 0x7f6b8819d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:58.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.691+0000 7f6b86a00700 1 -- 172.21.15.62:0/2199593472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8819de70 con 0x7f6b8806b4f0 2024-04-02T14:42:58.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.691+0000 7f6b87400700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b8806ab30 0x7f6b88198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:42:58.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.691+0000 7f6b86a00700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8806b4f0 0x7f6b881993f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400e9f0 tx=0x7f6b7400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:58.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.691+0000 7f6b7fe00700 1 -- 172.21.15.62:0/2199593472 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7400cb70 con 0x7f6b8806b4f0 2024-04-02T14:42:58.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.691+0000 7f6b8e600700 1 -- 172.21.15.62:0/2199593472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8819e160 con 0x7f6b8806b4f0 2024-04-02T14:42:58.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.691+0000 7f6b7fe00700 1 -- 172.21.15.62:0/2199593472 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6b7400ee90 con 0x7f6b8806b4f0 2024-04-02T14:42:58.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.691+0000 7f6b7fe00700 1 -- 172.21.15.62:0/2199593472 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74010430 con 0x7f6b8806b4f0 2024-04-02T14:42:58.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.692+0000 7f6b8e600700 1 -- 172.21.15.62:0/2199593472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b881a59e0 con 0x7f6b8806b4f0 2024-04-02T14:42:58.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.693+0000 7f6b8e600700 1 -- 172.21.15.62:0/2199593472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b880692b0 con 0x7f6b8806b4f0 2024-04-02T14:42:58.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.695+0000 7f6b7fe00700 1 -- 172.21.15.62:0/2199593472 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6b74003e90 con 0x7f6b8806b4f0 2024-04-02T14:42:58.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.696+0000 7f6b7fe00700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6b6c0a3c40 0x7f6b6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:42:58.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.696+0000 7f6b7fe00700 1 -- 172.21.15.62:0/2199593472 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f6b740d05e0 con 0x7f6b8806b4f0 2024-04-02T14:42:58.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.696+0000 7f6b87400700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6b6c0a3c40 0x7f6b6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:42:58.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.697+0000 7f6b87400700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6b6c0a3c40 0x7f6b6c0a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800b2e0 tx=0x7f6b78005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:42:58.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.700+0000 7f6b7fe00700 1 -- 172.21.15.62:0/2199593472 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6b7409cf90 con 0x7f6b8806b4f0 2024-04-02T14:42:58.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.929+0000 7f6b8e600700 1 -- 172.21.15.62:0/2199593472 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6b88062a30 con 0x7f6b6c0a3c40 2024-04-02T14:42:58.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.953+0000 7f6b7fe00700 1 -- 172.21.15.62:0/2199593472 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258940 (secure 0 0 0) 0x7f6b78067390 con 0x7f6b6c0a3c40 2024-04-02T14:42:58.958 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:42:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.959+0000 7f6b8e600700 1 -- 172.21.15.62:0/2199593472 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6b6c0a3c40 msgr2=0x7f6b6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.959+0000 7f6b8e600700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6b6c0a3c40 0x7f6b6c0a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800b2e0 tx=0x7f6b78005fd0 comp rx=0 tx=0).stop 2024-04-02T14:42:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.959+0000 7f6b8e600700 1 -- 172.21.15.62:0/2199593472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8806b4f0 msgr2=0x7f6b881993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:42:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.959+0000 7f6b8e600700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8806b4f0 0x7f6b881993f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400e9f0 tx=0x7f6b7400ed00 comp rx=0 tx=0).stop 2024-04-02T14:42:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.959+0000 7f6b8e600700 1 -- 172.21.15.62:0/2199593472 shutdown_connections 2024-04-02T14:42:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.959+0000 7f6b8e600700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6b6c0a3c40 0x7f6b6c0a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.959+0000 7f6b8e600700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b88108630 0x7f6b8819d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.959+0000 7f6b8e600700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b8806ab30 0x7f6b88198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:58.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.960+0000 7f6b8e600700 1 --2- 172.21.15.62:0/2199593472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8806b4f0 0x7f6b881993f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:42:58.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.960+0000 7f6b8e600700 1 -- 172.21.15.62:0/2199593472 >> 172.21.15.62:0/2199593472 conn(0x7f6b88077540 msgr2=0x7f6b881001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:42:58.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.960+0000 7f6b8e600700 1 -- 172.21.15.62:0/2199593472 shutdown_connections 2024-04-02T14:42:58.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:42:58.960+0000 7f6b8e600700 1 -- 172.21.15.62:0/2199593472 wait complete. 2024-04-02T14:42:58.962 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-02T14:42:59.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:42:59 smithi062 ceph-mon[32116]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T14:42:59.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:42:59 smithi113 ceph-mon[36135]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T14:42:59.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:42:59 smithi204 ceph-mon[35948]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T14:42:59.933 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-04-02T14:42:57.393637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656132,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1018,"num_read_kb":635,"num_write":426,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181749,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":849,"ondisk_log_size":849,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404396,"kb_used_data":13644,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537428,"statfs":{"total":1151940427776,"available":1151526326272,"internally_reserved":0,"allocated":13971456,"data_stored":7584384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":52,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005098"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372298+0000","last_change":"2024-04-02T14:41:22.546468+0000","last_active":"2024-04-02T14:42:49.372298+0000","last_peered":"2024-04-02T14:42:49.372298+0000","last_clean":"2024-04-02T14:42:49.372298+0000","last_became_active":"2024-04-02T14:41:22.545991+0000","last_became_peered":"2024-04-02T14:41:22.545991+0000","last_unstale":"2024-04-02T14:42:49.372298+0000","last_undegraded":"2024-04-02T14:42:49.372298+0000","last_fullsized":"2024-04-02T14:42:49.372298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:34:24.520404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.359664+0000","last_change":"2024-04-02T14:41:22.547738+0000","last_active":"2024-04-02T14:42:49.359664+0000","last_peered":"2024-04-02T14:42:49.359664+0000","last_clean":"2024-04-02T14:42:49.359664+0000","last_became_active":"2024-04-02T14:41:22.542372+0000","last_became_peered":"2024-04-02T14:41:22.542372+0000","last_unstale":"2024-04-02T14:42:49.359664+0000","last_undegraded":"2024-04-02T14:42:49.359664+0000","last_fullsized":"2024-04-02T14:42:49.359664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:07:26.951136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.365915+0000","last_change":"2024-04-02T14:41:22.540173+0000","last_active":"2024-04-02T14:42:49.365915+0000","last_peered":"2024-04-02T14:42:49.365915+0000","last_clean":"2024-04-02T14:42:49.365915+0000","last_became_active":"2024-04-02T14:41:22.536279+0000","last_became_peered":"2024-04-02T14:41:22.536279+0000","last_unstale":"2024-04-02T14:42:49.365915+0000","last_undegraded":"2024-04-02T14:42:49.365915+0000","last_fullsized":"2024-04-02T14:42:49.365915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:43:40.826934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.363245+0000","last_change":"2024-04-02T14:42:21.251675+0000","last_active":"2024-04-02T14:42:49.363245+0000","last_peered":"2024-04-02T14:42:49.363245+0000","last_clean":"2024-04-02T14:42:49.363245+0000","last_became_active":"2024-04-02T14:42:21.250446+0000","last_became_peered":"2024-04-02T14:42:21.250446+0000","last_unstale":"2024-04-02T14:42:49.363245+0000","last_undegraded":"2024-04-02T14:42:49.363245+0000","last_fullsized":"2024-04-02T14:42:49.363245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:11.005276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370555+0000","last_change":"2024-04-02T14:41:22.554301+0000","last_active":"2024-04-02T14:42:49.370555+0000","last_peered":"2024-04-02T14:42:49.370555+0000","last_clean":"2024-04-02T14:42:49.370555+0000","last_became_active":"2024-04-02T14:41:22.554028+0000","last_became_peered":"2024-04-02T14:41:22.554028+0000","last_unstale":"2024-04-02T14:42:49.370555+0000","last_undegraded":"2024-04-02T14:42:49.370555+0000","last_fullsized":"2024-04-02T14:42:49.370555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:51:59.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.359271+0000","last_change":"2024-04-02T14:41:22.543934+0000","last_active":"2024-04-02T14:42:49.359271+0000","last_peered":"2024-04-02T14:42:49.359271+0000","last_clean":"2024-04-02T14:42:49.359271+0000","last_became_active":"2024-04-02T14:41:22.543392+0000","last_became_peered":"2024-04-02T14:41:22.543392+0000","last_unstale":"2024-04-02T14:42:49.359271+0000","last_undegraded":"2024-04-02T14:42:49.359271+0000","last_fullsized":"2024-04-02T14:42:49.359271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:28:30.234356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.362683+0000","last_change":"2024-04-02T14:41:22.547460+0000","last_active":"2024-04-02T14:42:49.362683+0000","last_peered":"2024-04-02T14:42:49.362683+0000","last_clean":"2024-04-02T14:42:49.362683+0000","last_became_active":"2024-04-02T14:41:22.547123+0000","last_became_peered":"2024-04-02T14:41:22.547123+0000","last_unstale":"2024-04-02T14:42:49.362683+0000","last_undegraded":"2024-04-02T14:42:49.362683+0000","last_fullsized":"2024-04-02T14:42:49.362683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:24:34.302530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372164+0000","last_change":"2024-04-02T14:41:22.551407+0000","last_active":"2024-04-02T14:42:49.372164+0000","last_peered":"2024-04-02T14:42:49.372164+0000","last_clean":"2024-04-02T14:42:49.372164+0000","last_became_active":"2024-04-02T14:41:22.538304+0000","last_became_peered":"2024-04-02T14:41:22.538304+0000","last_unstale":"2024-04-02T14:42:49.372164+0000","last_undegraded":"2024-04-02T14:42:49.372164+0000","last_fullsized":"2024-04-02T14:42:49.372164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:27.933380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.361684+0000","last_change":"2024-04-02T14:41:22.523931+0000","last_active":"2024-04-02T14:42:49.361684+0000","last_peered":"2024-04-02T14:42:49.361684+0000","last_clean":"2024-04-02T14:42:49.361684+0000","last_became_active":"2024-04-02T14:41:22.523622+0000","last_became_peered":"2024-04-02T14:41:22.523622+0000","last_unstale":"2024-04-02T14:42:49.361684+0000","last_undegraded":"2024-04-02T14:42:49.361684+0000","last_fullsized":"2024-04-02T14:42:49.361684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:17.808290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.353241+0000","last_change":"2024-04-02T14:41:22.525791+0000","last_active":"2024-04-02T14:42:49.353241+0000","last_peered":"2024-04-02T14:42:49.353241+0000","last_clean":"2024-04-02T14:42:49.353241+0000","last_became_active":"2024-04-02T14:41:22.525434+0000","last_became_peered":"2024-04-02T14:41:22.525434+0000","last_unstale":"2024-04-02T14:42:49.353241+0000","last_undegraded":"2024-04-02T14:42:49.353241+0000","last_fullsized":"2024-04-02T14:42:49.353241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:59.905337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.357047+0000","last_change":"2024-04-02T14:41:22.527240+0000","last_active":"2024-04-02T14:42:49.357047+0000","last_peered":"2024-04-02T14:42:49.357047+0000","last_clean":"2024-04-02T14:42:49.357047+0000","last_became_active":"2024-04-02T14:41:22.526412+0000","last_became_peered":"2024-04-02T14:41:22.526412+0000","last_unstale":"2024-04-02T14:42:49.357047+0000","last_undegraded":"2024-04-02T14:42:49.357047+0000","last_fullsized":"2024-04-02T14:42:49.357047+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:47.397012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.346146+0000","last_change":"2024-04-02T14:41:22.527281+0000","last_active":"2024-04-02T14:42:49.346146+0000","last_peered":"2024-04-02T14:42:49.346146+0000","last_clean":"2024-04-02T14:42:49.346146+0000","last_became_active":"2024-04-02T14:41:22.526946+0000","last_became_peered":"2024-04-02T14:41:22.526946+0000","last_unstale":"2024-04-02T14:42:49.346146+0000","last_undegraded":"2024-04-02T14:42:49.346146+0000","last_fullsized":"2024-04-02T14:42:49.346146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:42.798282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372127+0000","last_change":"2024-04-02T14:41:22.534079+0000","last_active":"2024-04-02T14:42:49.372127+0000","last_peered":"2024-04-02T14:42:49.372127+0000","last_clean":"2024-04-02T14:42:49.372127+0000","last_became_active":"2024-04-02T14:41:22.532233+0000","last_became_peered":"2024-04-02T14:41:22.532233+0000","last_unstale":"2024-04-02T14:42:49.372127+0000","last_undegraded":"2024-04-02T14:42:49.372127+0000","last_fullsized":"2024-04-02T14:42:49.372127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:27:47.122933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.362395+0000","last_change":"2024-04-02T14:41:22.544936+0000","last_active":"2024-04-02T14:42:49.362395+0000","last_peered":"2024-04-02T14:42:49.362395+0000","last_clean":"2024-04-02T14:42:49.362395+0000","last_became_active":"2024-04-02T14:41:22.541805+0000","last_became_peered":"2024-04-02T14:41:22.541805+0000","last_unstale":"2024-04-02T14:42:49.362395+0000","last_undegraded":"2024-04-02T14:42:49.362395+0000","last_fullsized":"2024-04-02T14:42:49.362395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:23.992807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.361980+0000","last_change":"2024-04-02T14:41:22.534860+0000","last_active":"2024-04-02T14:42:49.361980+0000","last_peered":"2024-04-02T14:42:49.361980+0000","last_clean":"2024-04-02T14:42:49.361980+0000","last_became_active":"2024-04-02T14:41:22.534147+0000","last_became_peered":"2024-04-02T14:41:22.534147+0000","last_unstale":"2024-04-02T14:42:49.361980+0000","last_undegraded":"2024-04-02T14:42:49.361980+0000","last_fullsized":"2024-04-02T14:42:49.361980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:25.249364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.375984+0000","last_change":"2024-04-02T14:41:22.537654+0000","last_active":"2024-04-02T14:42:49.375984+0000","last_peered":"2024-04-02T14:42:49.375984+0000","last_clean":"2024-04-02T14:42:49.375984+0000","last_became_active":"2024-04-02T14:41:22.537304+0000","last_became_peered":"2024-04-02T14:41:22.537304+0000","last_unstale":"2024-04-02T14:42:49.375984+0000","last_undegraded":"2024-04-02T14:42:49.375984+0000","last_fullsized":"2024-04-02T14:42:49.375984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:25.599724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372388+0000","last_change":"2024-04-02T14:41:22.546775+0000","last_active":"2024-04-02T14:42:49.372388+0000","last_peered":"2024-04-02T14:42:49.372388+0000","last_clean":"2024-04-02T14:42:49.372388+0000","last_became_active":"2024-04-02T14:41:22.546100+0000","last_became_peered":"2024-04-02T14:41:22.546100+0000","last_unstale":"2024-04-02T14:42:49.372388+0000","last_undegraded":"2024-04-02T14:42:49.372388+0000","last_fullsized":"2024-04-02T14:42:49.372388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:42:02.559335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.364567+0000","last_change":"2024-04-02T14:41:22.543988+0000","last_active":"2024-04-02T14:42:49.364567+0000","last_peered":"2024-04-02T14:42:49.364567+0000","last_clean":"2024-04-02T14:42:49.364567+0000","last_became_active":"2024-04-02T14:41:22.543036+0000","last_became_peered":"2024-04-02T14:41:22.543036+0000","last_unstale":"2024-04-02T14:42:49.364567+0000","last_undegraded":"2024-04-02T14:42:49.364567+0000","last_fullsized":"2024-04-02T14:42:49.364567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:29.470929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.354246+0000","last_change":"2024-04-02T14:41:22.543954+0000","last_active":"2024-04-02T14:42:49.354246+0000","last_peered":"2024-04-02T14:42:49.354246+0000","last_clean":"2024-04-02T14:42:49.354246+0000","last_became_active":"2024-04-02T14:41:22.543624+0000","last_became_peered":"2024-04-02T14:41:22.543624+0000","last_unstale":"2024-04-02T14:42:49.354246+0000","last_undegraded":"2024-04-02T14:42:49.354246+0000","last_fullsized":"2024-04-02T14:42:49.354246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:46.933877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:53.271076+0000","last_change":"2024-04-02T14:41:21.477341+0000","last_active":"2024-04-02T14:42:53.271076+0000","last_peered":"2024-04-02T14:42:53.271076+0000","last_clean":"2024-04-02T14:42:53.271076+0000","last_became_active":"2024-04-02T14:41:21.476708+0000","last_became_peered":"2024-04-02T14:41:21.476708+0000","last_unstale":"2024-04-02T14:42:53.271076+0000","last_undegraded":"2024-04-02T14:42:53.271076+0000","last_fullsized":"2024-04-02T14:42:53.271076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:34.867570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.355502+0000","last_change":"2024-04-02T14:41:22.551293+0000","last_active":"2024-04-02T14:42:49.355502+0000","last_peered":"2024-04-02T14:42:49.355502+0000","last_clean":"2024-04-02T14:42:49.355502+0000","last_became_active":"2024-04-02T14:41:22.547819+0000","last_became_peered":"2024-04-02T14:41:22.547819+0000","last_unstale":"2024-04-02T14:42:49.355502+0000","last_undegraded":"2024-04-02T14:42:49.355502+0000","last_fullsized":"2024-04-02T14:42:49.355502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:40.393005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.239792+0000","last_change":"2024-04-02T14:41:21.487096+0000","last_active":"2024-04-02T14:42:21.239792+0000","last_peered":"2024-04-02T14:42:21.239792+0000","last_clean":"2024-04-02T14:42:21.239792+0000","last_became_active":"2024-04-02T14:41:21.485883+0000","last_became_peered":"2024-04-02T14:41:21.485883+0000","last_unstale":"2024-04-02T14:42:21.239792+0000","last_undegraded":"2024-04-02T14:42:21.239792+0000","last_fullsized":"2024-04-02T14:42:21.239792+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:24.017671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.371157+0000","last_change":"2024-04-02T14:41:22.549234+0000","last_active":"2024-04-02T14:42:49.371157+0000","last_peered":"2024-04-02T14:42:49.371157+0000","last_clean":"2024-04-02T14:42:49.371157+0000","last_became_active":"2024-04-02T14:41:22.548855+0000","last_became_peered":"2024-04-02T14:41:22.548855+0000","last_unstale":"2024-04-02T14:42:49.371157+0000","last_undegraded":"2024-04-02T14:42:49.371157+0000","last_fullsized":"2024-04-02T14:42:49.371157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:05.047383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.245395+0000","last_change":"2024-04-02T14:41:21.471673+0000","last_active":"2024-04-02T14:42:21.245395+0000","last_peered":"2024-04-02T14:42:21.245395+0000","last_clean":"2024-04-02T14:42:21.245395+0000","last_became_active":"2024-04-02T14:41:21.471292+0000","last_became_peered":"2024-04-02T14:41:21.471292+0000","last_unstale":"2024-04-02T14:42:21.245395+0000","last_undegraded":"2024-04-02T14:42:21.245395+0000","last_fullsized":"2024-04-02T14:42:21.245395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:42:55.665938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.373736+0000","last_change":"2024-04-02T14:41:22.537665+0000","last_active":"2024-04-02T14:42:49.373736+0000","last_peered":"2024-04-02T14:42:49.373736+0000","last_clean":"2024-04-02T14:42:49.373736+0000","last_became_active":"2024-04-02T14:41:22.537274+0000","last_became_peered":"2024-04-02T14:41:22.537274+0000","last_unstale":"2024-04-02T14:42:49.373736+0000","last_undegraded":"2024-04-02T14:42:49.373736+0000","last_fullsized":"2024-04-02T14:42:49.373736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:14:39.200735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.246137+0000","last_change":"2024-04-02T14:41:21.466547+0000","last_active":"2024-04-02T14:42:21.246137+0000","last_peered":"2024-04-02T14:42:21.246137+0000","last_clean":"2024-04-02T14:42:21.246137+0000","last_became_active":"2024-04-02T14:41:21.466063+0000","last_became_peered":"2024-04-02T14:41:21.466063+0000","last_unstale":"2024-04-02T14:42:21.246137+0000","last_undegraded":"2024-04-02T14:42:21.246137+0000","last_fullsized":"2024-04-02T14:42:21.246137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:39.201543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.351340+0000","last_change":"2024-04-02T14:41:22.523984+0000","last_active":"2024-04-02T14:42:49.351340+0000","last_peered":"2024-04-02T14:42:49.351340+0000","last_clean":"2024-04-02T14:42:49.351340+0000","last_became_active":"2024-04-02T14:41:22.523524+0000","last_became_peered":"2024-04-02T14:41:22.523524+0000","last_unstale":"2024-04-02T14:42:49.351340+0000","last_undegraded":"2024-04-02T14:42:49.351340+0000","last_fullsized":"2024-04-02T14:42:49.351340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:31.542588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":125,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:52.290278+0000","last_change":"2024-04-02T14:42:21.255204+0000","last_active":"2024-04-02T14:42:52.290278+0000","last_peered":"2024-04-02T14:42:52.290278+0000","last_clean":"2024-04-02T14:42:52.290278+0000","last_became_active":"2024-04-02T14:42:21.248702+0000","last_became_peered":"2024-04-02T14:42:21.248702+0000","last_unstale":"2024-04-02T14:42:52.290278+0000","last_undegraded":"2024-04-02T14:42:52.290278+0000","last_fullsized":"2024-04-02T14:42:52.290278+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.723036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.352203+0000","last_change":"2024-04-02T14:41:22.544107+0000","last_active":"2024-04-02T14:42:49.352203+0000","last_peered":"2024-04-02T14:42:49.352203+0000","last_clean":"2024-04-02T14:42:49.352203+0000","last_became_active":"2024-04-02T14:41:22.543739+0000","last_became_peered":"2024-04-02T14:41:22.543739+0000","last_unstale":"2024-04-02T14:42:49.352203+0000","last_undegraded":"2024-04-02T14:42:49.352203+0000","last_fullsized":"2024-04-02T14:42:49.352203+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:49:08.921668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.236590+0000","last_change":"2024-04-02T14:41:21.479714+0000","last_active":"2024-04-02T14:42:21.236590+0000","last_peered":"2024-04-02T14:42:21.236590+0000","last_clean":"2024-04-02T14:42:21.236590+0000","last_became_active":"2024-04-02T14:41:21.479329+0000","last_became_peered":"2024-04-02T14:41:21.479329+0000","last_unstale":"2024-04-02T14:42:21.236590+0000","last_undegraded":"2024-04-02T14:42:21.236590+0000","last_fullsized":"2024-04-02T14:42:21.236590+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:09.021017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.373049+0000","last_change":"2024-04-02T14:42:21.254332+0000","last_active":"2024-04-02T14:42:49.373049+0000","last_peered":"2024-04-02T14:42:49.373049+0000","last_clean":"2024-04-02T14:42:49.373049+0000","last_became_active":"2024-04-02T14:42:21.251266+0000","last_became_peered":"2024-04-02T14:42:21.251266+0000","last_unstale":"2024-04-02T14:42:49.373049+0000","last_undegraded":"2024-04-02T14:42:49.373049+0000","last_fullsized":"2024-04-02T14:42:49.373049+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:51:18.343098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.582968+0000","last_change":"2024-04-02T14:41:21.484054+0000","last_active":"2024-04-02T14:42:21.582968+0000","last_peered":"2024-04-02T14:42:21.582968+0000","last_clean":"2024-04-02T14:42:21.582968+0000","last_became_active":"2024-04-02T14:41:21.483688+0000","last_became_peered":"2024-04-02T14:41:21.483688+0000","last_unstale":"2024-04-02T14:42:21.582968+0000","last_undegraded":"2024-04-02T14:42:21.582968+0000","last_fullsized":"2024-04-02T14:42:21.582968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:24.114180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.378217+0000","last_change":"2024-04-02T14:41:22.553274+0000","last_active":"2024-04-02T14:42:49.378217+0000","last_peered":"2024-04-02T14:42:49.378217+0000","last_clean":"2024-04-02T14:42:49.378217+0000","last_became_active":"2024-04-02T14:41:22.545412+0000","last_became_peered":"2024-04-02T14:41:22.545412+0000","last_unstale":"2024-04-02T14:42:49.378217+0000","last_undegraded":"2024-04-02T14:42:49.378217+0000","last_fullsized":"2024-04-02T14:42:49.378217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:21.917119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.242398+0000","last_change":"2024-04-02T14:41:21.468582+0000","last_active":"2024-04-02T14:42:21.242398+0000","last_peered":"2024-04-02T14:42:21.242398+0000","last_clean":"2024-04-02T14:42:21.242398+0000","last_became_active":"2024-04-02T14:41:21.468081+0000","last_became_peered":"2024-04-02T14:41:21.468081+0000","last_unstale":"2024-04-02T14:42:21.242398+0000","last_undegraded":"2024-04-02T14:42:21.242398+0000","last_fullsized":"2024-04-02T14:42:21.242398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:19.593081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.374638+0000","last_change":"2024-04-02T14:41:22.555627+0000","last_active":"2024-04-02T14:42:49.374638+0000","last_peered":"2024-04-02T14:42:49.374638+0000","last_clean":"2024-04-02T14:42:49.374638+0000","last_became_active":"2024-04-02T14:41:22.555171+0000","last_became_peered":"2024-04-02T14:41:22.555171+0000","last_unstale":"2024-04-02T14:42:49.374638+0000","last_undegraded":"2024-04-02T14:42:49.374638+0000","last_fullsized":"2024-04-02T14:42:49.374638+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:21:28.161869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":194,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:54.282146+0000","last_change":"2024-04-02T14:41:21.475004+0000","last_active":"2024-04-02T14:42:54.282146+0000","last_peered":"2024-04-02T14:42:54.282146+0000","last_clean":"2024-04-02T14:42:54.282146+0000","last_became_active":"2024-04-02T14:41:21.474621+0000","last_became_peered":"2024-04-02T14:41:21.474621+0000","last_unstale":"2024-04-02T14:42:54.282146+0000","last_undegraded":"2024-04-02T14:42:54.282146+0000","last_fullsized":"2024-04-02T14:42:54.282146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:38.779276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.365367+0000","last_change":"2024-04-02T14:41:22.543100+0000","last_active":"2024-04-02T14:42:49.365367+0000","last_peered":"2024-04-02T14:42:49.365367+0000","last_clean":"2024-04-02T14:42:49.365367+0000","last_became_active":"2024-04-02T14:41:22.542697+0000","last_became_peered":"2024-04-02T14:41:22.542697+0000","last_unstale":"2024-04-02T14:42:49.365367+0000","last_undegraded":"2024-04-02T14:42:49.365367+0000","last_fullsized":"2024-04-02T14:42:49.365367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:50:52.412879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.245264+0000","last_change":"2024-04-02T14:41:21.468626+0000","last_active":"2024-04-02T14:42:21.245264+0000","last_peered":"2024-04-02T14:42:21.245264+0000","last_clean":"2024-04-02T14:42:21.245264+0000","last_became_active":"2024-04-02T14:41:21.468041+0000","last_became_peered":"2024-04-02T14:41:21.468041+0000","last_unstale":"2024-04-02T14:42:21.245264+0000","last_undegraded":"2024-04-02T14:42:21.245264+0000","last_fullsized":"2024-04-02T14:42:21.245264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:55.064033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370998+0000","last_change":"2024-04-02T14:41:22.539584+0000","last_active":"2024-04-02T14:42:49.370998+0000","last_peered":"2024-04-02T14:42:49.370998+0000","last_clean":"2024-04-02T14:42:49.370998+0000","last_became_active":"2024-04-02T14:41:22.539218+0000","last_became_peered":"2024-04-02T14:41:22.539218+0000","last_unstale":"2024-04-02T14:42:49.370998+0000","last_undegraded":"2024-04-02T14:42:49.370998+0000","last_fullsized":"2024-04-02T14:42:49.370998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:47:56.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.243164+0000","last_change":"2024-04-02T14:41:21.481208+0000","last_active":"2024-04-02T14:42:21.243164+0000","last_peered":"2024-04-02T14:42:21.243164+0000","last_clean":"2024-04-02T14:42:21.243164+0000","last_became_active":"2024-04-02T14:41:21.480782+0000","last_became_peered":"2024-04-02T14:41:21.480782+0000","last_unstale":"2024-04-02T14:42:21.243164+0000","last_undegraded":"2024-04-02T14:42:21.243164+0000","last_fullsized":"2024-04-02T14:42:21.243164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:12.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.363872+0000","last_change":"2024-04-02T14:41:22.517397+0000","last_active":"2024-04-02T14:42:49.363872+0000","last_peered":"2024-04-02T14:42:49.363872+0000","last_clean":"2024-04-02T14:42:49.363872+0000","last_became_active":"2024-04-02T14:41:22.517027+0000","last_became_peered":"2024-04-02T14:41:22.517027+0000","last_unstale":"2024-04-02T14:42:49.363872+0000","last_undegraded":"2024-04-02T14:42:49.363872+0000","last_fullsized":"2024-04-02T14:42:49.363872+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:53.129949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":227,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:54.292811+0000","last_change":"2024-04-02T14:41:21.469643+0000","last_active":"2024-04-02T14:42:54.292811+0000","last_peered":"2024-04-02T14:42:54.292811+0000","last_clean":"2024-04-02T14:42:54.292811+0000","last_became_active":"2024-04-02T14:41:21.468730+0000","last_became_peered":"2024-04-02T14:41:21.468730+0000","last_unstale":"2024-04-02T14:42:54.292811+0000","last_undegraded":"2024-04-02T14:42:54.292811+0000","last_fullsized":"2024-04-02T14:42:54.292811+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:10.983551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58576,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":177,"num_read_kb":147,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.344508+0000","last_change":"2024-04-02T14:41:22.525052+0000","last_active":"2024-04-02T14:42:49.344508+0000","last_peered":"2024-04-02T14:42:49.344508+0000","last_clean":"2024-04-02T14:42:49.344508+0000","last_became_active":"2024-04-02T14:41:22.524691+0000","last_became_peered":"2024-04-02T14:41:22.524691+0000","last_unstale":"2024-04-02T14:42:49.344508+0000","last_undegraded":"2024-04-02T14:42:49.344508+0000","last_fullsized":"2024-04-02T14:42:49.344508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:16.386020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.246975+0000","last_change":"2024-04-02T14:41:21.476988+0000","last_active":"2024-04-02T14:42:21.246975+0000","last_peered":"2024-04-02T14:42:21.246975+0000","last_clean":"2024-04-02T14:42:21.246975+0000","last_became_active":"2024-04-02T14:41:21.476549+0000","last_became_peered":"2024-04-02T14:41:21.476549+0000","last_unstale":"2024-04-02T14:42:21.246975+0000","last_undegraded":"2024-04-02T14:42:21.246975+0000","last_fullsized":"2024-04-02T14:42:21.246975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:47.498626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.376278+0000","last_change":"2024-04-02T14:41:22.544746+0000","last_active":"2024-04-02T14:42:49.376278+0000","last_peered":"2024-04-02T14:42:49.376278+0000","last_clean":"2024-04-02T14:42:49.376278+0000","last_became_active":"2024-04-02T14:41:22.544334+0000","last_became_peered":"2024-04-02T14:41:22.544334+0000","last_unstale":"2024-04-02T14:42:49.376278+0000","last_undegraded":"2024-04-02T14:42:49.376278+0000","last_fullsized":"2024-04-02T14:42:49.376278+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:45:20.094455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.243078+0000","last_change":"2024-04-02T14:41:21.472458+0000","last_active":"2024-04-02T14:42:21.243078+0000","last_peered":"2024-04-02T14:42:21.243078+0000","last_clean":"2024-04-02T14:42:21.243078+0000","last_became_active":"2024-04-02T14:41:21.472105+0000","last_became_peered":"2024-04-02T14:41:21.472105+0000","last_unstale":"2024-04-02T14:42:21.243078+0000","last_undegraded":"2024-04-02T14:42:21.243078+0000","last_fullsized":"2024-04-02T14:42:21.243078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:44.075807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372497+0000","last_change":"2024-04-02T14:41:22.537492+0000","last_active":"2024-04-02T14:42:49.372497+0000","last_peered":"2024-04-02T14:42:49.372497+0000","last_clean":"2024-04-02T14:42:49.372497+0000","last_became_active":"2024-04-02T14:41:22.537067+0000","last_became_peered":"2024-04-02T14:41:22.537067+0000","last_unstale":"2024-04-02T14:42:49.372497+0000","last_undegraded":"2024-04-02T14:42:49.372497+0000","last_fullsized":"2024-04-02T14:42:49.372497+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:08.704340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":110,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:52.277928+0000","last_change":"2024-04-02T14:41:21.466017+0000","last_active":"2024-04-02T14:42:52.277928+0000","last_peered":"2024-04-02T14:42:52.277928+0000","last_clean":"2024-04-02T14:42:52.277928+0000","last_became_active":"2024-04-02T14:41:21.465600+0000","last_became_peered":"2024-04-02T14:41:21.465600+0000","last_unstale":"2024-04-02T14:42:52.277928+0000","last_undegraded":"2024-04-02T14:42:52.277928+0000","last_fullsized":"2024-04-02T14:42:52.277928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:52:08.131125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.354336+0000","last_change":"2024-04-02T14:41:22.526628+0000","last_active":"2024-04-02T14:42:49.354336+0000","last_peered":"2024-04-02T14:42:49.354336+0000","last_clean":"2024-04-02T14:42:49.354336+0000","last_became_active":"2024-04-02T14:41:22.526283+0000","last_became_peered":"2024-04-02T14:41:22.526283+0000","last_unstale":"2024-04-02T14:42:49.354336+0000","last_undegraded":"2024-04-02T14:42:49.354336+0000","last_fullsized":"2024-04-02T14:42:49.354336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:41.812330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.244931+0000","last_change":"2024-04-02T14:41:21.475853+0000","last_active":"2024-04-02T14:42:21.244931+0000","last_peered":"2024-04-02T14:42:21.244931+0000","last_clean":"2024-04-02T14:42:21.244931+0000","last_became_active":"2024-04-02T14:41:21.475555+0000","last_became_peered":"2024-04-02T14:41:21.475555+0000","last_unstale":"2024-04-02T14:42:21.244931+0000","last_undegraded":"2024-04-02T14:42:21.244931+0000","last_fullsized":"2024-04-02T14:42:21.244931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:49.170906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.350247+0000","last_change":"2024-04-02T14:41:22.523848+0000","last_active":"2024-04-02T14:42:49.350247+0000","last_peered":"2024-04-02T14:42:49.350247+0000","last_clean":"2024-04-02T14:42:49.350247+0000","last_became_active":"2024-04-02T14:41:22.523476+0000","last_became_peered":"2024-04-02T14:41:22.523476+0000","last_unstale":"2024-04-02T14:42:49.350247+0000","last_undegraded":"2024-04-02T14:42:49.350247+0000","last_fullsized":"2024-04-02T14:42:49.350247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:47:46.933264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.246237+0000","last_change":"2024-04-02T14:41:21.468556+0000","last_active":"2024-04-02T14:42:21.246237+0000","last_peered":"2024-04-02T14:42:21.246237+0000","last_clean":"2024-04-02T14:42:21.246237+0000","last_became_active":"2024-04-02T14:41:21.468215+0000","last_became_peered":"2024-04-02T14:41:21.468215+0000","last_unstale":"2024-04-02T14:42:21.246237+0000","last_undegraded":"2024-04-02T14:42:21.246237+0000","last_fullsized":"2024-04-02T14:42:21.246237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:04.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.344760+0000","last_change":"2024-04-02T14:41:22.524228+0000","last_active":"2024-04-02T14:42:49.344760+0000","last_peered":"2024-04-02T14:42:49.344760+0000","last_clean":"2024-04-02T14:42:49.344760+0000","last_became_active":"2024-04-02T14:41:22.523839+0000","last_became_peered":"2024-04-02T14:41:22.523839+0000","last_unstale":"2024-04-02T14:42:49.344760+0000","last_undegraded":"2024-04-02T14:42:49.344760+0000","last_fullsized":"2024-04-02T14:42:49.344760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:13.094123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.260431+0000","last_change":"2024-04-02T14:42:21.260431+0000","last_active":"2024-04-02T14:42:21.260431+0000","last_peered":"2024-04-02T14:42:21.260431+0000","last_clean":"2024-04-02T14:42:21.260431+0000","last_became_active":"2024-04-02T14:42:21.259060+0000","last_became_peered":"2024-04-02T14:42:21.259060+0000","last_unstale":"2024-04-02T14:42:21.260431+0000","last_undegraded":"2024-04-02T14:42:21.260431+0000","last_fullsized":"2024-04-02T14:42:21.260431+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:07.753909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370091+0000","last_change":"2024-04-02T14:41:22.542847+0000","last_active":"2024-04-02T14:42:49.370091+0000","last_peered":"2024-04-02T14:42:49.370091+0000","last_clean":"2024-04-02T14:42:49.370091+0000","last_became_active":"2024-04-02T14:41:22.542476+0000","last_became_peered":"2024-04-02T14:41:22.542476+0000","last_unstale":"2024-04-02T14:42:49.370091+0000","last_undegraded":"2024-04-02T14:42:49.370091+0000","last_fullsized":"2024-04-02T14:42:49.370091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:27:12.102931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":110,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:53.271956+0000","last_change":"2024-04-02T14:41:21.483382+0000","last_active":"2024-04-02T14:42:53.271956+0000","last_peered":"2024-04-02T14:42:53.271956+0000","last_clean":"2024-04-02T14:42:53.271956+0000","last_became_active":"2024-04-02T14:41:21.482996+0000","last_became_peered":"2024-04-02T14:41:21.482996+0000","last_unstale":"2024-04-02T14:42:53.271956+0000","last_undegraded":"2024-04-02T14:42:53.271956+0000","last_fullsized":"2024-04-02T14:42:53.271956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:37.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.365155+0000","last_change":"2024-04-02T14:42:21.257441+0000","last_active":"2024-04-02T14:42:49.365155+0000","last_peered":"2024-04-02T14:42:49.365155+0000","last_clean":"2024-04-02T14:42:49.365155+0000","last_became_active":"2024-04-02T14:42:21.257003+0000","last_became_peered":"2024-04-02T14:42:21.257003+0000","last_unstale":"2024-04-02T14:42:49.365155+0000","last_undegraded":"2024-04-02T14:42:49.365155+0000","last_fullsized":"2024-04-02T14:42:49.365155+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:19:10.793720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.346266+0000","last_change":"2024-04-02T14:41:22.525853+0000","last_active":"2024-04-02T14:42:49.346266+0000","last_peered":"2024-04-02T14:42:49.346266+0000","last_clean":"2024-04-02T14:42:49.346266+0000","last_became_active":"2024-04-02T14:41:22.525553+0000","last_became_peered":"2024-04-02T14:41:22.525553+0000","last_unstale":"2024-04-02T14:42:49.346266+0000","last_undegraded":"2024-04-02T14:42:49.346266+0000","last_fullsized":"2024-04-02T14:42:49.346266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:25:30.340974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":110,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:53.279103+0000","last_change":"2024-04-02T14:41:21.490632+0000","last_active":"2024-04-02T14:42:53.279103+0000","last_peered":"2024-04-02T14:42:53.279103+0000","last_clean":"2024-04-02T14:42:53.279103+0000","last_became_active":"2024-04-02T14:41:21.490208+0000","last_became_peered":"2024-04-02T14:41:21.490208+0000","last_unstale":"2024-04-02T14:42:53.279103+0000","last_undegraded":"2024-04-02T14:42:53.279103+0000","last_fullsized":"2024-04-02T14:42:53.279103+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:18.566261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.373553+0000","last_change":"2024-04-02T14:41:22.544872+0000","last_active":"2024-04-02T14:42:49.373553+0000","last_peered":"2024-04-02T14:42:49.373553+0000","last_clean":"2024-04-02T14:42:49.373553+0000","last_became_active":"2024-04-02T14:41:22.544268+0000","last_became_peered":"2024-04-02T14:41:22.544268+0000","last_unstale":"2024-04-02T14:42:49.373553+0000","last_undegraded":"2024-04-02T14:42:49.373553+0000","last_fullsized":"2024-04-02T14:42:49.373553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:54.963084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.234967+0000","last_change":"2024-04-02T14:41:21.474411+0000","last_active":"2024-04-02T14:42:21.234967+0000","last_peered":"2024-04-02T14:42:21.234967+0000","last_clean":"2024-04-02T14:42:21.234967+0000","last_became_active":"2024-04-02T14:41:21.473950+0000","last_became_peered":"2024-04-02T14:41:21.473950+0000","last_unstale":"2024-04-02T14:42:21.234967+0000","last_undegraded":"2024-04-02T14:42:21.234967+0000","last_fullsized":"2024-04-02T14:42:21.234967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:46:05.892446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.245632+0000","last_change":"2024-04-02T14:40:03.537813+0000","last_active":"2024-04-02T14:42:21.245632+0000","last_peered":"2024-04-02T14:42:21.245632+0000","last_clean":"2024-04-02T14:42:21.245632+0000","last_became_active":"2024-04-02T14:40:03.537223+0000","last_became_peered":"2024-04-02T14:40:03.537223+0000","last_unstale":"2024-04-02T14:42:21.245632+0000","last_undegraded":"2024-04-02T14:42:21.245632+0000","last_fullsized":"2024-04-02T14:42:21.245632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_clean_scrub_stamp":"2024-04-02T14:36:11.612815+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:19:10.346096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.363055+0000","last_change":"2024-04-02T14:41:22.537906+0000","last_active":"2024-04-02T14:42:49.363055+0000","last_peered":"2024-04-02T14:42:49.363055+0000","last_clean":"2024-04-02T14:42:49.363055+0000","last_became_active":"2024-04-02T14:41:22.537238+0000","last_became_peered":"2024-04-02T14:41:22.537238+0000","last_unstale":"2024-04-02T14:42:49.363055+0000","last_undegraded":"2024-04-02T14:42:49.363055+0000","last_fullsized":"2024-04-02T14:42:49.363055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:57.089563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.245673+0000","last_change":"2024-04-02T14:41:21.466830+0000","last_active":"2024-04-02T14:42:21.245673+0000","last_peered":"2024-04-02T14:42:21.245673+0000","last_clean":"2024-04-02T14:42:21.245673+0000","last_became_active":"2024-04-02T14:41:21.466490+0000","last_became_peered":"2024-04-02T14:41:21.466490+0000","last_unstale":"2024-04-02T14:42:21.245673+0000","last_undegraded":"2024-04-02T14:42:21.245673+0000","last_fullsized":"2024-04-02T14:42:21.245673+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:15.597854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.245278+0000","last_change":"2024-04-02T14:41:21.481139+0000","last_active":"2024-04-02T14:42:21.245278+0000","last_peered":"2024-04-02T14:42:21.245278+0000","last_clean":"2024-04-02T14:42:21.245278+0000","last_became_active":"2024-04-02T14:41:21.480754+0000","last_became_peered":"2024-04-02T14:41:21.480754+0000","last_unstale":"2024-04-02T14:42:21.245278+0000","last_undegraded":"2024-04-02T14:42:21.245278+0000","last_fullsized":"2024-04-02T14:42:21.245278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:03.716223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.373455+0000","last_change":"2024-04-02T14:41:22.548483+0000","last_active":"2024-04-02T14:42:49.373455+0000","last_peered":"2024-04-02T14:42:49.373455+0000","last_clean":"2024-04-02T14:42:49.373455+0000","last_became_active":"2024-04-02T14:41:22.547463+0000","last_became_peered":"2024-04-02T14:41:22.547463+0000","last_unstale":"2024-04-02T14:42:49.373455+0000","last_undegraded":"2024-04-02T14:42:49.373455+0000","last_fullsized":"2024-04-02T14:42:49.373455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:33:58.076844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.244078+0000","last_change":"2024-04-02T14:41:21.491105+0000","last_active":"2024-04-02T14:42:21.244078+0000","last_peered":"2024-04-02T14:42:21.244078+0000","last_clean":"2024-04-02T14:42:21.244078+0000","last_became_active":"2024-04-02T14:41:21.490635+0000","last_became_peered":"2024-04-02T14:41:21.490635+0000","last_unstale":"2024-04-02T14:42:21.244078+0000","last_undegraded":"2024-04-02T14:42:21.244078+0000","last_fullsized":"2024-04-02T14:42:21.244078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:16.531064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.371753+0000","last_change":"2024-04-02T14:41:22.553497+0000","last_active":"2024-04-02T14:42:49.371753+0000","last_peered":"2024-04-02T14:42:49.371753+0000","last_clean":"2024-04-02T14:42:49.371753+0000","last_became_active":"2024-04-02T14:41:22.550474+0000","last_became_peered":"2024-04-02T14:41:22.550474+0000","last_unstale":"2024-04-02T14:42:49.371753+0000","last_undegraded":"2024-04-02T14:42:49.371753+0000","last_fullsized":"2024-04-02T14:42:49.371753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:52.212454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.235007+0000","last_change":"2024-04-02T14:41:21.479516+0000","last_active":"2024-04-02T14:42:21.235007+0000","last_peered":"2024-04-02T14:42:21.235007+0000","last_clean":"2024-04-02T14:42:21.235007+0000","last_became_active":"2024-04-02T14:41:21.478841+0000","last_became_peered":"2024-04-02T14:41:21.478841+0000","last_unstale":"2024-04-02T14:42:21.235007+0000","last_undegraded":"2024-04-02T14:42:21.235007+0000","last_fullsized":"2024-04-02T14:42:21.235007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:39.222316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.374983+0000","last_change":"2024-04-02T14:41:22.550095+0000","last_active":"2024-04-02T14:42:49.374983+0000","last_peered":"2024-04-02T14:42:49.374983+0000","last_clean":"2024-04-02T14:42:49.374983+0000","last_became_active":"2024-04-02T14:41:22.547438+0000","last_became_peered":"2024-04-02T14:41:22.547438+0000","last_unstale":"2024-04-02T14:42:49.374983+0000","last_undegraded":"2024-04-02T14:42:49.374983+0000","last_fullsized":"2024-04-02T14:42:49.374983+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:03:03.668289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.580733+0000","last_change":"2024-04-02T14:41:21.482926+0000","last_active":"2024-04-02T14:42:21.580733+0000","last_peered":"2024-04-02T14:42:21.580733+0000","last_clean":"2024-04-02T14:42:21.580733+0000","last_became_active":"2024-04-02T14:41:21.482517+0000","last_became_peered":"2024-04-02T14:41:21.482517+0000","last_unstale":"2024-04-02T14:42:21.580733+0000","last_undegraded":"2024-04-02T14:42:21.580733+0000","last_fullsized":"2024-04-02T14:42:21.580733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:43.052390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.317999+0000","last_change":"2024-04-02T14:41:22.527758+0000","last_active":"2024-04-02T14:42:49.317999+0000","last_peered":"2024-04-02T14:42:49.317999+0000","last_clean":"2024-04-02T14:42:49.317999+0000","last_became_active":"2024-04-02T14:41:22.526954+0000","last_became_peered":"2024-04-02T14:41:22.526954+0000","last_unstale":"2024-04-02T14:42:49.317999+0000","last_undegraded":"2024-04-02T14:42:49.317999+0000","last_fullsized":"2024-04-02T14:42:49.317999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:34.988661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.238683+0000","last_change":"2024-04-02T14:41:21.487534+0000","last_active":"2024-04-02T14:42:21.238683+0000","last_peered":"2024-04-02T14:42:21.238683+0000","last_clean":"2024-04-02T14:42:21.238683+0000","last_became_active":"2024-04-02T14:41:21.487168+0000","last_became_peered":"2024-04-02T14:41:21.487168+0000","last_unstale":"2024-04-02T14:42:21.238683+0000","last_undegraded":"2024-04-02T14:42:21.238683+0000","last_fullsized":"2024-04-02T14:42:21.238683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:03:20.636028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372613+0000","last_change":"2024-04-02T14:41:22.539246+0000","last_active":"2024-04-02T14:42:49.372613+0000","last_peered":"2024-04-02T14:42:49.372613+0000","last_clean":"2024-04-02T14:42:49.372613+0000","last_became_active":"2024-04-02T14:41:22.538864+0000","last_became_peered":"2024-04-02T14:41:22.538864+0000","last_unstale":"2024-04-02T14:42:49.372613+0000","last_undegraded":"2024-04-02T14:42:49.372613+0000","last_fullsized":"2024-04-02T14:42:49.372613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:40:13.418752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.246172+0000","last_change":"2024-04-02T14:41:21.477043+0000","last_active":"2024-04-02T14:42:21.246172+0000","last_peered":"2024-04-02T14:42:21.246172+0000","last_clean":"2024-04-02T14:42:21.246172+0000","last_became_active":"2024-04-02T14:41:21.476580+0000","last_became_peered":"2024-04-02T14:41:21.476580+0000","last_unstale":"2024-04-02T14:42:21.246172+0000","last_undegraded":"2024-04-02T14:42:21.246172+0000","last_fullsized":"2024-04-02T14:42:21.246172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:18.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.375880+0000","last_change":"2024-04-02T14:41:22.549758+0000","last_active":"2024-04-02T14:42:49.375880+0000","last_peered":"2024-04-02T14:42:49.375880+0000","last_clean":"2024-04-02T14:42:49.375880+0000","last_became_active":"2024-04-02T14:41:22.545582+0000","last_became_peered":"2024-04-02T14:41:22.545582+0000","last_unstale":"2024-04-02T14:42:49.375880+0000","last_undegraded":"2024-04-02T14:42:49.375880+0000","last_fullsized":"2024-04-02T14:42:49.375880+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:42.311921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.253781+0000","last_change":"2024-04-02T14:42:21.253221+0000","last_active":"2024-04-02T14:42:21.253781+0000","last_peered":"2024-04-02T14:42:21.253781+0000","last_clean":"2024-04-02T14:42:21.253781+0000","last_became_active":"2024-04-02T14:42:21.247890+0000","last_became_peered":"2024-04-02T14:42:21.247890+0000","last_unstale":"2024-04-02T14:42:21.253781+0000","last_undegraded":"2024-04-02T14:42:21.253781+0000","last_fullsized":"2024-04-02T14:42:21.253781+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:12.571769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370984+0000","last_change":"2024-04-02T14:41:22.531509+0000","last_active":"2024-04-02T14:42:49.370984+0000","last_peered":"2024-04-02T14:42:49.370984+0000","last_clean":"2024-04-02T14:42:49.370984+0000","last_became_active":"2024-04-02T14:41:22.531206+0000","last_became_peered":"2024-04-02T14:41:22.531206+0000","last_unstale":"2024-04-02T14:42:49.370984+0000","last_undegraded":"2024-04-02T14:42:49.370984+0000","last_fullsized":"2024-04-02T14:42:49.370984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:39.099177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":112,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:54.282743+0000","last_change":"2024-04-02T14:41:21.485271+0000","last_active":"2024-04-02T14:42:54.282743+0000","last_peered":"2024-04-02T14:42:54.282743+0000","last_clean":"2024-04-02T14:42:54.282743+0000","last_became_active":"2024-04-02T14:41:21.484895+0000","last_became_peered":"2024-04-02T14:41:21.484895+0000","last_unstale":"2024-04-02T14:42:54.282743+0000","last_undegraded":"2024-04-02T14:42:54.282743+0000","last_fullsized":"2024-04-02T14:42:54.282743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.552092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.357178+0000","last_change":"2024-04-02T14:41:22.524853+0000","last_active":"2024-04-02T14:42:49.357178+0000","last_peered":"2024-04-02T14:42:49.357178+0000","last_clean":"2024-04-02T14:42:49.357178+0000","last_became_active":"2024-04-02T14:41:22.524403+0000","last_became_peered":"2024-04-02T14:41:22.524403+0000","last_unstale":"2024-04-02T14:42:49.357178+0000","last_undegraded":"2024-04-02T14:42:49.357178+0000","last_fullsized":"2024-04-02T14:42:49.357178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:31.322626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.242989+0000","last_change":"2024-04-02T14:41:21.474340+0000","last_active":"2024-04-02T14:42:21.242989+0000","last_peered":"2024-04-02T14:42:21.242989+0000","last_clean":"2024-04-02T14:42:21.242989+0000","last_became_active":"2024-04-02T14:41:21.473934+0000","last_became_peered":"2024-04-02T14:41:21.473934+0000","last_unstale":"2024-04-02T14:42:21.242989+0000","last_undegraded":"2024-04-02T14:42:21.242989+0000","last_fullsized":"2024-04-02T14:42:21.242989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:28:42.711817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370321+0000","last_change":"2024-04-02T14:41:22.543018+0000","last_active":"2024-04-02T14:42:49.370321+0000","last_peered":"2024-04-02T14:42:49.370321+0000","last_clean":"2024-04-02T14:42:49.370321+0000","last_became_active":"2024-04-02T14:41:22.542621+0000","last_became_peered":"2024-04-02T14:41:22.542621+0000","last_unstale":"2024-04-02T14:42:49.370321+0000","last_undegraded":"2024-04-02T14:42:49.370321+0000","last_fullsized":"2024-04-02T14:42:49.370321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:55:03.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.583362+0000","last_change":"2024-04-02T14:41:21.493060+0000","last_active":"2024-04-02T14:42:21.583362+0000","last_peered":"2024-04-02T14:42:21.583362+0000","last_clean":"2024-04-02T14:42:21.583362+0000","last_became_active":"2024-04-02T14:41:21.492627+0000","last_became_peered":"2024-04-02T14:41:21.492627+0000","last_unstale":"2024-04-02T14:42:21.583362+0000","last_undegraded":"2024-04-02T14:42:21.583362+0000","last_fullsized":"2024-04-02T14:42:21.583362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:29:10.443866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.348340+0000","last_change":"2024-04-02T14:41:22.527742+0000","last_active":"2024-04-02T14:42:49.348340+0000","last_peered":"2024-04-02T14:42:49.348340+0000","last_clean":"2024-04-02T14:42:49.348340+0000","last_became_active":"2024-04-02T14:41:22.527436+0000","last_became_peered":"2024-04-02T14:41:22.527436+0000","last_unstale":"2024-04-02T14:42:49.348340+0000","last_undegraded":"2024-04-02T14:42:49.348340+0000","last_fullsized":"2024-04-02T14:42:49.348340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:04:24.754907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.348168+0000","last_change":"2024-04-02T14:41:22.526235+0000","last_active":"2024-04-02T14:42:49.348168+0000","last_peered":"2024-04-02T14:42:49.348168+0000","last_clean":"2024-04-02T14:42:49.348168+0000","last_became_active":"2024-04-02T14:41:22.525879+0000","last_became_peered":"2024-04-02T14:41:22.525879+0000","last_unstale":"2024-04-02T14:42:49.348168+0000","last_undegraded":"2024-04-02T14:42:49.348168+0000","last_fullsized":"2024-04-02T14:42:49.348168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:20:54.431223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.344861+0000","last_change":"2024-04-02T14:41:22.522498+0000","last_active":"2024-04-02T14:42:49.344861+0000","last_peered":"2024-04-02T14:42:49.344861+0000","last_clean":"2024-04-02T14:42:49.344861+0000","last_became_active":"2024-04-02T14:41:22.522162+0000","last_became_peered":"2024-04-02T14:41:22.522162+0000","last_unstale":"2024-04-02T14:42:49.344861+0000","last_undegraded":"2024-04-02T14:42:49.344861+0000","last_fullsized":"2024-04-02T14:42:49.344861+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:42:04.701228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370821+0000","last_change":"2024-04-02T14:41:22.543920+0000","last_active":"2024-04-02T14:42:49.370821+0000","last_peered":"2024-04-02T14:42:49.370821+0000","last_clean":"2024-04-02T14:42:49.370821+0000","last_became_active":"2024-04-02T14:41:22.543552+0000","last_became_peered":"2024-04-02T14:41:22.543552+0000","last_unstale":"2024-04-02T14:42:49.370821+0000","last_undegraded":"2024-04-02T14:42:49.370821+0000","last_fullsized":"2024-04-02T14:42:49.370821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:28.761770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.377002+0000","last_change":"2024-04-02T14:41:22.550507+0000","last_active":"2024-04-02T14:42:49.377002+0000","last_peered":"2024-04-02T14:42:49.377002+0000","last_clean":"2024-04-02T14:42:49.377002+0000","last_became_active":"2024-04-02T14:41:22.545824+0000","last_became_peered":"2024-04-02T14:41:22.545824+0000","last_unstale":"2024-04-02T14:42:49.377002+0000","last_undegraded":"2024-04-02T14:42:49.377002+0000","last_fullsized":"2024-04-02T14:42:49.377002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:11.175138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.377015+0000","last_change":"2024-04-02T14:41:22.541362+0000","last_active":"2024-04-02T14:42:49.377015+0000","last_peered":"2024-04-02T14:42:49.377015+0000","last_clean":"2024-04-02T14:42:49.377015+0000","last_became_active":"2024-04-02T14:41:22.540927+0000","last_became_peered":"2024-04-02T14:41:22.540927+0000","last_unstale":"2024-04-02T14:42:49.377015+0000","last_undegraded":"2024-04-02T14:42:49.377015+0000","last_fullsized":"2024-04-02T14:42:49.377015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:43:00.746274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.344216+0000","last_change":"2024-04-02T14:41:22.527146+0000","last_active":"2024-04-02T14:42:49.344216+0000","last_peered":"2024-04-02T14:42:49.344216+0000","last_clean":"2024-04-02T14:42:49.344216+0000","last_became_active":"2024-04-02T14:41:22.526810+0000","last_became_peered":"2024-04-02T14:41:22.526810+0000","last_unstale":"2024-04-02T14:42:49.344216+0000","last_undegraded":"2024-04-02T14:42:49.344216+0000","last_fullsized":"2024-04-02T14:42:49.344216+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:18:46.464701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.335577+0000","last_change":"2024-04-02T14:41:22.524458+0000","last_active":"2024-04-02T14:42:49.335577+0000","last_peered":"2024-04-02T14:42:49.335577+0000","last_clean":"2024-04-02T14:42:49.335577+0000","last_became_active":"2024-04-02T14:41:22.524138+0000","last_became_peered":"2024-04-02T14:41:22.524138+0000","last_unstale":"2024-04-02T14:42:49.335577+0000","last_undegraded":"2024-04-02T14:42:49.335577+0000","last_fullsized":"2024-04-02T14:42:49.335577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:50:04.667964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.364010+0000","last_change":"2024-04-02T14:42:21.251102+0000","last_active":"2024-04-02T14:42:49.364010+0000","last_peered":"2024-04-02T14:42:49.364010+0000","last_clean":"2024-04-02T14:42:49.364010+0000","last_became_active":"2024-04-02T14:42:21.249134+0000","last_became_peered":"2024-04-02T14:42:21.249134+0000","last_unstale":"2024-04-02T14:42:49.364010+0000","last_undegraded":"2024-04-02T14:42:49.364010+0000","last_fullsized":"2024-04-02T14:42:49.364010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:05:56.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.360587+0000","last_change":"2024-04-02T14:41:22.542319+0000","last_active":"2024-04-02T14:42:49.360587+0000","last_peered":"2024-04-02T14:42:49.360587+0000","last_clean":"2024-04-02T14:42:49.360587+0000","last_became_active":"2024-04-02T14:41:22.541959+0000","last_became_peered":"2024-04-02T14:41:22.541959+0000","last_unstale":"2024-04-02T14:42:49.360587+0000","last_undegraded":"2024-04-02T14:42:49.360587+0000","last_fullsized":"2024-04-02T14:42:49.360587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:49.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.373377+0000","last_change":"2024-04-02T14:41:22.549932+0000","last_active":"2024-04-02T14:42:49.373377+0000","last_peered":"2024-04-02T14:42:49.373377+0000","last_clean":"2024-04-02T14:42:49.373377+0000","last_became_active":"2024-04-02T14:41:22.549231+0000","last_became_peered":"2024-04-02T14:41:22.549231+0000","last_unstale":"2024-04-02T14:42:49.373377+0000","last_undegraded":"2024-04-02T14:42:49.373377+0000","last_fullsized":"2024-04-02T14:42:49.373377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:01.485343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.374119+0000","last_change":"2024-04-02T14:41:22.544712+0000","last_active":"2024-04-02T14:42:49.374119+0000","last_peered":"2024-04-02T14:42:49.374119+0000","last_clean":"2024-04-02T14:42:49.374119+0000","last_became_active":"2024-04-02T14:41:22.544043+0000","last_became_peered":"2024-04-02T14:41:22.544043+0000","last_unstale":"2024-04-02T14:42:49.374119+0000","last_undegraded":"2024-04-02T14:42:49.374119+0000","last_fullsized":"2024-04-02T14:42:49.374119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:07.931377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.377733+0000","last_change":"2024-04-02T14:41:22.542364+0000","last_active":"2024-04-02T14:42:49.377733+0000","last_peered":"2024-04-02T14:42:49.377733+0000","last_clean":"2024-04-02T14:42:49.377733+0000","last_became_active":"2024-04-02T14:41:22.541929+0000","last_became_peered":"2024-04-02T14:41:22.541929+0000","last_unstale":"2024-04-02T14:42:49.377733+0000","last_undegraded":"2024-04-02T14:42:49.377733+0000","last_fullsized":"2024-04-02T14:42:49.377733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:45.716132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.344955+0000","last_change":"2024-04-02T14:41:22.525295+0000","last_active":"2024-04-02T14:42:49.344955+0000","last_peered":"2024-04-02T14:42:49.344955+0000","last_clean":"2024-04-02T14:42:49.344955+0000","last_became_active":"2024-04-02T14:41:22.524484+0000","last_became_peered":"2024-04-02T14:41:22.524484+0000","last_unstale":"2024-04-02T14:42:49.344955+0000","last_undegraded":"2024-04-02T14:42:49.344955+0000","last_fullsized":"2024-04-02T14:42:49.344955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:01.819082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65530,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":882,"num_read_kb":516,"num_write":243,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":196590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285600,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":469191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547237,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1118796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710763,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":469121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972402,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":469499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266743,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1062158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430268,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":468838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":528327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":470177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":527925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117068,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":468663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855433,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":469853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691906,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1061836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":59926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:42:59.938 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T14:43:00.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:00 smithi113 ceph-mon[36135]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:43:00.417 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:43:00.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:00 smithi204 ceph-mon[35948]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:43:00.571 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:00 smithi062 ceph-mon[32116]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:43:01.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:01 smithi062 ceph-mon[32116]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T14:43:01.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:01 smithi113 ceph-mon[36135]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T14:43:01.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:01 smithi204 ceph-mon[35948]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T14:43:02.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.927+0000 7fbe43e00700 1 -- 172.21.15.62:0/864446109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe440749c0 msgr2=0x7fbe44074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.927+0000 7fbe43e00700 1 --2- 172.21.15.62:0/864446109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe440749c0 0x7fbe44074da0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400b0a0 tx=0x7fbe3400b3b0 comp rx=0 tx=0).stop 2024-04-02T14:43:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.927+0000 7fbe43e00700 1 -- 172.21.15.62:0/864446109 shutdown_connections 2024-04-02T14:43:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.927+0000 7fbe43e00700 1 --2- 172.21.15.62:0/864446109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe4410ed80 0x7fbe44111190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.927+0000 7fbe43e00700 1 --2- 172.21.15.62:0/864446109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe440749c0 0x7fbe44074da0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.927+0000 7fbe43e00700 1 --2- 172.21.15.62:0/864446109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe440752e0 0x7fbe4410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.927+0000 7fbe43e00700 1 -- 172.21.15.62:0/864446109 >> 172.21.15.62:0/864446109 conn(0x7fbe440fdff0 msgr2=0x7fbe44100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.927+0000 7fbe43e00700 1 -- 172.21.15.62:0/864446109 shutdown_connections 2024-04-02T14:43:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.927+0000 7fbe43e00700 1 -- 172.21.15.62:0/864446109 wait complete. 2024-04-02T14:43:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.928+0000 7fbe43e00700 1 Processor -- start 2024-04-02T14:43:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.928+0000 7fbe43e00700 1 -- start start 2024-04-02T14:43:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.929+0000 7fbe43e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe440749c0 0x7fbe441024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.929+0000 7fbe43e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe440752e0 0x7fbe441029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.929+0000 7fbe43e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe4410ed80 0x7fbe44102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.929+0000 7fbe43e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbe44114b50 con 0x7fbe4410ed80 2024-04-02T14:43:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.929+0000 7fbe43e00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe441149d0 con 0x7fbe440752e0 2024-04-02T14:43:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.929+0000 7fbe43e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe44114cd0 con 0x7fbe440749c0 2024-04-02T14:43:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.929+0000 7fbe42000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe440752e0 0x7fbe441029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.929+0000 7fbe42a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe440749c0 0x7fbe441024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.929+0000 7fbe42000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe440752e0 0x7fbe441029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:45480/0 (socket says 172.21.15.62:45480) 2024-04-02T14:43:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.929+0000 7fbe42000700 1 -- 172.21.15.62:0/2528781997 learned_addr learned my addr 172.21.15.62:0/2528781997 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.930+0000 7fbe43400700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe4410ed80 0x7fbe44102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.930+0000 7fbe42000700 1 -- 172.21.15.62:0/2528781997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe440749c0 msgr2=0x7fbe441024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.930+0000 7fbe42000700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe440749c0 0x7fbe441024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.930+0000 7fbe42000700 1 -- 172.21.15.62:0/2528781997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe4410ed80 msgr2=0x7fbe44102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.930+0000 7fbe42000700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe4410ed80 0x7fbe44102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.930+0000 7fbe42000700 1 -- 172.21.15.62:0/2528781997 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe441ab690 con 0x7fbe440752e0 2024-04-02T14:43:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.930+0000 7fbe42000700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe440752e0 0x7fbe441029e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000e9d0 tx=0x7fbe3000ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.931+0000 7fbe3b400700 1 -- 172.21.15.62:0/2528781997 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000cbd0 con 0x7fbe440752e0 2024-04-02T14:43:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.931+0000 7fbe43e00700 1 -- 172.21.15.62:0/2528781997 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe441ab980 con 0x7fbe440752e0 2024-04-02T14:43:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.931+0000 7fbe3b400700 1 -- 172.21.15.62:0/2528781997 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe3000ee60 con 0x7fbe440752e0 2024-04-02T14:43:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.931+0000 7fbe3b400700 1 -- 172.21.15.62:0/2528781997 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30018710 con 0x7fbe440752e0 2024-04-02T14:43:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.931+0000 7fbe43e00700 1 -- 172.21.15.62:0/2528781997 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe441abec0 con 0x7fbe440752e0 2024-04-02T14:43:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.932+0000 7fbe43e00700 1 -- 172.21.15.62:0/2528781997 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe44076070 con 0x7fbe440752e0 2024-04-02T14:43:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.934+0000 7fbe3b400700 1 -- 172.21.15.62:0/2528781997 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbe300188b0 con 0x7fbe440752e0 2024-04-02T14:43:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.935+0000 7fbe3b400700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbe280a3c40 0x7fbe280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.936+0000 7fbe3b400700 1 -- 172.21.15.62:0/2528781997 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fbe300d05e0 con 0x7fbe440752e0 2024-04-02T14:43:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.936+0000 7fbe42a00700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbe280a3c40 0x7fbe280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.936+0000 7fbe42a00700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbe280a3c40 0x7fbe280a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009e60 tx=0x7fbe3400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:02.938+0000 7fbe3b400700 1 -- 172.21.15.62:0/2528781997 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbe3009cf90 con 0x7fbe440752e0 2024-04-02T14:43:03.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.170+0000 7fbe43e00700 1 -- 172.21.15.62:0/2528781997 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe44062a30 con 0x7fbe280a3c40 2024-04-02T14:43:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.179+0000 7fbe3b400700 1 -- 172.21.15.62:0/2528781997 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258884 (secure 0 0 0) 0x7fbe44062a30 con 0x7fbe280a3c40 2024-04-02T14:43:03.183 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:03.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.186+0000 7fbe43e00700 1 -- 172.21.15.62:0/2528781997 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbe280a3c40 msgr2=0x7fbe280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:03.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.186+0000 7fbe43e00700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbe280a3c40 0x7fbe280a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009e60 tx=0x7fbe3400b700 comp rx=0 tx=0).stop 2024-04-02T14:43:03.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.186+0000 7fbe43e00700 1 -- 172.21.15.62:0/2528781997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe440752e0 msgr2=0x7fbe441029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:03.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.186+0000 7fbe43e00700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe440752e0 0x7fbe441029e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000e9d0 tx=0x7fbe3000ece0 comp rx=0 tx=0).stop 2024-04-02T14:43:03.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.187+0000 7fbe43e00700 1 -- 172.21.15.62:0/2528781997 shutdown_connections 2024-04-02T14:43:03.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.187+0000 7fbe43e00700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fbe280a3c40 0x7fbe280a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:03.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.187+0000 7fbe43e00700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe4410ed80 0x7fbe44102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:03.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.187+0000 7fbe43e00700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe440749c0 0x7fbe441024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:03.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.187+0000 7fbe43e00700 1 --2- 172.21.15.62:0/2528781997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe440752e0 0x7fbe441029e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:03.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.187+0000 7fbe43e00700 1 -- 172.21.15.62:0/2528781997 >> 172.21.15.62:0/2528781997 conn(0x7fbe440fdff0 msgr2=0x7fbe440fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:03.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.187+0000 7fbe43e00700 1 -- 172.21.15.62:0/2528781997 shutdown_connections 2024-04-02T14:43:03.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:03.187+0000 7fbe43e00700 1 -- 172.21.15.62:0/2528781997 wait complete. 2024-04-02T14:43:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-02T14:43:03.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:03 smithi113 ceph-mon[36135]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T14:43:03.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:03 smithi204 ceph-mon[35948]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T14:43:03.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:03 smithi062 ceph-mon[32116]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T14:43:04.060 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-04-02T14:43:01.395259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656132,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1068,"num_read_kb":660,"num_write":426,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181749,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":849,"ondisk_log_size":849,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404396,"kb_used_data":13644,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537428,"statfs":{"total":1151940427776,"available":1151526326272,"internally_reserved":0,"allocated":13971456,"data_stored":7584384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":77,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005273"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372298+0000","last_change":"2024-04-02T14:41:22.546468+0000","last_active":"2024-04-02T14:42:49.372298+0000","last_peered":"2024-04-02T14:42:49.372298+0000","last_clean":"2024-04-02T14:42:49.372298+0000","last_became_active":"2024-04-02T14:41:22.545991+0000","last_became_peered":"2024-04-02T14:41:22.545991+0000","last_unstale":"2024-04-02T14:42:49.372298+0000","last_undegraded":"2024-04-02T14:42:49.372298+0000","last_fullsized":"2024-04-02T14:42:49.372298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:34:24.520404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.359664+0000","last_change":"2024-04-02T14:41:22.547738+0000","last_active":"2024-04-02T14:42:49.359664+0000","last_peered":"2024-04-02T14:42:49.359664+0000","last_clean":"2024-04-02T14:42:49.359664+0000","last_became_active":"2024-04-02T14:41:22.542372+0000","last_became_peered":"2024-04-02T14:41:22.542372+0000","last_unstale":"2024-04-02T14:42:49.359664+0000","last_undegraded":"2024-04-02T14:42:49.359664+0000","last_fullsized":"2024-04-02T14:42:49.359664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:07:26.951136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.365915+0000","last_change":"2024-04-02T14:41:22.540173+0000","last_active":"2024-04-02T14:42:49.365915+0000","last_peered":"2024-04-02T14:42:49.365915+0000","last_clean":"2024-04-02T14:42:49.365915+0000","last_became_active":"2024-04-02T14:41:22.536279+0000","last_became_peered":"2024-04-02T14:41:22.536279+0000","last_unstale":"2024-04-02T14:42:49.365915+0000","last_undegraded":"2024-04-02T14:42:49.365915+0000","last_fullsized":"2024-04-02T14:42:49.365915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:43:40.826934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.363245+0000","last_change":"2024-04-02T14:42:21.251675+0000","last_active":"2024-04-02T14:42:49.363245+0000","last_peered":"2024-04-02T14:42:49.363245+0000","last_clean":"2024-04-02T14:42:49.363245+0000","last_became_active":"2024-04-02T14:42:21.250446+0000","last_became_peered":"2024-04-02T14:42:21.250446+0000","last_unstale":"2024-04-02T14:42:49.363245+0000","last_undegraded":"2024-04-02T14:42:49.363245+0000","last_fullsized":"2024-04-02T14:42:49.363245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:11.005276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370555+0000","last_change":"2024-04-02T14:41:22.554301+0000","last_active":"2024-04-02T14:42:49.370555+0000","last_peered":"2024-04-02T14:42:49.370555+0000","last_clean":"2024-04-02T14:42:49.370555+0000","last_became_active":"2024-04-02T14:41:22.554028+0000","last_became_peered":"2024-04-02T14:41:22.554028+0000","last_unstale":"2024-04-02T14:42:49.370555+0000","last_undegraded":"2024-04-02T14:42:49.370555+0000","last_fullsized":"2024-04-02T14:42:49.370555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:51:59.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.359271+0000","last_change":"2024-04-02T14:41:22.543934+0000","last_active":"2024-04-02T14:42:49.359271+0000","last_peered":"2024-04-02T14:42:49.359271+0000","last_clean":"2024-04-02T14:42:49.359271+0000","last_became_active":"2024-04-02T14:41:22.543392+0000","last_became_peered":"2024-04-02T14:41:22.543392+0000","last_unstale":"2024-04-02T14:42:49.359271+0000","last_undegraded":"2024-04-02T14:42:49.359271+0000","last_fullsized":"2024-04-02T14:42:49.359271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:28:30.234356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.362683+0000","last_change":"2024-04-02T14:41:22.547460+0000","last_active":"2024-04-02T14:42:49.362683+0000","last_peered":"2024-04-02T14:42:49.362683+0000","last_clean":"2024-04-02T14:42:49.362683+0000","last_became_active":"2024-04-02T14:41:22.547123+0000","last_became_peered":"2024-04-02T14:41:22.547123+0000","last_unstale":"2024-04-02T14:42:49.362683+0000","last_undegraded":"2024-04-02T14:42:49.362683+0000","last_fullsized":"2024-04-02T14:42:49.362683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:24:34.302530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372164+0000","last_change":"2024-04-02T14:41:22.551407+0000","last_active":"2024-04-02T14:42:49.372164+0000","last_peered":"2024-04-02T14:42:49.372164+0000","last_clean":"2024-04-02T14:42:49.372164+0000","last_became_active":"2024-04-02T14:41:22.538304+0000","last_became_peered":"2024-04-02T14:41:22.538304+0000","last_unstale":"2024-04-02T14:42:49.372164+0000","last_undegraded":"2024-04-02T14:42:49.372164+0000","last_fullsized":"2024-04-02T14:42:49.372164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:27.933380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.361684+0000","last_change":"2024-04-02T14:41:22.523931+0000","last_active":"2024-04-02T14:42:49.361684+0000","last_peered":"2024-04-02T14:42:49.361684+0000","last_clean":"2024-04-02T14:42:49.361684+0000","last_became_active":"2024-04-02T14:41:22.523622+0000","last_became_peered":"2024-04-02T14:41:22.523622+0000","last_unstale":"2024-04-02T14:42:49.361684+0000","last_undegraded":"2024-04-02T14:42:49.361684+0000","last_fullsized":"2024-04-02T14:42:49.361684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:17.808290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.353241+0000","last_change":"2024-04-02T14:41:22.525791+0000","last_active":"2024-04-02T14:42:49.353241+0000","last_peered":"2024-04-02T14:42:49.353241+0000","last_clean":"2024-04-02T14:42:49.353241+0000","last_became_active":"2024-04-02T14:41:22.525434+0000","last_became_peered":"2024-04-02T14:41:22.525434+0000","last_unstale":"2024-04-02T14:42:49.353241+0000","last_undegraded":"2024-04-02T14:42:49.353241+0000","last_fullsized":"2024-04-02T14:42:49.353241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:59.905337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.357047+0000","last_change":"2024-04-02T14:41:22.527240+0000","last_active":"2024-04-02T14:42:49.357047+0000","last_peered":"2024-04-02T14:42:49.357047+0000","last_clean":"2024-04-02T14:42:49.357047+0000","last_became_active":"2024-04-02T14:41:22.526412+0000","last_became_peered":"2024-04-02T14:41:22.526412+0000","last_unstale":"2024-04-02T14:42:49.357047+0000","last_undegraded":"2024-04-02T14:42:49.357047+0000","last_fullsized":"2024-04-02T14:42:49.357047+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:47.397012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.346146+0000","last_change":"2024-04-02T14:41:22.527281+0000","last_active":"2024-04-02T14:42:49.346146+0000","last_peered":"2024-04-02T14:42:49.346146+0000","last_clean":"2024-04-02T14:42:49.346146+0000","last_became_active":"2024-04-02T14:41:22.526946+0000","last_became_peered":"2024-04-02T14:41:22.526946+0000","last_unstale":"2024-04-02T14:42:49.346146+0000","last_undegraded":"2024-04-02T14:42:49.346146+0000","last_fullsized":"2024-04-02T14:42:49.346146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:42.798282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372127+0000","last_change":"2024-04-02T14:41:22.534079+0000","last_active":"2024-04-02T14:42:49.372127+0000","last_peered":"2024-04-02T14:42:49.372127+0000","last_clean":"2024-04-02T14:42:49.372127+0000","last_became_active":"2024-04-02T14:41:22.532233+0000","last_became_peered":"2024-04-02T14:41:22.532233+0000","last_unstale":"2024-04-02T14:42:49.372127+0000","last_undegraded":"2024-04-02T14:42:49.372127+0000","last_fullsized":"2024-04-02T14:42:49.372127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:27:47.122933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.362395+0000","last_change":"2024-04-02T14:41:22.544936+0000","last_active":"2024-04-02T14:42:49.362395+0000","last_peered":"2024-04-02T14:42:49.362395+0000","last_clean":"2024-04-02T14:42:49.362395+0000","last_became_active":"2024-04-02T14:41:22.541805+0000","last_became_peered":"2024-04-02T14:41:22.541805+0000","last_unstale":"2024-04-02T14:42:49.362395+0000","last_undegraded":"2024-04-02T14:42:49.362395+0000","last_fullsized":"2024-04-02T14:42:49.362395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:23.992807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.361980+0000","last_change":"2024-04-02T14:41:22.534860+0000","last_active":"2024-04-02T14:42:49.361980+0000","last_peered":"2024-04-02T14:42:49.361980+0000","last_clean":"2024-04-02T14:42:49.361980+0000","last_became_active":"2024-04-02T14:41:22.534147+0000","last_became_peered":"2024-04-02T14:41:22.534147+0000","last_unstale":"2024-04-02T14:42:49.361980+0000","last_undegraded":"2024-04-02T14:42:49.361980+0000","last_fullsized":"2024-04-02T14:42:49.361980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:25.249364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.375984+0000","last_change":"2024-04-02T14:41:22.537654+0000","last_active":"2024-04-02T14:42:49.375984+0000","last_peered":"2024-04-02T14:42:49.375984+0000","last_clean":"2024-04-02T14:42:49.375984+0000","last_became_active":"2024-04-02T14:41:22.537304+0000","last_became_peered":"2024-04-02T14:41:22.537304+0000","last_unstale":"2024-04-02T14:42:49.375984+0000","last_undegraded":"2024-04-02T14:42:49.375984+0000","last_fullsized":"2024-04-02T14:42:49.375984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:25.599724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372388+0000","last_change":"2024-04-02T14:41:22.546775+0000","last_active":"2024-04-02T14:42:49.372388+0000","last_peered":"2024-04-02T14:42:49.372388+0000","last_clean":"2024-04-02T14:42:49.372388+0000","last_became_active":"2024-04-02T14:41:22.546100+0000","last_became_peered":"2024-04-02T14:41:22.546100+0000","last_unstale":"2024-04-02T14:42:49.372388+0000","last_undegraded":"2024-04-02T14:42:49.372388+0000","last_fullsized":"2024-04-02T14:42:49.372388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:42:02.559335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.364567+0000","last_change":"2024-04-02T14:41:22.543988+0000","last_active":"2024-04-02T14:42:49.364567+0000","last_peered":"2024-04-02T14:42:49.364567+0000","last_clean":"2024-04-02T14:42:49.364567+0000","last_became_active":"2024-04-02T14:41:22.543036+0000","last_became_peered":"2024-04-02T14:41:22.543036+0000","last_unstale":"2024-04-02T14:42:49.364567+0000","last_undegraded":"2024-04-02T14:42:49.364567+0000","last_fullsized":"2024-04-02T14:42:49.364567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:29.470929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.354246+0000","last_change":"2024-04-02T14:41:22.543954+0000","last_active":"2024-04-02T14:42:49.354246+0000","last_peered":"2024-04-02T14:42:49.354246+0000","last_clean":"2024-04-02T14:42:49.354246+0000","last_became_active":"2024-04-02T14:41:22.543624+0000","last_became_peered":"2024-04-02T14:41:22.543624+0000","last_unstale":"2024-04-02T14:42:49.354246+0000","last_undegraded":"2024-04-02T14:42:49.354246+0000","last_fullsized":"2024-04-02T14:42:49.354246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:46.933877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:58.280774+0000","last_change":"2024-04-02T14:41:21.477341+0000","last_active":"2024-04-02T14:42:58.280774+0000","last_peered":"2024-04-02T14:42:58.280774+0000","last_clean":"2024-04-02T14:42:58.280774+0000","last_became_active":"2024-04-02T14:41:21.476708+0000","last_became_peered":"2024-04-02T14:41:21.476708+0000","last_unstale":"2024-04-02T14:42:58.280774+0000","last_undegraded":"2024-04-02T14:42:58.280774+0000","last_fullsized":"2024-04-02T14:42:58.280774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:34.867570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.355502+0000","last_change":"2024-04-02T14:41:22.551293+0000","last_active":"2024-04-02T14:42:49.355502+0000","last_peered":"2024-04-02T14:42:49.355502+0000","last_clean":"2024-04-02T14:42:49.355502+0000","last_became_active":"2024-04-02T14:41:22.547819+0000","last_became_peered":"2024-04-02T14:41:22.547819+0000","last_unstale":"2024-04-02T14:42:49.355502+0000","last_undegraded":"2024-04-02T14:42:49.355502+0000","last_fullsized":"2024-04-02T14:42:49.355502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:40.393005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.239792+0000","last_change":"2024-04-02T14:41:21.487096+0000","last_active":"2024-04-02T14:42:21.239792+0000","last_peered":"2024-04-02T14:42:21.239792+0000","last_clean":"2024-04-02T14:42:21.239792+0000","last_became_active":"2024-04-02T14:41:21.485883+0000","last_became_peered":"2024-04-02T14:41:21.485883+0000","last_unstale":"2024-04-02T14:42:21.239792+0000","last_undegraded":"2024-04-02T14:42:21.239792+0000","last_fullsized":"2024-04-02T14:42:21.239792+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:24.017671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.371157+0000","last_change":"2024-04-02T14:41:22.549234+0000","last_active":"2024-04-02T14:42:49.371157+0000","last_peered":"2024-04-02T14:42:49.371157+0000","last_clean":"2024-04-02T14:42:49.371157+0000","last_became_active":"2024-04-02T14:41:22.548855+0000","last_became_peered":"2024-04-02T14:41:22.548855+0000","last_unstale":"2024-04-02T14:42:49.371157+0000","last_undegraded":"2024-04-02T14:42:49.371157+0000","last_fullsized":"2024-04-02T14:42:49.371157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:05.047383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.245395+0000","last_change":"2024-04-02T14:41:21.471673+0000","last_active":"2024-04-02T14:42:21.245395+0000","last_peered":"2024-04-02T14:42:21.245395+0000","last_clean":"2024-04-02T14:42:21.245395+0000","last_became_active":"2024-04-02T14:41:21.471292+0000","last_became_peered":"2024-04-02T14:41:21.471292+0000","last_unstale":"2024-04-02T14:42:21.245395+0000","last_undegraded":"2024-04-02T14:42:21.245395+0000","last_fullsized":"2024-04-02T14:42:21.245395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:42:55.665938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.373736+0000","last_change":"2024-04-02T14:41:22.537665+0000","last_active":"2024-04-02T14:42:49.373736+0000","last_peered":"2024-04-02T14:42:49.373736+0000","last_clean":"2024-04-02T14:42:49.373736+0000","last_became_active":"2024-04-02T14:41:22.537274+0000","last_became_peered":"2024-04-02T14:41:22.537274+0000","last_unstale":"2024-04-02T14:42:49.373736+0000","last_undegraded":"2024-04-02T14:42:49.373736+0000","last_fullsized":"2024-04-02T14:42:49.373736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:14:39.200735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.246137+0000","last_change":"2024-04-02T14:41:21.466547+0000","last_active":"2024-04-02T14:42:21.246137+0000","last_peered":"2024-04-02T14:42:21.246137+0000","last_clean":"2024-04-02T14:42:21.246137+0000","last_became_active":"2024-04-02T14:41:21.466063+0000","last_became_peered":"2024-04-02T14:41:21.466063+0000","last_unstale":"2024-04-02T14:42:21.246137+0000","last_undegraded":"2024-04-02T14:42:21.246137+0000","last_fullsized":"2024-04-02T14:42:21.246137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:39.201543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.351340+0000","last_change":"2024-04-02T14:41:22.523984+0000","last_active":"2024-04-02T14:42:49.351340+0000","last_peered":"2024-04-02T14:42:49.351340+0000","last_clean":"2024-04-02T14:42:49.351340+0000","last_became_active":"2024-04-02T14:41:22.523524+0000","last_became_peered":"2024-04-02T14:41:22.523524+0000","last_unstale":"2024-04-02T14:42:49.351340+0000","last_undegraded":"2024-04-02T14:42:49.351340+0000","last_fullsized":"2024-04-02T14:42:49.351340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:31.542588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":130,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:57.297504+0000","last_change":"2024-04-02T14:42:21.255204+0000","last_active":"2024-04-02T14:42:57.297504+0000","last_peered":"2024-04-02T14:42:57.297504+0000","last_clean":"2024-04-02T14:42:57.297504+0000","last_became_active":"2024-04-02T14:42:21.248702+0000","last_became_peered":"2024-04-02T14:42:21.248702+0000","last_unstale":"2024-04-02T14:42:57.297504+0000","last_undegraded":"2024-04-02T14:42:57.297504+0000","last_fullsized":"2024-04-02T14:42:57.297504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.723036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.352203+0000","last_change":"2024-04-02T14:41:22.544107+0000","last_active":"2024-04-02T14:42:49.352203+0000","last_peered":"2024-04-02T14:42:49.352203+0000","last_clean":"2024-04-02T14:42:49.352203+0000","last_became_active":"2024-04-02T14:41:22.543739+0000","last_became_peered":"2024-04-02T14:41:22.543739+0000","last_unstale":"2024-04-02T14:42:49.352203+0000","last_undegraded":"2024-04-02T14:42:49.352203+0000","last_fullsized":"2024-04-02T14:42:49.352203+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:49:08.921668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.236590+0000","last_change":"2024-04-02T14:41:21.479714+0000","last_active":"2024-04-02T14:42:21.236590+0000","last_peered":"2024-04-02T14:42:21.236590+0000","last_clean":"2024-04-02T14:42:21.236590+0000","last_became_active":"2024-04-02T14:41:21.479329+0000","last_became_peered":"2024-04-02T14:41:21.479329+0000","last_unstale":"2024-04-02T14:42:21.236590+0000","last_undegraded":"2024-04-02T14:42:21.236590+0000","last_fullsized":"2024-04-02T14:42:21.236590+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:09.021017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.373049+0000","last_change":"2024-04-02T14:42:21.254332+0000","last_active":"2024-04-02T14:42:49.373049+0000","last_peered":"2024-04-02T14:42:49.373049+0000","last_clean":"2024-04-02T14:42:49.373049+0000","last_became_active":"2024-04-02T14:42:21.251266+0000","last_became_peered":"2024-04-02T14:42:21.251266+0000","last_unstale":"2024-04-02T14:42:49.373049+0000","last_undegraded":"2024-04-02T14:42:49.373049+0000","last_fullsized":"2024-04-02T14:42:49.373049+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:51:18.343098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.582968+0000","last_change":"2024-04-02T14:41:21.484054+0000","last_active":"2024-04-02T14:42:21.582968+0000","last_peered":"2024-04-02T14:42:21.582968+0000","last_clean":"2024-04-02T14:42:21.582968+0000","last_became_active":"2024-04-02T14:41:21.483688+0000","last_became_peered":"2024-04-02T14:41:21.483688+0000","last_unstale":"2024-04-02T14:42:21.582968+0000","last_undegraded":"2024-04-02T14:42:21.582968+0000","last_fullsized":"2024-04-02T14:42:21.582968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:24.114180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.378217+0000","last_change":"2024-04-02T14:41:22.553274+0000","last_active":"2024-04-02T14:42:49.378217+0000","last_peered":"2024-04-02T14:42:49.378217+0000","last_clean":"2024-04-02T14:42:49.378217+0000","last_became_active":"2024-04-02T14:41:22.545412+0000","last_became_peered":"2024-04-02T14:41:22.545412+0000","last_unstale":"2024-04-02T14:42:49.378217+0000","last_undegraded":"2024-04-02T14:42:49.378217+0000","last_fullsized":"2024-04-02T14:42:49.378217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:21.917119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.242398+0000","last_change":"2024-04-02T14:41:21.468582+0000","last_active":"2024-04-02T14:42:21.242398+0000","last_peered":"2024-04-02T14:42:21.242398+0000","last_clean":"2024-04-02T14:42:21.242398+0000","last_became_active":"2024-04-02T14:41:21.468081+0000","last_became_peered":"2024-04-02T14:41:21.468081+0000","last_unstale":"2024-04-02T14:42:21.242398+0000","last_undegraded":"2024-04-02T14:42:21.242398+0000","last_fullsized":"2024-04-02T14:42:21.242398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:19.593081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.374638+0000","last_change":"2024-04-02T14:41:22.555627+0000","last_active":"2024-04-02T14:42:49.374638+0000","last_peered":"2024-04-02T14:42:49.374638+0000","last_clean":"2024-04-02T14:42:49.374638+0000","last_became_active":"2024-04-02T14:41:22.555171+0000","last_became_peered":"2024-04-02T14:41:22.555171+0000","last_unstale":"2024-04-02T14:42:49.374638+0000","last_undegraded":"2024-04-02T14:42:49.374638+0000","last_fullsized":"2024-04-02T14:42:49.374638+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:21:28.161869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":204,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:59.291788+0000","last_change":"2024-04-02T14:41:21.475004+0000","last_active":"2024-04-02T14:42:59.291788+0000","last_peered":"2024-04-02T14:42:59.291788+0000","last_clean":"2024-04-02T14:42:59.291788+0000","last_became_active":"2024-04-02T14:41:21.474621+0000","last_became_peered":"2024-04-02T14:41:21.474621+0000","last_unstale":"2024-04-02T14:42:59.291788+0000","last_undegraded":"2024-04-02T14:42:59.291788+0000","last_fullsized":"2024-04-02T14:42:59.291788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:38.779276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":180,"num_read_kb":180,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.365367+0000","last_change":"2024-04-02T14:41:22.543100+0000","last_active":"2024-04-02T14:42:49.365367+0000","last_peered":"2024-04-02T14:42:49.365367+0000","last_clean":"2024-04-02T14:42:49.365367+0000","last_became_active":"2024-04-02T14:41:22.542697+0000","last_became_peered":"2024-04-02T14:41:22.542697+0000","last_unstale":"2024-04-02T14:42:49.365367+0000","last_undegraded":"2024-04-02T14:42:49.365367+0000","last_fullsized":"2024-04-02T14:42:49.365367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:50:52.412879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.245264+0000","last_change":"2024-04-02T14:41:21.468626+0000","last_active":"2024-04-02T14:42:21.245264+0000","last_peered":"2024-04-02T14:42:21.245264+0000","last_clean":"2024-04-02T14:42:21.245264+0000","last_became_active":"2024-04-02T14:41:21.468041+0000","last_became_peered":"2024-04-02T14:41:21.468041+0000","last_unstale":"2024-04-02T14:42:21.245264+0000","last_undegraded":"2024-04-02T14:42:21.245264+0000","last_fullsized":"2024-04-02T14:42:21.245264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:55.064033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370998+0000","last_change":"2024-04-02T14:41:22.539584+0000","last_active":"2024-04-02T14:42:49.370998+0000","last_peered":"2024-04-02T14:42:49.370998+0000","last_clean":"2024-04-02T14:42:49.370998+0000","last_became_active":"2024-04-02T14:41:22.539218+0000","last_became_peered":"2024-04-02T14:41:22.539218+0000","last_unstale":"2024-04-02T14:42:49.370998+0000","last_undegraded":"2024-04-02T14:42:49.370998+0000","last_fullsized":"2024-04-02T14:42:49.370998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:47:56.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.243164+0000","last_change":"2024-04-02T14:41:21.481208+0000","last_active":"2024-04-02T14:42:21.243164+0000","last_peered":"2024-04-02T14:42:21.243164+0000","last_clean":"2024-04-02T14:42:21.243164+0000","last_became_active":"2024-04-02T14:41:21.480782+0000","last_became_peered":"2024-04-02T14:41:21.480782+0000","last_unstale":"2024-04-02T14:42:21.243164+0000","last_undegraded":"2024-04-02T14:42:21.243164+0000","last_fullsized":"2024-04-02T14:42:21.243164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:12.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.363872+0000","last_change":"2024-04-02T14:41:22.517397+0000","last_active":"2024-04-02T14:42:49.363872+0000","last_peered":"2024-04-02T14:42:49.363872+0000","last_clean":"2024-04-02T14:42:49.363872+0000","last_became_active":"2024-04-02T14:41:22.517027+0000","last_became_peered":"2024-04-02T14:41:22.517027+0000","last_unstale":"2024-04-02T14:42:49.363872+0000","last_undegraded":"2024-04-02T14:42:49.363872+0000","last_fullsized":"2024-04-02T14:42:49.363872+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:53.129949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":237,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:59.300060+0000","last_change":"2024-04-02T14:41:21.469643+0000","last_active":"2024-04-02T14:42:59.300060+0000","last_peered":"2024-04-02T14:42:59.300060+0000","last_clean":"2024-04-02T14:42:59.300060+0000","last_became_active":"2024-04-02T14:41:21.468730+0000","last_became_peered":"2024-04-02T14:41:21.468730+0000","last_unstale":"2024-04-02T14:42:59.300060+0000","last_undegraded":"2024-04-02T14:42:59.300060+0000","last_fullsized":"2024-04-02T14:42:59.300060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:10.983551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58576,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":187,"num_read_kb":152,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.344508+0000","last_change":"2024-04-02T14:41:22.525052+0000","last_active":"2024-04-02T14:42:49.344508+0000","last_peered":"2024-04-02T14:42:49.344508+0000","last_clean":"2024-04-02T14:42:49.344508+0000","last_became_active":"2024-04-02T14:41:22.524691+0000","last_became_peered":"2024-04-02T14:41:22.524691+0000","last_unstale":"2024-04-02T14:42:49.344508+0000","last_undegraded":"2024-04-02T14:42:49.344508+0000","last_fullsized":"2024-04-02T14:42:49.344508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:16.386020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.246975+0000","last_change":"2024-04-02T14:41:21.476988+0000","last_active":"2024-04-02T14:42:21.246975+0000","last_peered":"2024-04-02T14:42:21.246975+0000","last_clean":"2024-04-02T14:42:21.246975+0000","last_became_active":"2024-04-02T14:41:21.476549+0000","last_became_peered":"2024-04-02T14:41:21.476549+0000","last_unstale":"2024-04-02T14:42:21.246975+0000","last_undegraded":"2024-04-02T14:42:21.246975+0000","last_fullsized":"2024-04-02T14:42:21.246975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:47.498626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.376278+0000","last_change":"2024-04-02T14:41:22.544746+0000","last_active":"2024-04-02T14:42:49.376278+0000","last_peered":"2024-04-02T14:42:49.376278+0000","last_clean":"2024-04-02T14:42:49.376278+0000","last_became_active":"2024-04-02T14:41:22.544334+0000","last_became_peered":"2024-04-02T14:41:22.544334+0000","last_unstale":"2024-04-02T14:42:49.376278+0000","last_undegraded":"2024-04-02T14:42:49.376278+0000","last_fullsized":"2024-04-02T14:42:49.376278+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:45:20.094455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.243078+0000","last_change":"2024-04-02T14:41:21.472458+0000","last_active":"2024-04-02T14:42:21.243078+0000","last_peered":"2024-04-02T14:42:21.243078+0000","last_clean":"2024-04-02T14:42:21.243078+0000","last_became_active":"2024-04-02T14:41:21.472105+0000","last_became_peered":"2024-04-02T14:41:21.472105+0000","last_unstale":"2024-04-02T14:42:21.243078+0000","last_undegraded":"2024-04-02T14:42:21.243078+0000","last_fullsized":"2024-04-02T14:42:21.243078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:44.075807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372497+0000","last_change":"2024-04-02T14:41:22.537492+0000","last_active":"2024-04-02T14:42:49.372497+0000","last_peered":"2024-04-02T14:42:49.372497+0000","last_clean":"2024-04-02T14:42:49.372497+0000","last_became_active":"2024-04-02T14:41:22.537067+0000","last_became_peered":"2024-04-02T14:41:22.537067+0000","last_unstale":"2024-04-02T14:42:49.372497+0000","last_undegraded":"2024-04-02T14:42:49.372497+0000","last_fullsized":"2024-04-02T14:42:49.372497+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:08.704340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":115,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:57.288413+0000","last_change":"2024-04-02T14:41:21.466017+0000","last_active":"2024-04-02T14:42:57.288413+0000","last_peered":"2024-04-02T14:42:57.288413+0000","last_clean":"2024-04-02T14:42:57.288413+0000","last_became_active":"2024-04-02T14:41:21.465600+0000","last_became_peered":"2024-04-02T14:41:21.465600+0000","last_unstale":"2024-04-02T14:42:57.288413+0000","last_undegraded":"2024-04-02T14:42:57.288413+0000","last_fullsized":"2024-04-02T14:42:57.288413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:52:08.131125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.354336+0000","last_change":"2024-04-02T14:41:22.526628+0000","last_active":"2024-04-02T14:42:49.354336+0000","last_peered":"2024-04-02T14:42:49.354336+0000","last_clean":"2024-04-02T14:42:49.354336+0000","last_became_active":"2024-04-02T14:41:22.526283+0000","last_became_peered":"2024-04-02T14:41:22.526283+0000","last_unstale":"2024-04-02T14:42:49.354336+0000","last_undegraded":"2024-04-02T14:42:49.354336+0000","last_fullsized":"2024-04-02T14:42:49.354336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:41.812330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.244931+0000","last_change":"2024-04-02T14:41:21.475853+0000","last_active":"2024-04-02T14:42:21.244931+0000","last_peered":"2024-04-02T14:42:21.244931+0000","last_clean":"2024-04-02T14:42:21.244931+0000","last_became_active":"2024-04-02T14:41:21.475555+0000","last_became_peered":"2024-04-02T14:41:21.475555+0000","last_unstale":"2024-04-02T14:42:21.244931+0000","last_undegraded":"2024-04-02T14:42:21.244931+0000","last_fullsized":"2024-04-02T14:42:21.244931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:49.170906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.350247+0000","last_change":"2024-04-02T14:41:22.523848+0000","last_active":"2024-04-02T14:42:49.350247+0000","last_peered":"2024-04-02T14:42:49.350247+0000","last_clean":"2024-04-02T14:42:49.350247+0000","last_became_active":"2024-04-02T14:41:22.523476+0000","last_became_peered":"2024-04-02T14:41:22.523476+0000","last_unstale":"2024-04-02T14:42:49.350247+0000","last_undegraded":"2024-04-02T14:42:49.350247+0000","last_fullsized":"2024-04-02T14:42:49.350247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:47:46.933264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.246237+0000","last_change":"2024-04-02T14:41:21.468556+0000","last_active":"2024-04-02T14:42:21.246237+0000","last_peered":"2024-04-02T14:42:21.246237+0000","last_clean":"2024-04-02T14:42:21.246237+0000","last_became_active":"2024-04-02T14:41:21.468215+0000","last_became_peered":"2024-04-02T14:41:21.468215+0000","last_unstale":"2024-04-02T14:42:21.246237+0000","last_undegraded":"2024-04-02T14:42:21.246237+0000","last_fullsized":"2024-04-02T14:42:21.246237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:04.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.344760+0000","last_change":"2024-04-02T14:41:22.524228+0000","last_active":"2024-04-02T14:42:49.344760+0000","last_peered":"2024-04-02T14:42:49.344760+0000","last_clean":"2024-04-02T14:42:49.344760+0000","last_became_active":"2024-04-02T14:41:22.523839+0000","last_became_peered":"2024-04-02T14:41:22.523839+0000","last_unstale":"2024-04-02T14:42:49.344760+0000","last_undegraded":"2024-04-02T14:42:49.344760+0000","last_fullsized":"2024-04-02T14:42:49.344760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:13.094123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.260431+0000","last_change":"2024-04-02T14:42:21.260431+0000","last_active":"2024-04-02T14:42:21.260431+0000","last_peered":"2024-04-02T14:42:21.260431+0000","last_clean":"2024-04-02T14:42:21.260431+0000","last_became_active":"2024-04-02T14:42:21.259060+0000","last_became_peered":"2024-04-02T14:42:21.259060+0000","last_unstale":"2024-04-02T14:42:21.260431+0000","last_undegraded":"2024-04-02T14:42:21.260431+0000","last_fullsized":"2024-04-02T14:42:21.260431+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:07.753909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370091+0000","last_change":"2024-04-02T14:41:22.542847+0000","last_active":"2024-04-02T14:42:49.370091+0000","last_peered":"2024-04-02T14:42:49.370091+0000","last_clean":"2024-04-02T14:42:49.370091+0000","last_became_active":"2024-04-02T14:41:22.542476+0000","last_became_peered":"2024-04-02T14:41:22.542476+0000","last_unstale":"2024-04-02T14:42:49.370091+0000","last_undegraded":"2024-04-02T14:42:49.370091+0000","last_fullsized":"2024-04-02T14:42:49.370091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:27:12.102931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":115,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:58.281744+0000","last_change":"2024-04-02T14:41:21.483382+0000","last_active":"2024-04-02T14:42:58.281744+0000","last_peered":"2024-04-02T14:42:58.281744+0000","last_clean":"2024-04-02T14:42:58.281744+0000","last_became_active":"2024-04-02T14:41:21.482996+0000","last_became_peered":"2024-04-02T14:41:21.482996+0000","last_unstale":"2024-04-02T14:42:58.281744+0000","last_undegraded":"2024-04-02T14:42:58.281744+0000","last_fullsized":"2024-04-02T14:42:58.281744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:37.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.365155+0000","last_change":"2024-04-02T14:42:21.257441+0000","last_active":"2024-04-02T14:42:49.365155+0000","last_peered":"2024-04-02T14:42:49.365155+0000","last_clean":"2024-04-02T14:42:49.365155+0000","last_became_active":"2024-04-02T14:42:21.257003+0000","last_became_peered":"2024-04-02T14:42:21.257003+0000","last_unstale":"2024-04-02T14:42:49.365155+0000","last_undegraded":"2024-04-02T14:42:49.365155+0000","last_fullsized":"2024-04-02T14:42:49.365155+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:19:10.793720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.346266+0000","last_change":"2024-04-02T14:41:22.525853+0000","last_active":"2024-04-02T14:42:49.346266+0000","last_peered":"2024-04-02T14:42:49.346266+0000","last_clean":"2024-04-02T14:42:49.346266+0000","last_became_active":"2024-04-02T14:41:22.525553+0000","last_became_peered":"2024-04-02T14:41:22.525553+0000","last_unstale":"2024-04-02T14:42:49.346266+0000","last_undegraded":"2024-04-02T14:42:49.346266+0000","last_fullsized":"2024-04-02T14:42:49.346266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:25:30.340974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":115,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:58.289483+0000","last_change":"2024-04-02T14:41:21.490632+0000","last_active":"2024-04-02T14:42:58.289483+0000","last_peered":"2024-04-02T14:42:58.289483+0000","last_clean":"2024-04-02T14:42:58.289483+0000","last_became_active":"2024-04-02T14:41:21.490208+0000","last_became_peered":"2024-04-02T14:41:21.490208+0000","last_unstale":"2024-04-02T14:42:58.289483+0000","last_undegraded":"2024-04-02T14:42:58.289483+0000","last_fullsized":"2024-04-02T14:42:58.289483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:18.566261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.373553+0000","last_change":"2024-04-02T14:41:22.544872+0000","last_active":"2024-04-02T14:42:49.373553+0000","last_peered":"2024-04-02T14:42:49.373553+0000","last_clean":"2024-04-02T14:42:49.373553+0000","last_became_active":"2024-04-02T14:41:22.544268+0000","last_became_peered":"2024-04-02T14:41:22.544268+0000","last_unstale":"2024-04-02T14:42:49.373553+0000","last_undegraded":"2024-04-02T14:42:49.373553+0000","last_fullsized":"2024-04-02T14:42:49.373553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:54.963084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.234967+0000","last_change":"2024-04-02T14:41:21.474411+0000","last_active":"2024-04-02T14:42:21.234967+0000","last_peered":"2024-04-02T14:42:21.234967+0000","last_clean":"2024-04-02T14:42:21.234967+0000","last_became_active":"2024-04-02T14:41:21.473950+0000","last_became_peered":"2024-04-02T14:41:21.473950+0000","last_unstale":"2024-04-02T14:42:21.234967+0000","last_undegraded":"2024-04-02T14:42:21.234967+0000","last_fullsized":"2024-04-02T14:42:21.234967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:46:05.892446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.245632+0000","last_change":"2024-04-02T14:40:03.537813+0000","last_active":"2024-04-02T14:42:21.245632+0000","last_peered":"2024-04-02T14:42:21.245632+0000","last_clean":"2024-04-02T14:42:21.245632+0000","last_became_active":"2024-04-02T14:40:03.537223+0000","last_became_peered":"2024-04-02T14:40:03.537223+0000","last_unstale":"2024-04-02T14:42:21.245632+0000","last_undegraded":"2024-04-02T14:42:21.245632+0000","last_fullsized":"2024-04-02T14:42:21.245632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_clean_scrub_stamp":"2024-04-02T14:36:11.612815+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:19:10.346096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.363055+0000","last_change":"2024-04-02T14:41:22.537906+0000","last_active":"2024-04-02T14:42:49.363055+0000","last_peered":"2024-04-02T14:42:49.363055+0000","last_clean":"2024-04-02T14:42:49.363055+0000","last_became_active":"2024-04-02T14:41:22.537238+0000","last_became_peered":"2024-04-02T14:41:22.537238+0000","last_unstale":"2024-04-02T14:42:49.363055+0000","last_undegraded":"2024-04-02T14:42:49.363055+0000","last_fullsized":"2024-04-02T14:42:49.363055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:57.089563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.245673+0000","last_change":"2024-04-02T14:41:21.466830+0000","last_active":"2024-04-02T14:42:21.245673+0000","last_peered":"2024-04-02T14:42:21.245673+0000","last_clean":"2024-04-02T14:42:21.245673+0000","last_became_active":"2024-04-02T14:41:21.466490+0000","last_became_peered":"2024-04-02T14:41:21.466490+0000","last_unstale":"2024-04-02T14:42:21.245673+0000","last_undegraded":"2024-04-02T14:42:21.245673+0000","last_fullsized":"2024-04-02T14:42:21.245673+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:15.597854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.245278+0000","last_change":"2024-04-02T14:41:21.481139+0000","last_active":"2024-04-02T14:42:21.245278+0000","last_peered":"2024-04-02T14:42:21.245278+0000","last_clean":"2024-04-02T14:42:21.245278+0000","last_became_active":"2024-04-02T14:41:21.480754+0000","last_became_peered":"2024-04-02T14:41:21.480754+0000","last_unstale":"2024-04-02T14:42:21.245278+0000","last_undegraded":"2024-04-02T14:42:21.245278+0000","last_fullsized":"2024-04-02T14:42:21.245278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:03.716223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.373455+0000","last_change":"2024-04-02T14:41:22.548483+0000","last_active":"2024-04-02T14:42:49.373455+0000","last_peered":"2024-04-02T14:42:49.373455+0000","last_clean":"2024-04-02T14:42:49.373455+0000","last_became_active":"2024-04-02T14:41:22.547463+0000","last_became_peered":"2024-04-02T14:41:22.547463+0000","last_unstale":"2024-04-02T14:42:49.373455+0000","last_undegraded":"2024-04-02T14:42:49.373455+0000","last_fullsized":"2024-04-02T14:42:49.373455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:33:58.076844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.244078+0000","last_change":"2024-04-02T14:41:21.491105+0000","last_active":"2024-04-02T14:42:21.244078+0000","last_peered":"2024-04-02T14:42:21.244078+0000","last_clean":"2024-04-02T14:42:21.244078+0000","last_became_active":"2024-04-02T14:41:21.490635+0000","last_became_peered":"2024-04-02T14:41:21.490635+0000","last_unstale":"2024-04-02T14:42:21.244078+0000","last_undegraded":"2024-04-02T14:42:21.244078+0000","last_fullsized":"2024-04-02T14:42:21.244078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:16.531064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.371753+0000","last_change":"2024-04-02T14:41:22.553497+0000","last_active":"2024-04-02T14:42:49.371753+0000","last_peered":"2024-04-02T14:42:49.371753+0000","last_clean":"2024-04-02T14:42:49.371753+0000","last_became_active":"2024-04-02T14:41:22.550474+0000","last_became_peered":"2024-04-02T14:41:22.550474+0000","last_unstale":"2024-04-02T14:42:49.371753+0000","last_undegraded":"2024-04-02T14:42:49.371753+0000","last_fullsized":"2024-04-02T14:42:49.371753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:52.212454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.235007+0000","last_change":"2024-04-02T14:41:21.479516+0000","last_active":"2024-04-02T14:42:21.235007+0000","last_peered":"2024-04-02T14:42:21.235007+0000","last_clean":"2024-04-02T14:42:21.235007+0000","last_became_active":"2024-04-02T14:41:21.478841+0000","last_became_peered":"2024-04-02T14:41:21.478841+0000","last_unstale":"2024-04-02T14:42:21.235007+0000","last_undegraded":"2024-04-02T14:42:21.235007+0000","last_fullsized":"2024-04-02T14:42:21.235007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:39.222316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.374983+0000","last_change":"2024-04-02T14:41:22.550095+0000","last_active":"2024-04-02T14:42:49.374983+0000","last_peered":"2024-04-02T14:42:49.374983+0000","last_clean":"2024-04-02T14:42:49.374983+0000","last_became_active":"2024-04-02T14:41:22.547438+0000","last_became_peered":"2024-04-02T14:41:22.547438+0000","last_unstale":"2024-04-02T14:42:49.374983+0000","last_undegraded":"2024-04-02T14:42:49.374983+0000","last_fullsized":"2024-04-02T14:42:49.374983+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:03:03.668289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.580733+0000","last_change":"2024-04-02T14:41:21.482926+0000","last_active":"2024-04-02T14:42:21.580733+0000","last_peered":"2024-04-02T14:42:21.580733+0000","last_clean":"2024-04-02T14:42:21.580733+0000","last_became_active":"2024-04-02T14:41:21.482517+0000","last_became_peered":"2024-04-02T14:41:21.482517+0000","last_unstale":"2024-04-02T14:42:21.580733+0000","last_undegraded":"2024-04-02T14:42:21.580733+0000","last_fullsized":"2024-04-02T14:42:21.580733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:43.052390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.317999+0000","last_change":"2024-04-02T14:41:22.527758+0000","last_active":"2024-04-02T14:42:49.317999+0000","last_peered":"2024-04-02T14:42:49.317999+0000","last_clean":"2024-04-02T14:42:49.317999+0000","last_became_active":"2024-04-02T14:41:22.526954+0000","last_became_peered":"2024-04-02T14:41:22.526954+0000","last_unstale":"2024-04-02T14:42:49.317999+0000","last_undegraded":"2024-04-02T14:42:49.317999+0000","last_fullsized":"2024-04-02T14:42:49.317999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:34.988661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.238683+0000","last_change":"2024-04-02T14:41:21.487534+0000","last_active":"2024-04-02T14:42:21.238683+0000","last_peered":"2024-04-02T14:42:21.238683+0000","last_clean":"2024-04-02T14:42:21.238683+0000","last_became_active":"2024-04-02T14:41:21.487168+0000","last_became_peered":"2024-04-02T14:41:21.487168+0000","last_unstale":"2024-04-02T14:42:21.238683+0000","last_undegraded":"2024-04-02T14:42:21.238683+0000","last_fullsized":"2024-04-02T14:42:21.238683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:03:20.636028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.372613+0000","last_change":"2024-04-02T14:41:22.539246+0000","last_active":"2024-04-02T14:42:49.372613+0000","last_peered":"2024-04-02T14:42:49.372613+0000","last_clean":"2024-04-02T14:42:49.372613+0000","last_became_active":"2024-04-02T14:41:22.538864+0000","last_became_peered":"2024-04-02T14:41:22.538864+0000","last_unstale":"2024-04-02T14:42:49.372613+0000","last_undegraded":"2024-04-02T14:42:49.372613+0000","last_fullsized":"2024-04-02T14:42:49.372613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:40:13.418752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.246172+0000","last_change":"2024-04-02T14:41:21.477043+0000","last_active":"2024-04-02T14:42:21.246172+0000","last_peered":"2024-04-02T14:42:21.246172+0000","last_clean":"2024-04-02T14:42:21.246172+0000","last_became_active":"2024-04-02T14:41:21.476580+0000","last_became_peered":"2024-04-02T14:41:21.476580+0000","last_unstale":"2024-04-02T14:42:21.246172+0000","last_undegraded":"2024-04-02T14:42:21.246172+0000","last_fullsized":"2024-04-02T14:42:21.246172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:18.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.375880+0000","last_change":"2024-04-02T14:41:22.549758+0000","last_active":"2024-04-02T14:42:49.375880+0000","last_peered":"2024-04-02T14:42:49.375880+0000","last_clean":"2024-04-02T14:42:49.375880+0000","last_became_active":"2024-04-02T14:41:22.545582+0000","last_became_peered":"2024-04-02T14:41:22.545582+0000","last_unstale":"2024-04-02T14:42:49.375880+0000","last_undegraded":"2024-04-02T14:42:49.375880+0000","last_fullsized":"2024-04-02T14:42:49.375880+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:42.311921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.253781+0000","last_change":"2024-04-02T14:42:21.253221+0000","last_active":"2024-04-02T14:42:21.253781+0000","last_peered":"2024-04-02T14:42:21.253781+0000","last_clean":"2024-04-02T14:42:21.253781+0000","last_became_active":"2024-04-02T14:42:21.247890+0000","last_became_peered":"2024-04-02T14:42:21.247890+0000","last_unstale":"2024-04-02T14:42:21.253781+0000","last_undegraded":"2024-04-02T14:42:21.253781+0000","last_fullsized":"2024-04-02T14:42:21.253781+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:12.571769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370984+0000","last_change":"2024-04-02T14:41:22.531509+0000","last_active":"2024-04-02T14:42:49.370984+0000","last_peered":"2024-04-02T14:42:49.370984+0000","last_clean":"2024-04-02T14:42:49.370984+0000","last_became_active":"2024-04-02T14:41:22.531206+0000","last_became_peered":"2024-04-02T14:41:22.531206+0000","last_unstale":"2024-04-02T14:42:49.370984+0000","last_undegraded":"2024-04-02T14:42:49.370984+0000","last_fullsized":"2024-04-02T14:42:49.370984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:39.099177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":117,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:59.292550+0000","last_change":"2024-04-02T14:41:21.485271+0000","last_active":"2024-04-02T14:42:59.292550+0000","last_peered":"2024-04-02T14:42:59.292550+0000","last_clean":"2024-04-02T14:42:59.292550+0000","last_became_active":"2024-04-02T14:41:21.484895+0000","last_became_peered":"2024-04-02T14:41:21.484895+0000","last_unstale":"2024-04-02T14:42:59.292550+0000","last_undegraded":"2024-04-02T14:42:59.292550+0000","last_fullsized":"2024-04-02T14:42:59.292550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.552092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":93,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.357178+0000","last_change":"2024-04-02T14:41:22.524853+0000","last_active":"2024-04-02T14:42:49.357178+0000","last_peered":"2024-04-02T14:42:49.357178+0000","last_clean":"2024-04-02T14:42:49.357178+0000","last_became_active":"2024-04-02T14:41:22.524403+0000","last_became_peered":"2024-04-02T14:41:22.524403+0000","last_unstale":"2024-04-02T14:42:49.357178+0000","last_undegraded":"2024-04-02T14:42:49.357178+0000","last_fullsized":"2024-04-02T14:42:49.357178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:31.322626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.242989+0000","last_change":"2024-04-02T14:41:21.474340+0000","last_active":"2024-04-02T14:42:21.242989+0000","last_peered":"2024-04-02T14:42:21.242989+0000","last_clean":"2024-04-02T14:42:21.242989+0000","last_became_active":"2024-04-02T14:41:21.473934+0000","last_became_peered":"2024-04-02T14:41:21.473934+0000","last_unstale":"2024-04-02T14:42:21.242989+0000","last_undegraded":"2024-04-02T14:42:21.242989+0000","last_fullsized":"2024-04-02T14:42:21.242989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:28:42.711817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370321+0000","last_change":"2024-04-02T14:41:22.543018+0000","last_active":"2024-04-02T14:42:49.370321+0000","last_peered":"2024-04-02T14:42:49.370321+0000","last_clean":"2024-04-02T14:42:49.370321+0000","last_became_active":"2024-04-02T14:41:22.542621+0000","last_became_peered":"2024-04-02T14:41:22.542621+0000","last_unstale":"2024-04-02T14:42:49.370321+0000","last_undegraded":"2024-04-02T14:42:49.370321+0000","last_fullsized":"2024-04-02T14:42:49.370321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:55:03.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:21.583362+0000","last_change":"2024-04-02T14:41:21.493060+0000","last_active":"2024-04-02T14:42:21.583362+0000","last_peered":"2024-04-02T14:42:21.583362+0000","last_clean":"2024-04-02T14:42:21.583362+0000","last_became_active":"2024-04-02T14:41:21.492627+0000","last_became_peered":"2024-04-02T14:41:21.492627+0000","last_unstale":"2024-04-02T14:42:21.583362+0000","last_undegraded":"2024-04-02T14:42:21.583362+0000","last_fullsized":"2024-04-02T14:42:21.583362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:29:10.443866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.348340+0000","last_change":"2024-04-02T14:41:22.527742+0000","last_active":"2024-04-02T14:42:49.348340+0000","last_peered":"2024-04-02T14:42:49.348340+0000","last_clean":"2024-04-02T14:42:49.348340+0000","last_became_active":"2024-04-02T14:41:22.527436+0000","last_became_peered":"2024-04-02T14:41:22.527436+0000","last_unstale":"2024-04-02T14:42:49.348340+0000","last_undegraded":"2024-04-02T14:42:49.348340+0000","last_fullsized":"2024-04-02T14:42:49.348340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:04:24.754907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.348168+0000","last_change":"2024-04-02T14:41:22.526235+0000","last_active":"2024-04-02T14:42:49.348168+0000","last_peered":"2024-04-02T14:42:49.348168+0000","last_clean":"2024-04-02T14:42:49.348168+0000","last_became_active":"2024-04-02T14:41:22.525879+0000","last_became_peered":"2024-04-02T14:41:22.525879+0000","last_unstale":"2024-04-02T14:42:49.348168+0000","last_undegraded":"2024-04-02T14:42:49.348168+0000","last_fullsized":"2024-04-02T14:42:49.348168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:20:54.431223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.344861+0000","last_change":"2024-04-02T14:41:22.522498+0000","last_active":"2024-04-02T14:42:49.344861+0000","last_peered":"2024-04-02T14:42:49.344861+0000","last_clean":"2024-04-02T14:42:49.344861+0000","last_became_active":"2024-04-02T14:41:22.522162+0000","last_became_peered":"2024-04-02T14:41:22.522162+0000","last_unstale":"2024-04-02T14:42:49.344861+0000","last_undegraded":"2024-04-02T14:42:49.344861+0000","last_fullsized":"2024-04-02T14:42:49.344861+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:42:04.701228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.370821+0000","last_change":"2024-04-02T14:41:22.543920+0000","last_active":"2024-04-02T14:42:49.370821+0000","last_peered":"2024-04-02T14:42:49.370821+0000","last_clean":"2024-04-02T14:42:49.370821+0000","last_became_active":"2024-04-02T14:41:22.543552+0000","last_became_peered":"2024-04-02T14:41:22.543552+0000","last_unstale":"2024-04-02T14:42:49.370821+0000","last_undegraded":"2024-04-02T14:42:49.370821+0000","last_fullsized":"2024-04-02T14:42:49.370821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:28.761770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.377002+0000","last_change":"2024-04-02T14:41:22.550507+0000","last_active":"2024-04-02T14:42:49.377002+0000","last_peered":"2024-04-02T14:42:49.377002+0000","last_clean":"2024-04-02T14:42:49.377002+0000","last_became_active":"2024-04-02T14:41:22.545824+0000","last_became_peered":"2024-04-02T14:41:22.545824+0000","last_unstale":"2024-04-02T14:42:49.377002+0000","last_undegraded":"2024-04-02T14:42:49.377002+0000","last_fullsized":"2024-04-02T14:42:49.377002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:11.175138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.377015+0000","last_change":"2024-04-02T14:41:22.541362+0000","last_active":"2024-04-02T14:42:49.377015+0000","last_peered":"2024-04-02T14:42:49.377015+0000","last_clean":"2024-04-02T14:42:49.377015+0000","last_became_active":"2024-04-02T14:41:22.540927+0000","last_became_peered":"2024-04-02T14:41:22.540927+0000","last_unstale":"2024-04-02T14:42:49.377015+0000","last_undegraded":"2024-04-02T14:42:49.377015+0000","last_fullsized":"2024-04-02T14:42:49.377015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:43:00.746274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.344216+0000","last_change":"2024-04-02T14:41:22.527146+0000","last_active":"2024-04-02T14:42:49.344216+0000","last_peered":"2024-04-02T14:42:49.344216+0000","last_clean":"2024-04-02T14:42:49.344216+0000","last_became_active":"2024-04-02T14:41:22.526810+0000","last_became_peered":"2024-04-02T14:41:22.526810+0000","last_unstale":"2024-04-02T14:42:49.344216+0000","last_undegraded":"2024-04-02T14:42:49.344216+0000","last_fullsized":"2024-04-02T14:42:49.344216+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:18:46.464701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.335577+0000","last_change":"2024-04-02T14:41:22.524458+0000","last_active":"2024-04-02T14:42:49.335577+0000","last_peered":"2024-04-02T14:42:49.335577+0000","last_clean":"2024-04-02T14:42:49.335577+0000","last_became_active":"2024-04-02T14:41:22.524138+0000","last_became_peered":"2024-04-02T14:41:22.524138+0000","last_unstale":"2024-04-02T14:42:49.335577+0000","last_undegraded":"2024-04-02T14:42:49.335577+0000","last_fullsized":"2024-04-02T14:42:49.335577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:50:04.667964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.364010+0000","last_change":"2024-04-02T14:42:21.251102+0000","last_active":"2024-04-02T14:42:49.364010+0000","last_peered":"2024-04-02T14:42:49.364010+0000","last_clean":"2024-04-02T14:42:49.364010+0000","last_became_active":"2024-04-02T14:42:21.249134+0000","last_became_peered":"2024-04-02T14:42:21.249134+0000","last_unstale":"2024-04-02T14:42:49.364010+0000","last_undegraded":"2024-04-02T14:42:49.364010+0000","last_fullsized":"2024-04-02T14:42:49.364010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:05:56.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.360587+0000","last_change":"2024-04-02T14:41:22.542319+0000","last_active":"2024-04-02T14:42:49.360587+0000","last_peered":"2024-04-02T14:42:49.360587+0000","last_clean":"2024-04-02T14:42:49.360587+0000","last_became_active":"2024-04-02T14:41:22.541959+0000","last_became_peered":"2024-04-02T14:41:22.541959+0000","last_unstale":"2024-04-02T14:42:49.360587+0000","last_undegraded":"2024-04-02T14:42:49.360587+0000","last_fullsized":"2024-04-02T14:42:49.360587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:49.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.373377+0000","last_change":"2024-04-02T14:41:22.549932+0000","last_active":"2024-04-02T14:42:49.373377+0000","last_peered":"2024-04-02T14:42:49.373377+0000","last_clean":"2024-04-02T14:42:49.373377+0000","last_became_active":"2024-04-02T14:41:22.549231+0000","last_became_peered":"2024-04-02T14:41:22.549231+0000","last_unstale":"2024-04-02T14:42:49.373377+0000","last_undegraded":"2024-04-02T14:42:49.373377+0000","last_fullsized":"2024-04-02T14:42:49.373377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:01.485343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.374119+0000","last_change":"2024-04-02T14:41:22.544712+0000","last_active":"2024-04-02T14:42:49.374119+0000","last_peered":"2024-04-02T14:42:49.374119+0000","last_clean":"2024-04-02T14:42:49.374119+0000","last_became_active":"2024-04-02T14:41:22.544043+0000","last_became_peered":"2024-04-02T14:41:22.544043+0000","last_unstale":"2024-04-02T14:42:49.374119+0000","last_undegraded":"2024-04-02T14:42:49.374119+0000","last_fullsized":"2024-04-02T14:42:49.374119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:07.931377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.377733+0000","last_change":"2024-04-02T14:41:22.542364+0000","last_active":"2024-04-02T14:42:49.377733+0000","last_peered":"2024-04-02T14:42:49.377733+0000","last_clean":"2024-04-02T14:42:49.377733+0000","last_became_active":"2024-04-02T14:41:22.541929+0000","last_became_peered":"2024-04-02T14:41:22.541929+0000","last_unstale":"2024-04-02T14:42:49.377733+0000","last_undegraded":"2024-04-02T14:42:49.377733+0000","last_fullsized":"2024-04-02T14:42:49.377733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:45.716132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T14:42:49.344955+0000","last_change":"2024-04-02T14:41:22.525295+0000","last_active":"2024-04-02T14:42:49.344955+0000","last_peered":"2024-04-02T14:42:49.344955+0000","last_clean":"2024-04-02T14:42:49.344955+0000","last_became_active":"2024-04-02T14:41:22.524484+0000","last_became_peered":"2024-04-02T14:41:22.524484+0000","last_unstale":"2024-04-02T14:42:49.344955+0000","last_undegraded":"2024-04-02T14:42:49.344955+0000","last_fullsized":"2024-04-02T14:42:49.344955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:01.819082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65530,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":932,"num_read_kb":541,"num_write":243,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":196590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285601,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":469191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547238,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1118796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710764,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":469121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972403,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":469499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1062158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430269,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":468838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":528327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":470177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":527925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117069,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":468663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855434,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":469853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1061836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":59926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:43:04.065 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-02T14:43:04.065 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-02T14:43:04.065 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-02T14:43:04.065 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph health --format=json 2024-04-02T14:43:04.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:04 smithi062 ceph-mon[32116]: from='client.24905 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:43:04.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:04 smithi204 ceph-mon[35948]: from='client.24905 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:43:04.574 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:43:04.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:04 smithi113 ceph-mon[36135]: from='client.24905 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:43:05.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:05 smithi204 ceph-mon[35948]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:43:05.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:05 smithi062 ceph-mon[32116]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:43:05.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:05 smithi113 ceph-mon[36135]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:43:06.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.987+0000 7f3d0c400700 1 -- 172.21.15.62:0/2828054854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d0406a7e0 msgr2=0x7f3d0406ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:06.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.987+0000 7f3d0c400700 1 --2- 172.21.15.62:0/2828054854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d0406a7e0 0x7f3d0406ac40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009990 tx=0x7f3cf0009ca0 comp rx=0 tx=0).stop 2024-04-02T14:43:06.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.988+0000 7f3d0c400700 1 -- 172.21.15.62:0/2828054854 shutdown_connections 2024-04-02T14:43:06.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.988+0000 7f3d0c400700 1 --2- 172.21.15.62:0/2828054854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d041074e0 0x7f3d0406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:06.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.988+0000 7f3d0c400700 1 --2- 172.21.15.62:0/2828054854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d04104ff0 0x7f3d04105480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:06.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.988+0000 7f3d0c400700 1 --2- 172.21.15.62:0/2828054854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d0406a7e0 0x7f3d0406ac40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:06.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.988+0000 7f3d0c400700 1 -- 172.21.15.62:0/2828054854 >> 172.21.15.62:0/2828054854 conn(0x7f3d04078420 msgr2=0x7f3d04078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:06.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.988+0000 7f3d0c400700 1 -- 172.21.15.62:0/2828054854 shutdown_connections 2024-04-02T14:43:06.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.988+0000 7f3d0c400700 1 -- 172.21.15.62:0/2828054854 wait complete. 2024-04-02T14:43:06.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.989+0000 7f3d0c400700 1 Processor -- start 2024-04-02T14:43:06.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.989+0000 7f3d0c400700 1 -- start start 2024-04-02T14:43:06.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.989+0000 7f3d0c400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d0406a7e0 0x7f3d0419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:06.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.989+0000 7f3d0c400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d04104ff0 0x7f3d0419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:06.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.989+0000 7f3d0c400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d041074e0 0x7f3d041a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:06.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.989+0000 7f3d0c400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04119010 con 0x7f3d0406a7e0 2024-04-02T14:43:06.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.989+0000 7f3d0c400700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04118e90 con 0x7f3d04104ff0 2024-04-02T14:43:06.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.989+0000 7f3d0c400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04119190 con 0x7f3d041074e0 2024-04-02T14:43:06.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d09a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d0406a7e0 0x7f3d0419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:06.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d09a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d0406a7e0 0x7f3d0419f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42966/0 (socket says 172.21.15.62:42966) 2024-04-02T14:43:06.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d09a00700 1 -- 172.21.15.62:0/3405574996 learned_addr learned my addr 172.21.15.62:0/3405574996 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:06.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d0a400700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d041074e0 0x7f3d041a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:06.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d09000700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d04104ff0 0x7f3d0419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:06.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d09a00700 1 -- 172.21.15.62:0/3405574996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d041074e0 msgr2=0x7f3d041a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:06.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d09a00700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d041074e0 0x7f3d041a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:06.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d09a00700 1 -- 172.21.15.62:0/3405574996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d04104ff0 msgr2=0x7f3d0419f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:06.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d09a00700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d04104ff0 0x7f3d0419f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:06.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d09a00700 1 -- 172.21.15.62:0/3405574996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d041a4460 con 0x7f3d0406a7e0 2024-04-02T14:43:06.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d0a400700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d041074e0 0x7f3d041a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:06.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.990+0000 7f3d09a00700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d0406a7e0 0x7f3d0419f4a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80060b0 tx=0x7f3cf800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:06.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.991+0000 7f3d02000700 1 -- 172.21.15.62:0/3405574996 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf801a070 con 0x7f3d0406a7e0 2024-04-02T14:43:06.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.991+0000 7f3d02000700 1 -- 172.21.15.62:0/3405574996 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3cf800dd20 con 0x7f3d0406a7e0 2024-04-02T14:43:06.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.991+0000 7f3d02000700 1 -- 172.21.15.62:0/3405574996 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8014790 con 0x7f3d0406a7e0 2024-04-02T14:43:06.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.991+0000 7f3d0c400700 1 -- 172.21.15.62:0/3405574996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d041a4750 con 0x7f3d0406a7e0 2024-04-02T14:43:06.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.991+0000 7f3d0c400700 1 -- 172.21.15.62:0/3405574996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d041abfd0 con 0x7f3d0406a7e0 2024-04-02T14:43:06.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.992+0000 7f3d0c400700 1 -- 172.21.15.62:0/3405574996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d040692b0 con 0x7f3d0406a7e0 2024-04-02T14:43:06.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.994+0000 7f3d02000700 1 -- 172.21.15.62:0/3405574996 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3cf801d070 con 0x7f3d0406a7e0 2024-04-02T14:43:06.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.996+0000 7f3d02000700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3cec0a3cc0 0x7f3cec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:06.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.997+0000 7f3d02000700 1 -- 172.21.15.62:0/3405574996 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f3cf80cfab0 con 0x7f3d0406a7e0 2024-04-02T14:43:06.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.997+0000 7f3d09000700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3cec0a3cc0 0x7f3cec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:06.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:06.997+0000 7f3d09000700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3cec0a3cc0 0x7f3cec0a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00098e0 tx=0x7f3cf0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:06.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.001+0000 7f3d02000700 1 -- 172.21.15.62:0/3405574996 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3cf8010070 con 0x7f3d0406a7e0 2024-04-02T14:43:07.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.287+0000 7f3d0c400700 1 -- 172.21.15.62:0/3405574996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3d041a07b0 con 0x7f3d0406a7e0 2024-04-02T14:43:07.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.288+0000 7f3d02000700 1 -- 172.21.15.62:0/3405574996 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3cf809c460 con 0x7f3d0406a7e0 2024-04-02T14:43:07.287 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:07.287 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-02T14:43:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 -- 172.21.15.62:0/3405574996 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3cec0a3cc0 msgr2=0x7f3cec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3cec0a3cc0 0x7f3cec0a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00098e0 tx=0x7f3cf0005960 comp rx=0 tx=0).stop 2024-04-02T14:43:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 -- 172.21.15.62:0/3405574996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d0406a7e0 msgr2=0x7f3d0419f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d0406a7e0 0x7f3d0419f4a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80060b0 tx=0x7f3cf800d3b0 comp rx=0 tx=0).stop 2024-04-02T14:43:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 -- 172.21.15.62:0/3405574996 shutdown_connections 2024-04-02T14:43:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3cec0a3cc0 0x7f3cec0a6180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d041074e0 0x7f3d041a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d0406a7e0 0x7f3d0419f4a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:07.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 --2- 172.21.15.62:0/3405574996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d04104ff0 0x7f3d0419f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:07.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 -- 172.21.15.62:0/3405574996 >> 172.21.15.62:0/3405574996 conn(0x7f3d04078420 msgr2=0x7f3d04103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:07.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 -- 172.21.15.62:0/3405574996 shutdown_connections 2024-04-02T14:43:07.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:07.291+0000 7f3d0c400700 1 -- 172.21.15.62:0/3405574996 wait complete. 2024-04-02T14:43:07.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:07 smithi204 ceph-mon[35948]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:43:07.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:07 smithi062 ceph-mon[32116]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:43:07.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:07 smithi113 ceph-mon[36135]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:43:08.271 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-02T14:43:08.271 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:43:08.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.508+0000 7fab62a00640 1 -- 172.21.15.62:0/4169615152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5c1012a0 msgr2=0x7fab5c101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:08.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.508+0000 7fab62a00640 1 --2- 172.21.15.62:0/4169615152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5c1012a0 0x7fab5c101720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab54009ca0 tx=0x7fab5402c100 comp rx=0 tx=0).stop 2024-04-02T14:43:08.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.508+0000 7fab62a00640 1 -- 172.21.15.62:0/4169615152 shutdown_connections 2024-04-02T14:43:08.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.508+0000 7fab62a00640 1 --2- 172.21.15.62:0/4169615152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5c1012a0 0x7fab5c101720 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.508+0000 7fab62a00640 1 --2- 172.21.15.62:0/4169615152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab5c067f80 0x7fab5c100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.508+0000 7fab62a00640 1 --2- 172.21.15.62:0/4169615152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab5c0ffb70 0x7fab5c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.508+0000 7fab62a00640 1 -- 172.21.15.62:0/4169615152 >> 172.21.15.62:0/4169615152 conn(0x7fab5c0fb0e0 msgr2=0x7fab5c0fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:08.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.508+0000 7fab62a00640 1 -- 172.21.15.62:0/4169615152 shutdown_connections 2024-04-02T14:43:08.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.508+0000 7fab62a00640 1 -- 172.21.15.62:0/4169615152 wait complete. 2024-04-02T14:43:08.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:08 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3405574996' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab62a00640 1 Processor -- start 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab62a00640 1 -- start start 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab62a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5c067f80 0x7fab5c104c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab62a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab5c0ffb70 0x7fab5c103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab62a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab5c1012a0 0x7fab5c103790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab62a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c10c620 con 0x7fab5c0ffb70 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab62a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c10c4a0 con 0x7fab5c1012a0 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab62a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c10c7a0 con 0x7fab5c067f80 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab61600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5c067f80 0x7fab5c104c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab61600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5c067f80 0x7fab5c104c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:41326/0 (socket says 172.21.15.62:41326) 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab60c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab5c0ffb70 0x7fab5c103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:08.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab61600640 1 -- 172.21.15.62:0/192063385 learned_addr learned my addr 172.21.15.62:0/192063385 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.510+0000 7fab62000640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab5c1012a0 0x7fab5c103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.511+0000 7fab61600640 1 -- 172.21.15.62:0/192063385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab5c1012a0 msgr2=0x7fab5c103790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.511+0000 7fab61600640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab5c1012a0 0x7fab5c103790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.511+0000 7fab61600640 1 -- 172.21.15.62:0/192063385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab5c0ffb70 msgr2=0x7fab5c103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.511+0000 7fab61600640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab5c0ffb70 0x7fab5c103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.511+0000 7fab61600640 1 -- 172.21.15.62:0/192063385 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c104020 con 0x7fab5c067f80 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.511+0000 7fab60c00640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab5c0ffb70 0x7fab5c103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.511+0000 7fab61600640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5c067f80 0x7fab5c104c10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fab5000e9c0 tx=0x7fab5000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.511+0000 7fab5a000640 1 -- 172.21.15.62:0/192063385 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5000cd80 con 0x7fab5c067f80 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.511+0000 7fab5a000640 1 -- 172.21.15.62:0/192063385 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fab500035f0 con 0x7fab5c067f80 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.511+0000 7fab62a00640 1 -- 172.21.15.62:0/192063385 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5c1a1dd0 con 0x7fab5c067f80 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.511+0000 7fab5a000640 1 -- 172.21.15.62:0/192063385 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50010690 con 0x7fab5c067f80 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.512+0000 7fab62a00640 1 -- 172.21.15.62:0/192063385 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5c1a21e0 con 0x7fab5c067f80 2024-04-02T14:43:08.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.513+0000 7fab62a00640 1 -- 172.21.15.62:0/192063385 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5c1089c0 con 0x7fab5c067f80 2024-04-02T14:43:08.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.515+0000 7fab5a000640 1 -- 172.21.15.62:0/192063385 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fab500037a0 con 0x7fab5c067f80 2024-04-02T14:43:08.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.516+0000 7fab5a000640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fab440a3bb0 0x7fab440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:08.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.516+0000 7fab5a000640 1 -- 172.21.15.62:0/192063385 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fab500d07b0 con 0x7fab5c067f80 2024-04-02T14:43:08.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.516+0000 7fab60c00640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fab440a3bb0 0x7fab440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:08.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.517+0000 7fab60c00640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fab440a3bb0 0x7fab440a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab480095f0 tx=0x7fab48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:08.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.518+0000 7fab5a000640 1 -- 172.21.15.62:0/192063385 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fab5009d160 con 0x7fab5c067f80 2024-04-02T14:43:08.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:08 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3405574996' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T14:43:08.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:08 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3405574996' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T14:43:08.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.729+0000 7fab62a00640 1 -- 172.21.15.62:0/192063385 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab5c02d620 con 0x7fab5c067f80 2024-04-02T14:43:08.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.731+0000 7fab5a000640 1 -- 172.21.15.62:0/192063385 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22583 (secure 0 0 0) 0x7fab5009ca60 con 0x7fab5c067f80 2024-04-02T14:43:08.729 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:42:21.228892+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.734+0000 7fab62a00640 1 -- 172.21.15.62:0/192063385 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fab440a3bb0 msgr2=0x7fab440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.734+0000 7fab62a00640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fab440a3bb0 0x7fab440a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab480095f0 tx=0x7fab48005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.734+0000 7fab62a00640 1 -- 172.21.15.62:0/192063385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5c067f80 msgr2=0x7fab5c104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.734+0000 7fab62a00640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5c067f80 0x7fab5c104c10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fab5000e9c0 tx=0x7fab5000ee80 comp rx=0 tx=0).stop 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.734+0000 7fab62a00640 1 -- 172.21.15.62:0/192063385 shutdown_connections 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.734+0000 7fab62a00640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fab440a3bb0 0x7fab440a6070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.734+0000 7fab62a00640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab5c1012a0 0x7fab5c103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.734+0000 7fab62a00640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab5c0ffb70 0x7fab5c103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.734+0000 7fab62a00640 1 --2- 172.21.15.62:0/192063385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5c067f80 0x7fab5c104c10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.734+0000 7fab62a00640 1 -- 172.21.15.62:0/192063385 >> 172.21.15.62:0/192063385 conn(0x7fab5c0fb0e0 msgr2=0x7fab5c102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.734+0000 7fab62a00640 1 -- 172.21.15.62:0/192063385 shutdown_connections 2024-04-02T14:43:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.735+0000 7fab62a00640 1 -- 172.21.15.62:0/192063385 wait complete. 2024-04-02T14:43:08.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T14:41:20.030393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T14:41:20.917405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:43:08.752 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:43:08.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.982+0000 7f7af9c00640 1 -- 172.21.15.62:0/3903792165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7af4107f20 msgr2=0x7f7af4108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:08.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.982+0000 7f7af9c00640 1 --2- 172.21.15.62:0/3903792165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7af4107f20 0x7f7af4108340 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009900 tx=0x7f7ae402bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:08.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.982+0000 7f7af9c00640 1 -- 172.21.15.62:0/3903792165 shutdown_connections 2024-04-02T14:43:08.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.982+0000 7f7af9c00640 1 --2- 172.21.15.62:0/3903792165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4109a30 0x7f7af4110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.982+0000 7f7af9c00640 1 --2- 172.21.15.62:0/3903792165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af4109070 0x7f7af41094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.982+0000 7f7af9c00640 1 --2- 172.21.15.62:0/3903792165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7af4107f20 0x7f7af4108340 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.982+0000 7f7af9c00640 1 -- 172.21.15.62:0/3903792165 >> 172.21.15.62:0/3903792165 conn(0x7f7af4076ec0 msgr2=0x7f7af4079300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:08.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.983+0000 7f7af9c00640 1 -- 172.21.15.62:0/3903792165 shutdown_connections 2024-04-02T14:43:08.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.983+0000 7f7af9c00640 1 -- 172.21.15.62:0/3903792165 wait complete. 2024-04-02T14:43:08.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.984+0000 7f7af9c00640 1 Processor -- start 2024-04-02T14:43:08.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.984+0000 7f7af9c00640 1 -- start start 2024-04-02T14:43:08.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.984+0000 7f7af9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4107f20 0x7f7af419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.984+0000 7f7af9c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af4109070 0x7f7af419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.984+0000 7f7af9c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7af4109a30 0x7f7af41a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.984+0000 7f7af9c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4113cc0 con 0x7f7af4107f20 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.984+0000 7f7af9c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4113b40 con 0x7f7af4109a30 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.984+0000 7f7af9c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4113e40 con 0x7f7af4109070 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.985+0000 7f7af3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4107f20 0x7f7af419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.985+0000 7f7af3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4107f20 0x7f7af419e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42996/0 (socket says 172.21.15.62:42996) 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.985+0000 7f7af3e00640 1 -- 172.21.15.62:0/2803782076 learned_addr learned my addr 172.21.15.62:0/2803782076 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.985+0000 7f7af3400640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af4109070 0x7f7af419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.985+0000 7f7af9200640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7af4109a30 0x7f7af41a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.985+0000 7f7af3e00640 1 -- 172.21.15.62:0/2803782076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af4109070 msgr2=0x7f7af419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.985+0000 7f7af3e00640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af4109070 0x7f7af419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.985+0000 7f7af3e00640 1 -- 172.21.15.62:0/2803782076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7af4109a30 msgr2=0x7f7af41a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.985+0000 7f7af3e00640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7af4109a30 0x7f7af41a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.985+0000 7f7af3e00640 1 -- 172.21.15.62:0/2803782076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af41a5f20 con 0x7f7af4107f20 2024-04-02T14:43:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.986+0000 7f7af3e00640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4107f20 0x7f7af419e300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009230 tx=0x7f7ae4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.986+0000 7f7af0c00640 1 -- 172.21.15.62:0/2803782076 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4043020 con 0x7f7af4107f20 2024-04-02T14:43:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.986+0000 7f7af9c00640 1 -- 172.21.15.62:0/2803782076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af41a61b0 con 0x7f7af4107f20 2024-04-02T14:43:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.986+0000 7f7af0c00640 1 -- 172.21.15.62:0/2803782076 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7ae4004250 con 0x7f7af4107f20 2024-04-02T14:43:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.986+0000 7f7af0c00640 1 -- 172.21.15.62:0/2803782076 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4003b50 con 0x7f7af4107f20 2024-04-02T14:43:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.986+0000 7f7af9c00640 1 -- 172.21.15.62:0/2803782076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af41a6750 con 0x7f7af4107f20 2024-04-02T14:43:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.987+0000 7f7af9c00640 1 -- 172.21.15.62:0/2803782076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af410f230 con 0x7f7af4107f20 2024-04-02T14:43:08.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.989+0000 7f7af0c00640 1 -- 172.21.15.62:0/2803782076 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7ae4046020 con 0x7f7af4107f20 2024-04-02T14:43:08.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.991+0000 7f7af0c00640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7ad80a3c30 0x7f7ad80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:08.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.992+0000 7f7af0c00640 1 -- 172.21.15.62:0/2803782076 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f7ae40f2440 con 0x7f7af4107f20 2024-04-02T14:43:08.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.992+0000 7f7af3400640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7ad80a3c30 0x7f7ad80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:08.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.992+0000 7f7af3400640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7ad80a3c30 0x7f7ad80a60f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7af419f810 tx=0x7f7ae0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:08.996+0000 7f7af0c00640 1 -- 172.21.15.62:0/2803782076 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7ae404d020 con 0x7f7af4107f20 2024-04-02T14:43:09.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.203+0000 7f7af9c00640 1 -- 172.21.15.62:0/2803782076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7af402d590 con 0x7f7af4107f20 2024-04-02T14:43:09.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.204+0000 7f7af0c00640 1 -- 172.21.15.62:0/2803782076 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f7ae40bee50 con 0x7f7af4107f20 2024-04-02T14:43:09.202 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:43:09.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 -- 172.21.15.62:0/2803782076 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7ad80a3c30 msgr2=0x7f7ad80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:09.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7ad80a3c30 0x7f7ad80a60f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7af419f810 tx=0x7f7ae0005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:09.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 -- 172.21.15.62:0/2803782076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4107f20 msgr2=0x7f7af419e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:09.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4107f20 0x7f7af419e300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009230 tx=0x7f7ae4004ab0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 -- 172.21.15.62:0/2803782076 shutdown_connections 2024-04-02T14:43:09.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7ad80a3c30 0x7f7ad80a60f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7af4109a30 0x7f7af41a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af4109070 0x7f7af419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 --2- 172.21.15.62:0/2803782076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4107f20 0x7f7af419e300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 -- 172.21.15.62:0/2803782076 >> 172.21.15.62:0/2803782076 conn(0x7f7af4076ec0 msgr2=0x7f7af410c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:09.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 -- 172.21.15.62:0/2803782076 shutdown_connections 2024-04-02T14:43:09.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.207+0000 7f7af9c00640 1 -- 172.21.15.62:0/2803782076 wait complete. 2024-04-02T14:43:09.226 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T14:43:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.458+0000 7f06c9a00640 1 -- 172.21.15.62:0/1131081477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06c4109bd0 msgr2=0x7f06c4110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.458+0000 7f06c9a00640 1 --2- 172.21.15.62:0/1131081477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06c4109bd0 0x7f06c4110460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009be0 tx=0x7f06bc02bef0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.458+0000 7f06c9a00640 1 -- 172.21.15.62:0/1131081477 shutdown_connections 2024-04-02T14:43:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.458+0000 7f06c9a00640 1 --2- 172.21.15.62:0/1131081477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06c4109bd0 0x7f06c4110460 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.458+0000 7f06c9a00640 1 --2- 172.21.15.62:0/1131081477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c4109210 0x7f06c4109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.458+0000 7f06c9a00640 1 --2- 172.21.15.62:0/1131081477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c4108020 0x7f06c4108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.458+0000 7f06c9a00640 1 -- 172.21.15.62:0/1131081477 >> 172.21.15.62:0/1131081477 conn(0x7f06c4076fd0 msgr2=0x7f06c40793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.458+0000 7f06c9a00640 1 -- 172.21.15.62:0/1131081477 shutdown_connections 2024-04-02T14:43:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.458+0000 7f06c9a00640 1 -- 172.21.15.62:0/1131081477 wait complete. 2024-04-02T14:43:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.459+0000 7f06c9a00640 1 Processor -- start 2024-04-02T14:43:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c9a00640 1 -- start start 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c9a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c4108020 0x7f06c419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06c4109210 0x7f06c419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c4109bd0 0x7f06c41a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c9a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4113e20 con 0x7f06c4109bd0 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c9a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4113ca0 con 0x7f06c4108020 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c9a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4113fa0 con 0x7f06c4109210 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c4109bd0 0x7f06c41a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06c4109210 0x7f06c419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c4109bd0 0x7f06c41a5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43012/0 (socket says 172.21.15.62:43012) 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06c4109210 0x7f06c419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:41376/0 (socket says 172.21.15.62:41376) 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c9000640 1 -- 172.21.15.62:0/2794661869 learned_addr learned my addr 172.21.15.62:0/2794661869 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.460+0000 7f06c3e00640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c4108020 0x7f06c419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.461+0000 7f06c9000640 1 -- 172.21.15.62:0/2794661869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06c4109210 msgr2=0x7f06c419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.461+0000 7f06c9000640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06c4109210 0x7f06c419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.461+0000 7f06c9000640 1 -- 172.21.15.62:0/2794661869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c4108020 msgr2=0x7f06c419e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.461+0000 7f06c9000640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c4108020 0x7f06c419e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.461+0000 7f06c9000640 1 -- 172.21.15.62:0/2794661869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c41a5f70 con 0x7f06c4109bd0 2024-04-02T14:43:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.461+0000 7f06c9000640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c4109bd0 0x7f06c41a5900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009370 tx=0x7f06bc033820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.461+0000 7f06c0c00640 1 -- 172.21.15.62:0/2794661869 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc043020 con 0x7f06c4109bd0 2024-04-02T14:43:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.461+0000 7f06c9a00640 1 -- 172.21.15.62:0/2794661869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c41a6200 con 0x7f06c4109bd0 2024-04-02T14:43:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.461+0000 7f06c0c00640 1 -- 172.21.15.62:0/2794661869 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f06bc003b80 con 0x7f06c4109bd0 2024-04-02T14:43:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.461+0000 7f06c0c00640 1 -- 172.21.15.62:0/2794661869 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc03e740 con 0x7f06c4109bd0 2024-04-02T14:43:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.461+0000 7f06c9a00640 1 -- 172.21.15.62:0/2794661869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c41a67a0 con 0x7f06c4109bd0 2024-04-02T14:43:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.463+0000 7f06c9a00640 1 -- 172.21.15.62:0/2794661869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c410f270 con 0x7f06c4109bd0 2024-04-02T14:43:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.465+0000 7f06c0c00640 1 -- 172.21.15.62:0/2794661869 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f06bc046020 con 0x7f06c4109bd0 2024-04-02T14:43:09.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.466+0000 7f06c0c00640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06a80a3bb0 0x7f06a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:09.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.466+0000 7f06c0c00640 1 -- 172.21.15.62:0/2794661869 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f06bc0f2a00 con 0x7f06c4109bd0 2024-04-02T14:43:09.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.466+0000 7f06c3e00640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06a80a3bb0 0x7f06a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:09.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.467+0000 7f06c3e00640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06a80a3bb0 0x7f06a80a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f06c4109070 tx=0x7f06b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:09.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.470+0000 7f06c0c00640 1 -- 172.21.15.62:0/2794661869 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f06bc04d020 con 0x7f06c4109bd0 2024-04-02T14:43:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:09 smithi204 ceph-mon[35948]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:09 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/192063385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:09.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:09 smithi062 ceph-mon[32116]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:09.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:09 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/192063385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:09.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:09 smithi113 ceph-mon[36135]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:09.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:09 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/192063385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:09.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.679+0000 7f06c9a00640 1 -- 172.21.15.62:0/2794661869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f06c4062700 con 0x7f06c4109bd0 2024-04-02T14:43:09.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.680+0000 7f06c0c00640 1 -- 172.21.15.62:0/2794661869 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f06bc0bf410 con 0x7f06c4109bd0 2024-04-02T14:43:09.678 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.682+0000 7f06c9a00640 1 -- 172.21.15.62:0/2794661869 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06a80a3bb0 msgr2=0x7f06a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.682+0000 7f06c9a00640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06a80a3bb0 0x7f06a80a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f06c4109070 tx=0x7f06b4005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.682+0000 7f06c9a00640 1 -- 172.21.15.62:0/2794661869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c4109bd0 msgr2=0x7f06c41a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.682+0000 7f06c9a00640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c4109bd0 0x7f06c41a5900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009370 tx=0x7f06bc033820 comp rx=0 tx=0).stop 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.682+0000 7f06c9a00640 1 -- 172.21.15.62:0/2794661869 shutdown_connections 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.683+0000 7f06c9a00640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06a80a3bb0 0x7f06a80a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.683+0000 7f06c9a00640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c4109bd0 0x7f06c41a5900 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.683+0000 7f06c9a00640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06c4109210 0x7f06c419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.683+0000 7f06c9a00640 1 --2- 172.21.15.62:0/2794661869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c4108020 0x7f06c419e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.683+0000 7f06c9a00640 1 -- 172.21.15.62:0/2794661869 >> 172.21.15.62:0/2794661869 conn(0x7f06c4076fd0 msgr2=0x7f06c410c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.683+0000 7f06c9a00640 1 -- 172.21.15.62:0/2794661869 shutdown_connections 2024-04-02T14:43:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.683+0000 7f06c9a00640 1 -- 172.21.15.62:0/2794661869 wait complete. 2024-04-02T14:43:09.703 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T14:43:09.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.929+0000 7fb5bf200640 1 -- 172.21.15.62:0/2867901164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c0103c00 msgr2=0x7fb5c0104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:09.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.929+0000 7fb5bf200640 1 --2- 172.21.15.62:0/2867901164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c0103c00 0x7fb5c0104020 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009860 tx=0x7fb5b002bb70 comp rx=0 tx=0).stop 2024-04-02T14:43:09.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.929+0000 7fb5bf200640 1 -- 172.21.15.62:0/2867901164 shutdown_connections 2024-04-02T14:43:09.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.929+0000 7fb5bf200640 1 --2- 172.21.15.62:0/2867901164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5c01057b0 0x7fb5c010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.929+0000 7fb5bf200640 1 --2- 172.21.15.62:0/2867901164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c0104df0 0x7fb5c0105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.929+0000 7fb5bf200640 1 --2- 172.21.15.62:0/2867901164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c0103c00 0x7fb5c0104020 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.929+0000 7fb5bf200640 1 -- 172.21.15.62:0/2867901164 >> 172.21.15.62:0/2867901164 conn(0x7fb5c00ff170 msgr2=0x7fb5c01015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:09.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.929+0000 7fb5bf200640 1 -- 172.21.15.62:0/2867901164 shutdown_connections 2024-04-02T14:43:09.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.929+0000 7fb5bf200640 1 -- 172.21.15.62:0/2867901164 wait complete. 2024-04-02T14:43:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.930+0000 7fb5bf200640 1 Processor -- start 2024-04-02T14:43:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.930+0000 7fb5bf200640 1 -- start start 2024-04-02T14:43:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5bf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5c0103c00 0x7fb5c0199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5bf200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c0104df0 0x7fb5c019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5bf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c01057b0 0x7fb5c01a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5bf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c010f9b0 con 0x7fb5c01057b0 2024-04-02T14:43:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5bf200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c010f830 con 0x7fb5c0104df0 2024-04-02T14:43:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5bf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c010fb30 con 0x7fb5c0103c00 2024-04-02T14:43:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5b5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c0104df0 0x7fb5c019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5bde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5c0103c00 0x7fb5c0199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5b5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c0104df0 0x7fb5c019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:45556/0 (socket says 172.21.15.62:45556) 2024-04-02T14:43:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5b5400640 1 -- 172.21.15.62:0/2201835477 learned_addr learned my addr 172.21.15.62:0/2201835477 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5be800640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c01057b0 0x7fb5c01a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5b5400640 1 -- 172.21.15.62:0/2201835477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5c0103c00 msgr2=0x7fb5c0199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.931+0000 7fb5b5400640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5c0103c00 0x7fb5c0199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.932+0000 7fb5b5400640 1 -- 172.21.15.62:0/2201835477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c01057b0 msgr2=0x7fb5c01a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.932+0000 7fb5b5400640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c01057b0 0x7fb5c01a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.932+0000 7fb5b5400640 1 -- 172.21.15.62:0/2201835477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c01a1be0 con 0x7fb5c0104df0 2024-04-02T14:43:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.932+0000 7fb5bde00640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5c0103c00 0x7fb5c0199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.932+0000 7fb5b5400640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c0104df0 0x7fb5c019a500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800e9c0 tx=0x7fb5a800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:09.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.932+0000 7fb5b7400640 1 -- 172.21.15.62:0/2201835477 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a800ccc0 con 0x7fb5c0104df0 2024-04-02T14:43:09.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.932+0000 7fb5b7400640 1 -- 172.21.15.62:0/2201835477 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb5a8005520 con 0x7fb5c0104df0 2024-04-02T14:43:09.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.932+0000 7fb5bf200640 1 -- 172.21.15.62:0/2201835477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c01a1ed0 con 0x7fb5c0104df0 2024-04-02T14:43:09.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.932+0000 7fb5b7400640 1 -- 172.21.15.62:0/2201835477 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8010690 con 0x7fb5c0104df0 2024-04-02T14:43:09.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.932+0000 7fb5bf200640 1 -- 172.21.15.62:0/2201835477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c01a2390 con 0x7fb5c0104df0 2024-04-02T14:43:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.933+0000 7fb5bf200640 1 -- 172.21.15.62:0/2201835477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c010aec0 con 0x7fb5c0104df0 2024-04-02T14:43:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.936+0000 7fb5b7400640 1 -- 172.21.15.62:0/2201835477 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb5a80046c0 con 0x7fb5c0104df0 2024-04-02T14:43:09.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.937+0000 7fb5b7400640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5a40a3bb0 0x7fb5a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:09.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.937+0000 7fb5b7400640 1 -- 172.21.15.62:0/2201835477 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fb5a80d0670 con 0x7fb5c0104df0 2024-04-02T14:43:09.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.937+0000 7fb5bde00640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5a40a3bb0 0x7fb5a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:09.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.938+0000 7fb5bde00640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5a40a3bb0 0x7fb5a40a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0034a80 tx=0x7fb5b0034b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:09.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:09.942+0000 7fb5b7400640 1 -- 172.21.15.62:0/2201835477 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb5a809d020 con 0x7fb5c0104df0 2024-04-02T14:43:10.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.148+0000 7fb5bf200640 1 -- 172.21.15.62:0/2201835477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5c0062700 con 0x7fb5c0104df0 2024-04-02T14:43:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.152+0000 7fb5b7400640 1 -- 172.21.15.62:0/2201835477 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fb5a809c920 con 0x7fb5c0104df0 2024-04-02T14:43:10.151 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-02T14:43:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.155+0000 7fb5bf200640 1 -- 172.21.15.62:0/2201835477 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5a40a3bb0 msgr2=0x7fb5a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.155+0000 7fb5bf200640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5a40a3bb0 0x7fb5a40a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0034a80 tx=0x7fb5b0034b50 comp rx=0 tx=0).stop 2024-04-02T14:43:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.155+0000 7fb5bf200640 1 -- 172.21.15.62:0/2201835477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c0104df0 msgr2=0x7fb5c019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.155+0000 7fb5bf200640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c0104df0 0x7fb5c019a500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800e9c0 tx=0x7fb5a800ee80 comp rx=0 tx=0).stop 2024-04-02T14:43:10.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.155+0000 7fb5bf200640 1 -- 172.21.15.62:0/2201835477 shutdown_connections 2024-04-02T14:43:10.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.155+0000 7fb5bf200640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb5a40a3bb0 0x7fb5a40a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.155+0000 7fb5bf200640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c01057b0 0x7fb5c01a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.155+0000 7fb5bf200640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c0104df0 0x7fb5c019a500 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.155+0000 7fb5bf200640 1 --2- 172.21.15.62:0/2201835477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5c0103c00 0x7fb5c0199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.155+0000 7fb5bf200640 1 -- 172.21.15.62:0/2201835477 >> 172.21.15.62:0/2201835477 conn(0x7fb5c00ff170 msgr2=0x7fb5c0108090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:10.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.156+0000 7fb5bf200640 1 -- 172.21.15.62:0/2201835477 shutdown_connections 2024-04-02T14:43:10.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.156+0000 7fb5bf200640 1 -- 172.21.15.62:0/2201835477 wait complete. 2024-04-02T14:43:10.172 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:43:10.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.380+0000 7f1463200640 1 -- 172.21.15.62:0/3635159393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1464107f10 msgr2=0x7f1464108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:10.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.380+0000 7f1463200640 1 --2- 172.21.15.62:0/3635159393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1464107f10 0x7f1464108350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1454009860 tx=0x7f145402bb70 comp rx=0 tx=0).stop 2024-04-02T14:43:10.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.380+0000 7f1463200640 1 -- 172.21.15.62:0/3635159393 shutdown_connections 2024-04-02T14:43:10.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.380+0000 7f1463200640 1 --2- 172.21.15.62:0/3635159393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1464109aa0 0x7f14641103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.380+0000 7f1463200640 1 --2- 172.21.15.62:0/3635159393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14641090e0 0x7f1464109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.380+0000 7f1463200640 1 --2- 172.21.15.62:0/3635159393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1464107f10 0x7f1464108350 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.380+0000 7f1463200640 1 -- 172.21.15.62:0/3635159393 >> 172.21.15.62:0/3635159393 conn(0x7f1464076f10 msgr2=0x7f1464079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:10.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.380+0000 7f1463200640 1 -- 172.21.15.62:0/3635159393 shutdown_connections 2024-04-02T14:43:10.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.380+0000 7f1463200640 1 -- 172.21.15.62:0/3635159393 wait complete. 2024-04-02T14:43:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.381+0000 7f1463200640 1 Processor -- start 2024-04-02T14:43:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.381+0000 7f1463200640 1 -- start start 2024-04-02T14:43:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.381+0000 7f1463200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1464107f10 0x7f146419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.381+0000 7f1463200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14641090e0 0x7f146419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.381+0000 7f1463200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1464109aa0 0x7f14641a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.381+0000 7f1463200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1464113d20 con 0x7f1464109aa0 2024-04-02T14:43:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.381+0000 7f1463200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1464113ba0 con 0x7f1464107f10 2024-04-02T14:43:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.381+0000 7f1463200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1464113ea0 con 0x7f14641090e0 2024-04-02T14:43:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1462800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1464109aa0 0x7f14641a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1462800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1464109aa0 0x7f14641a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43072/0 (socket says 172.21.15.62:43072) 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1461e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1464107f10 0x7f146419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1462800640 1 -- 172.21.15.62:0/1813409253 learned_addr learned my addr 172.21.15.62:0/1813409253 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1461400640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14641090e0 0x7f146419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1462800640 1 -- 172.21.15.62:0/1813409253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14641090e0 msgr2=0x7f146419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1462800640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14641090e0 0x7f146419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1462800640 1 -- 172.21.15.62:0/1813409253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1464107f10 msgr2=0x7f146419e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1462800640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1464107f10 0x7f146419e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1462800640 1 -- 172.21.15.62:0/1813409253 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14641a5f50 con 0x7f1464109aa0 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1461e00640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1464107f10 0x7f146419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.382+0000 7f1462800640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1464109aa0 0x7f14641a58e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f145c00eb10 tx=0x7f145c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.383+0000 7f145aa00640 1 -- 172.21.15.62:0/1813409253 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c00ce20 con 0x7f1464109aa0 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.383+0000 7f1463200640 1 -- 172.21.15.62:0/1813409253 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14641a6240 con 0x7f1464109aa0 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.383+0000 7f145aa00640 1 -- 172.21.15.62:0/1813409253 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f145c004b20 con 0x7f1464109aa0 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.383+0000 7f145aa00640 1 -- 172.21.15.62:0/1813409253 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c010700 con 0x7f1464109aa0 2024-04-02T14:43:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.383+0000 7f1463200640 1 -- 172.21.15.62:0/1813409253 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14641a67e0 con 0x7f1464109aa0 2024-04-02T14:43:10.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.384+0000 7f1463200640 1 -- 172.21.15.62:0/1813409253 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f146410f2a0 con 0x7f1464109aa0 2024-04-02T14:43:10.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.386+0000 7f145aa00640 1 -- 172.21.15.62:0/1813409253 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f145c005020 con 0x7f1464109aa0 2024-04-02T14:43:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.387+0000 7f145aa00640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f14480a3bb0 0x7f14480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.387+0000 7f145aa00640 1 -- 172.21.15.62:0/1813409253 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f145c0d0ac0 con 0x7f1464109aa0 2024-04-02T14:43:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.387+0000 7f1461e00640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f14480a3bb0 0x7f14480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.388+0000 7f1461e00640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f14480a3bb0 0x7f14480a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1454034a80 tx=0x7f1454034b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.392+0000 7f145aa00640 1 -- 172.21.15.62:0/1813409253 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f145c09d470 con 0x7f1464109aa0 2024-04-02T14:43:10.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:10 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2803782076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:10.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:10 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2794661869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:10.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:10 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2803782076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:10.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:10 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2794661869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:10.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:10 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2803782076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:10.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:10 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2794661869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:10.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.633+0000 7f1463200640 1 -- 172.21.15.62:0/1813409253 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1464062700 con 0x7f1464109aa0 2024-04-02T14:43:10.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.634+0000 7f145aa00640 1 -- 172.21.15.62:0/1813409253 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f145c09cd70 con 0x7f1464109aa0 2024-04-02T14:43:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:10.633 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:41:30.090009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24553,24565,24533,24535,24584]},"id":1}]} 2024-04-02T14:43:10.633 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-02T14:43:10.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.637+0000 7f1463200640 1 -- 172.21.15.62:0/1813409253 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f14480a3bb0 msgr2=0x7f14480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:10.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.637+0000 7f1463200640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f14480a3bb0 0x7f14480a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1454034a80 tx=0x7f1454034b50 comp rx=0 tx=0).stop 2024-04-02T14:43:10.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.637+0000 7f1463200640 1 -- 172.21.15.62:0/1813409253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1464109aa0 msgr2=0x7f14641a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:10.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.637+0000 7f1463200640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1464109aa0 0x7f14641a58e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f145c00eb10 tx=0x7f145c00ef10 comp rx=0 tx=0).stop 2024-04-02T14:43:10.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.637+0000 7f1463200640 1 -- 172.21.15.62:0/1813409253 shutdown_connections 2024-04-02T14:43:10.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.637+0000 7f1463200640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f14480a3bb0 0x7f14480a6070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.637+0000 7f1463200640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1464109aa0 0x7f14641a58e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.638+0000 7f1463200640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14641090e0 0x7f146419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.638+0000 7f1463200640 1 --2- 172.21.15.62:0/1813409253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1464107f10 0x7f146419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.638+0000 7f1463200640 1 -- 172.21.15.62:0/1813409253 >> 172.21.15.62:0/1813409253 conn(0x7f1464076f10 msgr2=0x7f146410c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:10.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.638+0000 7f1463200640 1 -- 172.21.15.62:0/1813409253 shutdown_connections 2024-04-02T14:43:10.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.638+0000 7f1463200640 1 -- 172.21.15.62:0/1813409253 wait complete. 2024-04-02T14:43:10.653 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:43:10.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.864+0000 7f6ff7e00640 1 -- 172.21.15.62:0/715291143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ff80744e0 msgr2=0x7f6ff8074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:10.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.864+0000 7f6ff7e00640 1 --2- 172.21.15.62:0/715291143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ff80744e0 0x7f6ff8074940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009900 tx=0x7f6fe002bcc0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.864+0000 7f6ff7e00640 1 -- 172.21.15.62:0/715291143 shutdown_connections 2024-04-02T14:43:10.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.864+0000 7f6ff7e00640 1 --2- 172.21.15.62:0/715291143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ff8109810 0x7f6ff810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.864+0000 7f6ff7e00640 1 --2- 172.21.15.62:0/715291143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ff80744e0 0x7f6ff8074940 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.864+0000 7f6ff7e00640 1 --2- 172.21.15.62:0/715291143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ff8075ac0 0x7f6ff8073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.864+0000 7f6ff7e00640 1 -- 172.21.15.62:0/715291143 >> 172.21.15.62:0/715291143 conn(0x7f6ff80fced0 msgr2=0x7f6ff80ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:10.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.864+0000 7f6ff7e00640 1 -- 172.21.15.62:0/715291143 shutdown_connections 2024-04-02T14:43:10.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.864+0000 7f6ff7e00640 1 -- 172.21.15.62:0/715291143 wait complete. 2024-04-02T14:43:10.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.865+0000 7f6ff7e00640 1 Processor -- start 2024-04-02T14:43:10.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff7e00640 1 -- start start 2024-04-02T14:43:10.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ff80744e0 0x7f6ff8199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:10.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ff8075ac0 0x7f6ff819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:10.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ff8109810 0x7f6ff81a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:10.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff810f920 con 0x7f6ff8109810 2024-04-02T14:43:10.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff810f7a0 con 0x7f6ff80744e0 2024-04-02T14:43:10.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff810faa0 con 0x7f6ff8075ac0 2024-04-02T14:43:10.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ff80744e0 0x7f6ff8199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:10.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ff80744e0 0x7f6ff8199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:45594/0 (socket says 172.21.15.62:45594) 2024-04-02T14:43:10.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff6a00640 1 -- 172.21.15.62:0/1700252946 learned_addr learned my addr 172.21.15.62:0/1700252946 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:10.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff7400640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ff8109810 0x7f6ff81a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:10.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.866+0000 7f6ff6000640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ff8075ac0 0x7f6ff819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:10.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.867+0000 7f6ff6a00640 1 -- 172.21.15.62:0/1700252946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ff8075ac0 msgr2=0x7f6ff819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:10.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.867+0000 7f6ff6a00640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ff8075ac0 0x7f6ff819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.867+0000 7f6ff6a00640 1 -- 172.21.15.62:0/1700252946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ff8109810 msgr2=0x7f6ff81a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:10.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.867+0000 7f6ff6a00640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ff8109810 0x7f6ff81a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:10.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.867+0000 7f6ff6a00640 1 -- 172.21.15.62:0/1700252946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff81a1b70 con 0x7f6ff80744e0 2024-04-02T14:43:10.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.867+0000 7f6ff6a00640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ff80744e0 0x7f6ff8199f50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800eb90 tx=0x7f6fe800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:10.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.867+0000 7f6fef400640 1 -- 172.21.15.62:0/1700252946 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8004d20 con 0x7f6ff80744e0 2024-04-02T14:43:10.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.867+0000 7f6fef400640 1 -- 172.21.15.62:0/1700252946 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6fe800ee90 con 0x7f6ff80744e0 2024-04-02T14:43:10.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.867+0000 7f6ff7e00640 1 -- 172.21.15.62:0/1700252946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff81a1e60 con 0x7f6ff80744e0 2024-04-02T14:43:10.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.868+0000 7f6fef400640 1 -- 172.21.15.62:0/1700252946 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80108b0 con 0x7f6ff80744e0 2024-04-02T14:43:10.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.868+0000 7f6ff7e00640 1 -- 172.21.15.62:0/1700252946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff81a22c0 con 0x7f6ff80744e0 2024-04-02T14:43:10.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.869+0000 7f6ff7e00640 1 -- 172.21.15.62:0/1700252946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff81085d0 con 0x7f6ff80744e0 2024-04-02T14:43:10.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.871+0000 7f6fef400640 1 -- 172.21.15.62:0/1700252946 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6fe8010a50 con 0x7f6ff80744e0 2024-04-02T14:43:10.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.872+0000 7f6fef400640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6fdc0a3c30 0x7f6fdc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:10.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.873+0000 7f6fef400640 1 -- 172.21.15.62:0/1700252946 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f6fe80d0bc0 con 0x7f6ff80744e0 2024-04-02T14:43:10.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.873+0000 7f6ff6000640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6fdc0a3c30 0x7f6fdc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:10.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.873+0000 7f6ff6000640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6fdc0a3c30 0x7f6fdc0a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0000c00 tx=0x7f6fe0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:10.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:10.874+0000 7f6fef400640 1 -- 172.21.15.62:0/1700252946 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6fe8053f20 con 0x7f6ff80744e0 2024-04-02T14:43:11.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.081+0000 7f6ff7e00640 1 -- 172.21.15.62:0/1700252946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ff802d5c0 con 0x7f6ff80744e0 2024-04-02T14:43:11.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.083+0000 7f6fef400640 1 -- 172.21.15.62:0/1700252946 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22583 (secure 0 0 0) 0x7f6fe809d570 con 0x7f6ff80744e0 2024-04-02T14:43:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:11.083 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:42:21.228892+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:43:11.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 -- 172.21.15.62:0/1700252946 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6fdc0a3c30 msgr2=0x7f6fdc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6fdc0a3c30 0x7f6fdc0a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0000c00 tx=0x7f6fe0005fd0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 -- 172.21.15.62:0/1700252946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ff80744e0 msgr2=0x7f6ff8199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ff80744e0 0x7f6ff8199f50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800eb90 tx=0x7f6fe800c6a0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 -- 172.21.15.62:0/1700252946 shutdown_connections 2024-04-02T14:43:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6fdc0a3c30 0x7f6fdc0a60f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ff8109810 0x7f6ff81a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ff8075ac0 0x7f6ff819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 --2- 172.21.15.62:0/1700252946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ff80744e0 0x7f6ff8199f50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 -- 172.21.15.62:0/1700252946 >> 172.21.15.62:0/1700252946 conn(0x7f6ff80fced0 msgr2=0x7f6ff81057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 -- 172.21.15.62:0/1700252946 shutdown_connections 2024-04-02T14:43:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.086+0000 7f6ff7e00640 1 -- 172.21.15.62:0/1700252946 wait complete. 2024-04-02T14:43:11.102 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T14:41:20.030393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T14:41:20.917405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:43:11.102 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:43:11.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.319+0000 7f2cc6600640 1 -- 172.21.15.62:0/198750990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc0104f20 msgr2=0x7f2cc01053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:11.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.319+0000 7f2cc6600640 1 --2- 172.21.15.62:0/198750990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc0104f20 0x7f2cc01053a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009900 tx=0x7f2cb002bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:11.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.319+0000 7f2cc6600640 1 -- 172.21.15.62:0/198750990 shutdown_connections 2024-04-02T14:43:11.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.319+0000 7f2cc6600640 1 --2- 172.21.15.62:0/198750990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cc01058e0 0x7f2cc010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.319+0000 7f2cc6600640 1 --2- 172.21.15.62:0/198750990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc0104f20 0x7f2cc01053a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.319+0000 7f2cc6600640 1 --2- 172.21.15.62:0/198750990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc0103d30 0x7f2cc0104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.319+0000 7f2cc6600640 1 -- 172.21.15.62:0/198750990 >> 172.21.15.62:0/198750990 conn(0x7f2cc00ff2c0 msgr2=0x7f2cc01016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:11.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.319+0000 7f2cc6600640 1 -- 172.21.15.62:0/198750990 shutdown_connections 2024-04-02T14:43:11.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.319+0000 7f2cc6600640 1 -- 172.21.15.62:0/198750990 wait complete. 2024-04-02T14:43:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.320+0000 7f2cc6600640 1 Processor -- start 2024-04-02T14:43:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cc6600640 1 -- start start 2024-04-02T14:43:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cc6600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc0103d30 0x7f2cc019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cc6600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc0104f20 0x7f2cc019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cc6600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cc01058e0 0x7f2cc01a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cc6600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc010fa90 con 0x7f2cc01058e0 2024-04-02T14:43:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cc6600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc010f910 con 0x7f2cc0104f20 2024-04-02T14:43:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cc6600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc010fc10 con 0x7f2cc0103d30 2024-04-02T14:43:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cbfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cc01058e0 0x7f2cc01a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cbf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc0103d30 0x7f2cc019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cbea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc0104f20 0x7f2cc019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cbea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc0104f20 0x7f2cc019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:45618/0 (socket says 172.21.15.62:45618) 2024-04-02T14:43:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cbea00640 1 -- 172.21.15.62:0/2795808746 learned_addr learned my addr 172.21.15.62:0/2795808746 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.321+0000 7f2cbf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc0103d30 0x7f2cc019a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:41466/0 (socket says 172.21.15.62:41466) 2024-04-02T14:43:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.322+0000 7f2cbea00640 1 -- 172.21.15.62:0/2795808746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc0103d30 msgr2=0x7f2cc019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.322+0000 7f2cbea00640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc0103d30 0x7f2cc019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.322+0000 7f2cbea00640 1 -- 172.21.15.62:0/2795808746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cc01058e0 msgr2=0x7f2cc01a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.322+0000 7f2cbea00640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cc01058e0 0x7f2cc01a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.322+0000 7f2cbea00640 1 -- 172.21.15.62:0/2795808746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc01a1ca0 con 0x7f2cc0104f20 2024-04-02T14:43:11.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.322+0000 7f2cbea00640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc0104f20 0x7f2cc019a5c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0005ab0 tx=0x7f2cb0004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:11.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.322+0000 7f2cb7e00640 1 -- 172.21.15.62:0/2795808746 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0043020 con 0x7f2cc0104f20 2024-04-02T14:43:11.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.322+0000 7f2cb7e00640 1 -- 172.21.15.62:0/2795808746 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2cb0004250 con 0x7f2cc0104f20 2024-04-02T14:43:11.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.322+0000 7f2cc6600640 1 -- 172.21.15.62:0/2795808746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc01a1f30 con 0x7f2cc0104f20 2024-04-02T14:43:11.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.322+0000 7f2cb7e00640 1 -- 172.21.15.62:0/2795808746 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0003c00 con 0x7f2cc0104f20 2024-04-02T14:43:11.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.323+0000 7f2cc6600640 1 -- 172.21.15.62:0/2795808746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc01a23f0 con 0x7f2cc0104f20 2024-04-02T14:43:11.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.324+0000 7f2cc6600640 1 -- 172.21.15.62:0/2795808746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc010aff0 con 0x7f2cc0104f20 2024-04-02T14:43:11.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.326+0000 7f2cb7e00640 1 -- 172.21.15.62:0/2795808746 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2cb0046020 con 0x7f2cc0104f20 2024-04-02T14:43:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.327+0000 7f2cb7e00640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2ca40a3bb0 0x7f2ca40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.327+0000 7f2cb7e00640 1 -- 172.21.15.62:0/2795808746 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f2cb00f2450 con 0x7f2cc0104f20 2024-04-02T14:43:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.327+0000 7f2cbf400640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2ca40a3bb0 0x7f2ca40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.328+0000 7f2cbf400640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2ca40a3bb0 0x7f2ca40a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0104d80 tx=0x7f2cac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:11.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.331+0000 7f2cb7e00640 1 -- 172.21.15.62:0/2795808746 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2cb004d020 con 0x7f2cc0104f20 2024-04-02T14:43:11.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:11 smithi204 ceph-mon[35948]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:43:11.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:11 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2201835477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:11.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:11 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1813409253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:11.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:11 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1700252946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.538+0000 7f2cc6600640 1 -- 172.21.15.62:0/2795808746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2cc00627a0 con 0x7f2cc0104f20 2024-04-02T14:43:11.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.539+0000 7f2cb7e00640 1 -- 172.21.15.62:0/2795808746 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f2cb00bee00 con 0x7f2cc0104f20 2024-04-02T14:43:11.537 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:43:11.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.541+0000 7f2cc6600640 1 -- 172.21.15.62:0/2795808746 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2ca40a3bb0 msgr2=0x7f2ca40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:11.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.541+0000 7f2cc6600640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2ca40a3bb0 0x7f2ca40a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0104d80 tx=0x7f2cac005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:11.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.541+0000 7f2cc6600640 1 -- 172.21.15.62:0/2795808746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc0104f20 msgr2=0x7f2cc019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:11.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.541+0000 7f2cc6600640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc0104f20 0x7f2cc019a5c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0005ab0 tx=0x7f2cb0004a50 comp rx=0 tx=0).stop 2024-04-02T14:43:11.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.542+0000 7f2cc6600640 1 -- 172.21.15.62:0/2795808746 shutdown_connections 2024-04-02T14:43:11.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.542+0000 7f2cc6600640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f2ca40a3bb0 0x7f2ca40a6070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.542+0000 7f2cc6600640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cc01058e0 0x7f2cc01a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.542+0000 7f2cc6600640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc0104f20 0x7f2cc019a5c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.542+0000 7f2cc6600640 1 --2- 172.21.15.62:0/2795808746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc0103d30 0x7f2cc019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.542+0000 7f2cc6600640 1 -- 172.21.15.62:0/2795808746 >> 172.21.15.62:0/2795808746 conn(0x7f2cc00ff2c0 msgr2=0x7f2cc01081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:11.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.542+0000 7f2cc6600640 1 -- 172.21.15.62:0/2795808746 shutdown_connections 2024-04-02T14:43:11.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.542+0000 7f2cc6600640 1 -- 172.21.15.62:0/2795808746 wait complete. 2024-04-02T14:43:11.557 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:11 smithi062 ceph-mon[32116]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:43:11.557 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:11 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2201835477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:11.558 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:11 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1813409253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:11.558 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:11 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1700252946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:11.559 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T14:43:11.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:11 smithi113 ceph-mon[36135]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:43:11.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:11 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2201835477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:11.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:11 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1813409253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:11.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:11 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1700252946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.793+0000 7f1f33200640 1 -- 172.21.15.62:0/3819521391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f34103c20 msgr2=0x7f1f34104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.793+0000 7f1f33200640 1 --2- 172.21.15.62:0/3819521391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f34103c20 0x7f1f34104040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009900 tx=0x7f1f2402bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.793+0000 7f1f33200640 1 -- 172.21.15.62:0/3819521391 shutdown_connections 2024-04-02T14:43:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.793+0000 7f1f33200640 1 --2- 172.21.15.62:0/3819521391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f341057d0 0x7f1f3410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.793+0000 7f1f33200640 1 --2- 172.21.15.62:0/3819521391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34104e10 0x7f1f34105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.793+0000 7f1f33200640 1 --2- 172.21.15.62:0/3819521391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f34103c20 0x7f1f34104040 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.793+0000 7f1f33200640 1 -- 172.21.15.62:0/3819521391 >> 172.21.15.62:0/3819521391 conn(0x7f1f340ff1b0 msgr2=0x7f1f341015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.793+0000 7f1f33200640 1 -- 172.21.15.62:0/3819521391 shutdown_connections 2024-04-02T14:43:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.793+0000 7f1f33200640 1 -- 172.21.15.62:0/3819521391 wait complete. 2024-04-02T14:43:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.794+0000 7f1f33200640 1 Processor -- start 2024-04-02T14:43:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.795+0000 7f1f33200640 1 -- start start 2024-04-02T14:43:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.795+0000 7f1f33200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34103c20 0x7f1f3419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.795+0000 7f1f33200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f34104e10 0x7f1f3419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.795+0000 7f1f33200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f341057d0 0x7f1f341a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.795+0000 7f1f33200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3410f9a0 con 0x7f1f341057d0 2024-04-02T14:43:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.795+0000 7f1f33200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3410f820 con 0x7f1f34103c20 2024-04-02T14:43:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.795+0000 7f1f33200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3410fb20 con 0x7f1f34104e10 2024-04-02T14:43:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.795+0000 7f1f31e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34103c20 0x7f1f3419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.795+0000 7f1f32800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f341057d0 0x7f1f341a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.795+0000 7f1f31e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34103c20 0x7f1f3419a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:45640/0 (socket says 172.21.15.62:45640) 2024-04-02T14:43:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.795+0000 7f1f31e00640 1 -- 172.21.15.62:0/2694995777 learned_addr learned my addr 172.21.15.62:0/2694995777 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.796+0000 7f1f2be00640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f34104e10 0x7f1f3419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.796+0000 7f1f31e00640 1 -- 172.21.15.62:0/2694995777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f34104e10 msgr2=0x7f1f3419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.796+0000 7f1f31e00640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f34104e10 0x7f1f3419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.796+0000 7f1f31e00640 1 -- 172.21.15.62:0/2694995777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f341057d0 msgr2=0x7f1f341a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.796+0000 7f1f31e00640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f341057d0 0x7f1f341a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.796+0000 7f1f31e00640 1 -- 172.21.15.62:0/2694995777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f341a1c60 con 0x7f1f34103c20 2024-04-02T14:43:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.796+0000 7f1f32800640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f341057d0 0x7f1f341a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.796+0000 7f1f2be00640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f34104e10 0x7f1f3419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.796+0000 7f1f31e00640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34103c20 0x7f1f3419a040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000c00 tx=0x7f1f24004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.797+0000 7f1f2aa00640 1 -- 172.21.15.62:0/2694995777 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24043020 con 0x7f1f34103c20 2024-04-02T14:43:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.797+0000 7f1f2aa00640 1 -- 172.21.15.62:0/2694995777 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1f24004250 con 0x7f1f34103c20 2024-04-02T14:43:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.797+0000 7f1f33200640 1 -- 172.21.15.62:0/2694995777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f341a1ef0 con 0x7f1f34103c20 2024-04-02T14:43:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.797+0000 7f1f2aa00640 1 -- 172.21.15.62:0/2694995777 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24003c00 con 0x7f1f34103c20 2024-04-02T14:43:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.797+0000 7f1f33200640 1 -- 172.21.15.62:0/2694995777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f341a23b0 con 0x7f1f34103c20 2024-04-02T14:43:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.798+0000 7f1f33200640 1 -- 172.21.15.62:0/2694995777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3410af40 con 0x7f1f34103c20 2024-04-02T14:43:11.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.800+0000 7f1f2aa00640 1 -- 172.21.15.62:0/2694995777 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1f24046020 con 0x7f1f34103c20 2024-04-02T14:43:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.801+0000 7f1f2aa00640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f180a3c30 0x7f1f180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.801+0000 7f1f2aa00640 1 -- 172.21.15.62:0/2694995777 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f1f240f2ae0 con 0x7f1f34103c20 2024-04-02T14:43:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.802+0000 7f1f2be00640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f180a3c30 0x7f1f180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.802+0000 7f1f2be00640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f180a3c30 0x7f1f180a60f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1f3419b550 tx=0x7f1f1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:11.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:11.804+0000 7f1f2aa00640 1 -- 172.21.15.62:0/2694995777 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1f2404d020 con 0x7f1f34103c20 2024-04-02T14:43:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.009+0000 7f1f33200640 1 -- 172.21.15.62:0/2694995777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1f3402d5c0 con 0x7f1f34103c20 2024-04-02T14:43:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.010+0000 7f1f2aa00640 1 -- 172.21.15.62:0/2694995777 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f1f240bf490 con 0x7f1f34103c20 2024-04-02T14:43:12.008 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-02T14:43:12.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 -- 172.21.15.62:0/2694995777 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f180a3c30 msgr2=0x7f1f180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:12.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f180a3c30 0x7f1f180a60f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1f3419b550 tx=0x7f1f1c005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:12.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 -- 172.21.15.62:0/2694995777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34103c20 msgr2=0x7f1f3419a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:12.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34103c20 0x7f1f3419a040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000c00 tx=0x7f1f24004a30 comp rx=0 tx=0).stop 2024-04-02T14:43:12.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 -- 172.21.15.62:0/2694995777 shutdown_connections 2024-04-02T14:43:12.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f180a3c30 0x7f1f180a60f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f341057d0 0x7f1f341a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f34104e10 0x7f1f3419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 --2- 172.21.15.62:0/2694995777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34103c20 0x7f1f3419a040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 -- 172.21.15.62:0/2694995777 >> 172.21.15.62:0/2694995777 conn(0x7f1f340ff1b0 msgr2=0x7f1f34108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:12.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 -- 172.21.15.62:0/2694995777 shutdown_connections 2024-04-02T14:43:12.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.013+0000 7f1f33200640 1 -- 172.21.15.62:0/2694995777 wait complete. 2024-04-02T14:43:12.032 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T14:43:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.263+0000 7ff889e00640 1 -- 172.21.15.62:0/3231208358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8841036c0 msgr2=0x7ff884109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.263+0000 7ff889e00640 1 --2- 172.21.15.62:0/3231208358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8841036c0 0x7ff884109f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009900 tx=0x7ff87c02bc40 comp rx=0 tx=0).stop 2024-04-02T14:43:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.263+0000 7ff889e00640 1 -- 172.21.15.62:0/3231208358 shutdown_connections 2024-04-02T14:43:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.263+0000 7ff889e00640 1 --2- 172.21.15.62:0/3231208358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8841036c0 0x7ff884109f50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.263+0000 7ff889e00640 1 --2- 172.21.15.62:0/3231208358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff884102d00 0x7ff884103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.263+0000 7ff889e00640 1 --2- 172.21.15.62:0/3231208358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff884101b10 0x7ff884101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.263+0000 7ff889e00640 1 -- 172.21.15.62:0/3231208358 >> 172.21.15.62:0/3231208358 conn(0x7ff8840fd0a0 msgr2=0x7ff8840ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.263+0000 7ff889e00640 1 -- 172.21.15.62:0/3231208358 shutdown_connections 2024-04-02T14:43:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.263+0000 7ff889e00640 1 -- 172.21.15.62:0/3231208358 wait complete. 2024-04-02T14:43:12.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.264+0000 7ff889e00640 1 Processor -- start 2024-04-02T14:43:12.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff889e00640 1 -- start start 2024-04-02T14:43:12.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff889e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff884101b10 0x7ff88419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:12.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff889e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff884102d00 0x7ff88419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:12.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff889e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8841036c0 0x7ff8841a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:12.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff889e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff88410d8b0 con 0x7ff8841036c0 2024-04-02T14:43:12.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff889e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff88410d730 con 0x7ff884101b10 2024-04-02T14:43:12.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff889e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff88410da30 con 0x7ff884102d00 2024-04-02T14:43:12.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff882a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff884101b10 0x7ff88419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:12.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff882a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff884101b10 0x7ff88419a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:43592/0 (socket says 172.21.15.62:43592) 2024-04-02T14:43:12.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff882a00640 1 -- 172.21.15.62:0/1916075830 learned_addr learned my addr 172.21.15.62:0/1916075830 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:12.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff883400640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8841036c0 0x7ff8841a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:12.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.265+0000 7ff87a000640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff884102d00 0x7ff88419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:12.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.266+0000 7ff882a00640 1 -- 172.21.15.62:0/1916075830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff884102d00 msgr2=0x7ff88419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:12.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.266+0000 7ff882a00640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff884102d00 0x7ff88419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.266+0000 7ff882a00640 1 -- 172.21.15.62:0/1916075830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8841036c0 msgr2=0x7ff8841a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:12.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.266+0000 7ff882a00640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8841036c0 0x7ff8841a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.266+0000 7ff882a00640 1 -- 172.21.15.62:0/1916075830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8841a1d00 con 0x7ff884101b10 2024-04-02T14:43:12.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.266+0000 7ff882a00640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff884101b10 0x7ff88419a0e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff87400ea80 tx=0x7ff87400ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:12.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.266+0000 7ff888a00640 1 -- 172.21.15.62:0/1916075830 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87400ce40 con 0x7ff884101b10 2024-04-02T14:43:12.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.266+0000 7ff888a00640 1 -- 172.21.15.62:0/1916075830 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff874011970 con 0x7ff884101b10 2024-04-02T14:43:12.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.266+0000 7ff888a00640 1 -- 172.21.15.62:0/1916075830 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87401ac10 con 0x7ff884101b10 2024-04-02T14:43:12.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.266+0000 7ff889e00640 1 -- 172.21.15.62:0/1916075830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8841a1ff0 con 0x7ff884101b10 2024-04-02T14:43:12.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.267+0000 7ff889e00640 1 -- 172.21.15.62:0/1916075830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8841a2400 con 0x7ff884101b10 2024-04-02T14:43:12.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.268+0000 7ff889e00640 1 -- 172.21.15.62:0/1916075830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8841a26c0 con 0x7ff884101b10 2024-04-02T14:43:12.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.270+0000 7ff888a00640 1 -- 172.21.15.62:0/1916075830 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff874011b20 con 0x7ff884101b10 2024-04-02T14:43:12.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.271+0000 7ff888a00640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff8680a3bb0 0x7ff8680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:12.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.271+0000 7ff888a00640 1 -- 172.21.15.62:0/1916075830 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7ff8740d0950 con 0x7ff884101b10 2024-04-02T14:43:12.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.271+0000 7ff87a000640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff8680a3bb0 0x7ff8680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:12.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.272+0000 7ff87a000640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff8680a3bb0 0x7ff8680a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff88419b5f0 tx=0x7ff86c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:12.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.276+0000 7ff888a00640 1 -- 172.21.15.62:0/1916075830 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff874016030 con 0x7ff884101b10 2024-04-02T14:43:12.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:12 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2795808746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:12.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:12 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2694995777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:12.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.487+0000 7ff889e00640 1 -- 172.21.15.62:0/1916075830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff8841a2fb0 con 0x7ff884101b10 2024-04-02T14:43:12.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.488+0000 7ff888a00640 1 -- 172.21.15.62:0/1916075830 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7ff87409d300 con 0x7ff884101b10 2024-04-02T14:43:12.486 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-02T14:43:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 -- 172.21.15.62:0/1916075830 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff8680a3bb0 msgr2=0x7ff8680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff8680a3bb0 0x7ff8680a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff88419b5f0 tx=0x7ff86c005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 -- 172.21.15.62:0/1916075830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff884101b10 msgr2=0x7ff88419a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff884101b10 0x7ff88419a0e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff87400ea80 tx=0x7ff87400ef40 comp rx=0 tx=0).stop 2024-04-02T14:43:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 -- 172.21.15.62:0/1916075830 shutdown_connections 2024-04-02T14:43:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff8680a3bb0 0x7ff8680a6070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8841036c0 0x7ff8841a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff884102d00 0x7ff88419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 --2- 172.21.15.62:0/1916075830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff884101b10 0x7ff88419a0e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 -- 172.21.15.62:0/1916075830 >> 172.21.15.62:0/1916075830 conn(0x7ff8840fd0a0 msgr2=0x7ff884073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 -- 172.21.15.62:0/1916075830 shutdown_connections 2024-04-02T14:43:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.491+0000 7ff889e00640 1 -- 172.21.15.62:0/1916075830 wait complete. 2024-04-02T14:43:12.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:12 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2795808746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:12.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:12 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2694995777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:12.510 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:43:12.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:12 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2795808746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:12.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:12 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2694995777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:12.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.739+0000 7f7055800640 1 -- 172.21.15.62:0/3511614378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7050108140 msgr2=0x7f7050108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:12.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.739+0000 7f7055800640 1 --2- 172.21.15.62:0/3511614378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7050108140 0x7f7050108560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7040009900 tx=0x7f704002bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:12.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.739+0000 7f7055800640 1 -- 172.21.15.62:0/3511614378 shutdown_connections 2024-04-02T14:43:12.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.739+0000 7f7055800640 1 --2- 172.21.15.62:0/3511614378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7050109cf0 0x7f7050110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.739+0000 7f7055800640 1 --2- 172.21.15.62:0/3511614378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7050109330 0x7f70501097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.739+0000 7f7055800640 1 --2- 172.21.15.62:0/3511614378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7050108140 0x7f7050108560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.739+0000 7f7055800640 1 -- 172.21.15.62:0/3511614378 >> 172.21.15.62:0/3511614378 conn(0x7f7050077110 msgr2=0x7f7050079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:12.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.739+0000 7f7055800640 1 -- 172.21.15.62:0/3511614378 shutdown_connections 2024-04-02T14:43:12.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.739+0000 7f7055800640 1 -- 172.21.15.62:0/3511614378 wait complete. 2024-04-02T14:43:12.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.740+0000 7f7055800640 1 Processor -- start 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.740+0000 7f7055800640 1 -- start start 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.740+0000 7f7055800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7050108140 0x7f705019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f7055800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7050109330 0x7f705019ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f7055800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7050109cf0 0x7f70501a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f7055800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7050113ea0 con 0x7f7050109cf0 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f7055800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7050113d20 con 0x7f7050108140 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f7055800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7050114020 con 0x7f7050109330 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f704f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7050109cf0 0x7f70501a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f704e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7050109330 0x7f705019ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f704ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7050108140 0x7f705019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f704f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7050109cf0 0x7f70501a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41138/0 (socket says 172.21.15.62:41138) 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f704f400640 1 -- 172.21.15.62:0/4203372780 learned_addr learned my addr 172.21.15.62:0/4203372780 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:12.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f704e000640 1 -- 172.21.15.62:0/4203372780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7050108140 msgr2=0x7f705019e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:12.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f704e000640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7050108140 0x7f705019e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f704e000640 1 -- 172.21.15.62:0/4203372780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7050109cf0 msgr2=0x7f70501a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:12.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f704e000640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7050109cf0 0x7f70501a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:12.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f704e000640 1 -- 172.21.15.62:0/4203372780 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70501a6100 con 0x7f7050109330 2024-04-02T14:43:12.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.741+0000 7f704f400640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7050109cf0 0x7f70501a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:12.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.742+0000 7f704e000640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7050109330 0x7f705019ea20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e750 tx=0x7f703c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:12.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.742+0000 7f7047400640 1 -- 172.21.15.62:0/4203372780 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c00ccc0 con 0x7f7050109330 2024-04-02T14:43:12.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.742+0000 7f7055800640 1 -- 172.21.15.62:0/4203372780 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70501a63f0 con 0x7f7050109330 2024-04-02T14:43:12.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.742+0000 7f7055800640 1 -- 172.21.15.62:0/4203372780 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70501a6940 con 0x7f7050109330 2024-04-02T14:43:12.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.742+0000 7f7047400640 1 -- 172.21.15.62:0/4203372780 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f703c00ee60 con 0x7f7050109330 2024-04-02T14:43:12.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.742+0000 7f7047400640 1 -- 172.21.15.62:0/4203372780 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c01c070 con 0x7f7050109330 2024-04-02T14:43:12.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.743+0000 7f7055800640 1 -- 172.21.15.62:0/4203372780 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70300052c0 con 0x7f7050109330 2024-04-02T14:43:12.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.746+0000 7f7047400640 1 -- 172.21.15.62:0/4203372780 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f703c004a00 con 0x7f7050109330 2024-04-02T14:43:12.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.747+0000 7f7047400640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f70340a3c30 0x7f70340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:12.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.748+0000 7f7047400640 1 -- 172.21.15.62:0/4203372780 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f703c0d0150 con 0x7f7050109330 2024-04-02T14:43:12.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.748+0000 7f704ea00640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f70340a3c30 0x7f70340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:12.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.748+0000 7f704ea00640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f70340a3c30 0x7f70340a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f704002c900 tx=0x7f7040005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:12.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:12.749+0000 7f7047400640 1 -- 172.21.15.62:0/4203372780 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f703c09cb90 con 0x7f7050109330 2024-04-02T14:43:13.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.003+0000 7f7055800640 1 -- 172.21.15.62:0/4203372780 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70300055b0 con 0x7f7050109330 2024-04-02T14:43:13.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.005+0000 7f7047400640 1 -- 172.21.15.62:0/4203372780 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f703c09c490 con 0x7f7050109330 2024-04-02T14:43:13.003 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:13.003 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:41:30.090009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24553,24565,24533,24535,24584]},"id":1}]} 2024-04-02T14:43:13.003 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-02T14:43:13.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 -- 172.21.15.62:0/4203372780 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f70340a3c30 msgr2=0x7f70340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f70340a3c30 0x7f70340a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f704002c900 tx=0x7f7040005fd0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 -- 172.21.15.62:0/4203372780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7050109330 msgr2=0x7f705019ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7050109330 0x7f705019ea20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e750 tx=0x7f703c00ec10 comp rx=0 tx=0).stop 2024-04-02T14:43:13.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 -- 172.21.15.62:0/4203372780 shutdown_connections 2024-04-02T14:43:13.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f70340a3c30 0x7f70340a60f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7050109cf0 0x7f70501a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7050109330 0x7f705019ea20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 --2- 172.21.15.62:0/4203372780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7050108140 0x7f705019e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 -- 172.21.15.62:0/4203372780 >> 172.21.15.62:0/4203372780 conn(0x7f7050077110 msgr2=0x7f705010c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:13.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 -- 172.21.15.62:0/4203372780 shutdown_connections 2024-04-02T14:43:13.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.007+0000 7f7055800640 1 -- 172.21.15.62:0/4203372780 wait complete. 2024-04-02T14:43:13.025 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:43:13.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.249+0000 7f4b8c000640 1 -- 172.21.15.62:0/2708264656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b841099e0 msgr2=0x7f4b8410be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.249+0000 7f4b8c000640 1 --2- 172.21.15.62:0/2708264656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b841099e0 0x7f4b8410be70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009be0 tx=0x7f4b7c02bef0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.249+0000 7f4b8c000640 1 -- 172.21.15.62:0/2708264656 shutdown_connections 2024-04-02T14:43:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.249+0000 7f4b8c000640 1 --2- 172.21.15.62:0/2708264656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b841099e0 0x7f4b8410be70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.249+0000 7f4b8c000640 1 --2- 172.21.15.62:0/2708264656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b84074560 0x7f4b840749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.249+0000 7f4b8c000640 1 --2- 172.21.15.62:0/2708264656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b84075b40 0x7f4b84073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.249+0000 7f4b8c000640 1 -- 172.21.15.62:0/2708264656 >> 172.21.15.62:0/2708264656 conn(0x7f4b840fd0b0 msgr2=0x7f4b840ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.249+0000 7f4b8c000640 1 -- 172.21.15.62:0/2708264656 shutdown_connections 2024-04-02T14:43:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.249+0000 7f4b8c000640 1 -- 172.21.15.62:0/2708264656 wait complete. 2024-04-02T14:43:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.250+0000 7f4b8c000640 1 Processor -- start 2024-04-02T14:43:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.250+0000 7f4b8c000640 1 -- start start 2024-04-02T14:43:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b8c000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b84074560 0x7f4b8419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b8c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b84075b40 0x7f4b8419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b8c000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b841099e0 0x7f4b841a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b8c000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8410fb10 con 0x7f4b841099e0 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b8c000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8410f990 con 0x7f4b84074560 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b8c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8410fc90 con 0x7f4b84075b40 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b89600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b84074560 0x7f4b8419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b89600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b84074560 0x7f4b8419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:43616/0 (socket says 172.21.15.62:43616) 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b89600640 1 -- 172.21.15.62:0/2694422097 learned_addr learned my addr 172.21.15.62:0/2694422097 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b8a000640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b841099e0 0x7f4b841a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b88c00640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b84075b40 0x7f4b8419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b89600640 1 -- 172.21.15.62:0/2694422097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b84075b40 msgr2=0x7f4b8419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b89600640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b84075b40 0x7f4b8419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b89600640 1 -- 172.21.15.62:0/2694422097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b841099e0 msgr2=0x7f4b841a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.251+0000 7f4b89600640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b841099e0 0x7f4b841a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.252+0000 7f4b89600640 1 -- 172.21.15.62:0/2694422097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b841a1c90 con 0x7f4b84074560 2024-04-02T14:43:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.252+0000 7f4b89600640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b84074560 0x7f4b8419a070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800e9c0 tx=0x7f4b7800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:13.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.252+0000 7f4b82000640 1 -- 172.21.15.62:0/2694422097 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800ccc0 con 0x7f4b84074560 2024-04-02T14:43:13.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.252+0000 7f4b82000640 1 -- 172.21.15.62:0/2694422097 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4b78005520 con 0x7f4b84074560 2024-04-02T14:43:13.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.252+0000 7f4b82000640 1 -- 172.21.15.62:0/2694422097 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78010690 con 0x7f4b84074560 2024-04-02T14:43:13.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.252+0000 7f4b8c000640 1 -- 172.21.15.62:0/2694422097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b841a1f80 con 0x7f4b84074560 2024-04-02T14:43:13.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.252+0000 7f4b8c000640 1 -- 172.21.15.62:0/2694422097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b841a2390 con 0x7f4b84074560 2024-04-02T14:43:13.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.254+0000 7f4b8c000640 1 -- 172.21.15.62:0/2694422097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b84108710 con 0x7f4b84074560 2024-04-02T14:43:13.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.255+0000 7f4b82000640 1 -- 172.21.15.62:0/2694422097 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4b780046c0 con 0x7f4b84074560 2024-04-02T14:43:13.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.256+0000 7f4b82000640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4b6c0a3c30 0x7f4b6c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:13.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.257+0000 7f4b82000640 1 -- 172.21.15.62:0/2694422097 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f4b780d0640 con 0x7f4b84074560 2024-04-02T14:43:13.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.257+0000 7f4b88c00640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4b6c0a3c30 0x7f4b6c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:13.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.257+0000 7f4b88c00640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4b6c0a3c30 0x7f4b6c0a60f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b8419b580 tx=0x7f4b70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:13.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.261+0000 7f4b82000640 1 -- 172.21.15.62:0/2694422097 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4b7809cff0 con 0x7f4b84074560 2024-04-02T14:43:13.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:13 smithi062 ceph-mon[32116]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:13.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:13 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1916075830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:13.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:13 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/4203372780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.469+0000 7f4b8c000640 1 -- 172.21.15.62:0/2694422097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b840627a0 con 0x7f4b84074560 2024-04-02T14:43:13.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.471+0000 7f4b82000640 1 -- 172.21.15.62:0/2694422097 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22583 (secure 0 0 0) 0x7f4b7809c8f0 con 0x7f4b84074560 2024-04-02T14:43:13.470 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:13.470 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:42:21.228892+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:43:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.473+0000 7f4b8c000640 1 -- 172.21.15.62:0/2694422097 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4b6c0a3c30 msgr2=0x7f4b6c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.473+0000 7f4b8c000640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4b6c0a3c30 0x7f4b6c0a60f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b8419b580 tx=0x7f4b70005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.473+0000 7f4b8c000640 1 -- 172.21.15.62:0/2694422097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b84074560 msgr2=0x7f4b8419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.473+0000 7f4b8c000640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b84074560 0x7f4b8419a070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800e9c0 tx=0x7f4b7800ee80 comp rx=0 tx=0).stop 2024-04-02T14:43:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.473+0000 7f4b8c000640 1 -- 172.21.15.62:0/2694422097 shutdown_connections 2024-04-02T14:43:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.473+0000 7f4b8c000640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4b6c0a3c30 0x7f4b6c0a60f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.473+0000 7f4b8c000640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b841099e0 0x7f4b841a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.473+0000 7f4b8c000640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b84075b40 0x7f4b8419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.473+0000 7f4b8c000640 1 --2- 172.21.15.62:0/2694422097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b84074560 0x7f4b8419a070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.473+0000 7f4b8c000640 1 -- 172.21.15.62:0/2694422097 >> 172.21.15.62:0/2694422097 conn(0x7f4b840fd0b0 msgr2=0x7f4b841058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.474+0000 7f4b8c000640 1 -- 172.21.15.62:0/2694422097 shutdown_connections 2024-04-02T14:43:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.474+0000 7f4b8c000640 1 -- 172.21.15.62:0/2694422097 wait complete. 2024-04-02T14:43:13.492 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:43:13.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:13 smithi204 ceph-mon[35948]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:13.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:13 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1916075830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:13.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:13 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/4203372780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:13.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:13 smithi113 ceph-mon[36135]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:13.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:13 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1916075830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:13.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:13 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/4203372780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:13.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.715+0000 7f1c43e00640 1 -- 172.21.15.62:0/2771744035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c440780a0 msgr2=0x7f1c44078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.715+0000 7f1c43e00640 1 --2- 172.21.15.62:0/2771744035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c440780a0 0x7f1c44078940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009be0 tx=0x7f1c3c02bef0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.715+0000 7f1c43e00640 1 -- 172.21.15.62:0/2771744035 shutdown_connections 2024-04-02T14:43:13.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.715+0000 7f1c43e00640 1 --2- 172.21.15.62:0/2771744035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c440780a0 0x7f1c44078940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.715+0000 7f1c43e00640 1 --2- 172.21.15.62:0/2771744035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c44077700 0x7f1c44077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.715+0000 7f1c43e00640 1 --2- 172.21.15.62:0/2771744035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c440764c0 0x7f1c440768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.715+0000 7f1c43e00640 1 -- 172.21.15.62:0/2771744035 >> 172.21.15.62:0/2771744035 conn(0x7f1c440ff0e0 msgr2=0x7f1c44101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:13.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.715+0000 7f1c43e00640 1 -- 172.21.15.62:0/2771744035 shutdown_connections 2024-04-02T14:43:13.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.715+0000 7f1c43e00640 1 -- 172.21.15.62:0/2771744035 wait complete. 2024-04-02T14:43:13.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.716+0000 7f1c43e00640 1 Processor -- start 2024-04-02T14:43:13.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c43e00640 1 -- start start 2024-04-02T14:43:13.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c43e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c440764c0 0x7f1c4419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c43e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c44077700 0x7f1c4419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c43e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c440780a0 0x7f1c441a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c43e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1c44113d00 con 0x7f1c44077700 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c43e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1c44113b80 con 0x7f1c440764c0 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c43e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1c44113e80 con 0x7f1c440780a0 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c42000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c44077700 0x7f1c4419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c43400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c440780a0 0x7f1c441a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c42a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c440764c0 0x7f1c4419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c42000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c44077700 0x7f1c4419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41180/0 (socket says 172.21.15.62:41180) 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c43400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c440780a0 0x7f1c441a5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:36102/0 (socket says 172.21.15.62:36102) 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.717+0000 7f1c43400640 1 -- 172.21.15.62:0/2784594622 learned_addr learned my addr 172.21.15.62:0/2784594622 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c42000640 1 -- 172.21.15.62:0/2784594622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c440780a0 msgr2=0x7f1c441a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c42000640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c440780a0 0x7f1c441a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c42000640 1 -- 172.21.15.62:0/2784594622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c440764c0 msgr2=0x7f1c4419e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c42000640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c440764c0 0x7f1c4419e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c42000640 1 -- 172.21.15.62:0/2784594622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c441a5eb0 con 0x7f1c44077700 2024-04-02T14:43:13.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c43400640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c440780a0 0x7f1c441a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:13.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c42a00640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c440764c0 0x7f1c4419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:13.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c42000640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c44077700 0x7f1c4419e7d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000e9f0 tx=0x7f1c3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:13.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c3b400640 1 -- 172.21.15.62:0/2784594622 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3000ccc0 con 0x7f1c44077700 2024-04-02T14:43:13.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c43e00640 1 -- 172.21.15.62:0/2784594622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c441a61a0 con 0x7f1c44077700 2024-04-02T14:43:13.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c3b400640 1 -- 172.21.15.62:0/2784594622 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1c30010930 con 0x7f1c44077700 2024-04-02T14:43:13.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c3b400640 1 -- 172.21.15.62:0/2784594622 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30018920 con 0x7f1c44077700 2024-04-02T14:43:13.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.718+0000 7f1c43e00640 1 -- 172.21.15.62:0/2784594622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c441a66f0 con 0x7f1c44077700 2024-04-02T14:43:13.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.720+0000 7f1c43e00640 1 -- 172.21.15.62:0/2784594622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c44110590 con 0x7f1c44077700 2024-04-02T14:43:13.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.722+0000 7f1c3b400640 1 -- 172.21.15.62:0/2784594622 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1c30010ae0 con 0x7f1c44077700 2024-04-02T14:43:13.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.724+0000 7f1c3b400640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1c280a3bb0 0x7f1c280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:13.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.724+0000 7f1c3b400640 1 -- 172.21.15.62:0/2784594622 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f1c300d0690 con 0x7f1c44077700 2024-04-02T14:43:13.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.724+0000 7f1c42a00640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1c280a3bb0 0x7f1c280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:13.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.725+0000 7f1c42a00640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1c280a3bb0 0x7f1c280a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1c34005fd0 tx=0x7f1c34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.729+0000 7f1c3b400640 1 -- 172.21.15.62:0/2784594622 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1c3009d040 con 0x7f1c44077700 2024-04-02T14:43:13.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.977+0000 7f1c43e00640 1 -- 172.21.15.62:0/2784594622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c44062700 con 0x7f1c44077700 2024-04-02T14:43:13.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.978+0000 7f1c3b400640 1 -- 172.21.15.62:0/2784594622 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f1c3009c940 con 0x7f1c44077700 2024-04-02T14:43:13.977 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:13.977 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:41:30.090009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24553,24565,24533,24535,24584]},"id":1}]} 2024-04-02T14:43:13.977 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-02T14:43:13.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 -- 172.21.15.62:0/2784594622 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1c280a3bb0 msgr2=0x7f1c280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1c280a3bb0 0x7f1c280a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1c34005fd0 tx=0x7f1c34009260 comp rx=0 tx=0).stop 2024-04-02T14:43:13.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 -- 172.21.15.62:0/2784594622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c44077700 msgr2=0x7f1c4419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:13.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c44077700 0x7f1c4419e7d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000e9f0 tx=0x7f1c3000edf0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 -- 172.21.15.62:0/2784594622 shutdown_connections 2024-04-02T14:43:13.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1c280a3bb0 0x7f1c280a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c440780a0 0x7f1c441a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c44077700 0x7f1c4419e7d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 --2- 172.21.15.62:0/2784594622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c440764c0 0x7f1c4419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:13.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 -- 172.21.15.62:0/2784594622 >> 172.21.15.62:0/2784594622 conn(0x7f1c440ff0e0 msgr2=0x7f1c44100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:13.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 -- 172.21.15.62:0/2784594622 shutdown_connections 2024-04-02T14:43:13.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:13.981+0000 7f1c43e00640 1 -- 172.21.15.62:0/2784594622 wait complete. 2024-04-02T14:43:13.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-02T14:41:22.009309+0000', 'modified': '2024-04-02T14:41:30.090009+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24584, 'mds_1': 24565, 'mds_2': 24553, 'mds_3': 24535, 'mds_4': 24533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24461': {'gid': 24461, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.113:6833/1862883972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6832', 'nonce': 1862883972}, {'type': 'v1', 'addr': '172.21.15.113:6833', 'nonce': 1862883972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'd', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/3805165269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 3805165269}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 3805165269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/3756932584', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 3756932584}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 3756932584}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.113:6835/360641878', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 360641878}, {'type': 'v1', 'addr': '172.21.15.113:6835', 'nonce': 360641878}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/3537379638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 3537379638}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 3537379638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.113:6837/1122486361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 1122486361}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 1122486361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.204:6839/631749651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 631749651}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 631749651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24553': {'gid': 24553, 'name': 'g', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.62:6839/3188264140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 3188264140}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 3188264140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24565': {'gid': 24565, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.62:6841/450176372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6840', 'nonce': 450176372}, {'type': 'v1', 'addr': '172.21.15.62:6841', 'nonce': 450176372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24584': {'gid': 24584, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6839/1791174899', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6838', 'nonce': 1791174899}, {'type': 'v1', 'addr': '172.21.15.113:6839', 'nonce': 1791174899}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24584, 'qdb_cluster': [24553, 24565, 24533, 24535, 24584]} 2024-04-02T14:43:13.999 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-02T14:43:13.999 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-02T14:43:14.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.211+0000 7f6aa3e00640 1 -- 172.21.15.62:0/1897112692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aa4109bd0 msgr2=0x7f6aa4110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.211+0000 7f6aa3e00640 1 --2- 172.21.15.62:0/1897112692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aa4109bd0 0x7f6aa4110460 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009ca0 tx=0x7f6a9c02c100 comp rx=0 tx=0).stop 2024-04-02T14:43:14.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.211+0000 7f6aa3e00640 1 -- 172.21.15.62:0/1897112692 shutdown_connections 2024-04-02T14:43:14.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.211+0000 7f6aa3e00640 1 --2- 172.21.15.62:0/1897112692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aa4109bd0 0x7f6aa4110460 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.211+0000 7f6aa3e00640 1 --2- 172.21.15.62:0/1897112692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aa4109210 0x7f6aa4109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.211+0000 7f6aa3e00640 1 --2- 172.21.15.62:0/1897112692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6aa4108020 0x7f6aa4108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.211+0000 7f6aa3e00640 1 -- 172.21.15.62:0/1897112692 >> 172.21.15.62:0/1897112692 conn(0x7f6aa4076fd0 msgr2=0x7f6aa40793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.211+0000 7f6aa3e00640 1 -- 172.21.15.62:0/1897112692 shutdown_connections 2024-04-02T14:43:14.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.211+0000 7f6aa3e00640 1 -- 172.21.15.62:0/1897112692 wait complete. 2024-04-02T14:43:14.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.212+0000 7f6aa3e00640 1 Processor -- start 2024-04-02T14:43:14.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.212+0000 7f6aa3e00640 1 -- start start 2024-04-02T14:43:14.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.212+0000 7f6aa3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6aa4108020 0x7f6aa419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.212+0000 7f6aa3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aa4109210 0x7f6aa419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.212+0000 7f6aa3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aa4109bd0 0x7f6aa41a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.212+0000 7f6aa3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa4113dc0 con 0x7f6aa4109210 2024-04-02T14:43:14.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.212+0000 7f6aa3e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa4113c40 con 0x7f6aa4109bd0 2024-04-02T14:43:14.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.212+0000 7f6aa3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa4113f40 con 0x7f6aa4108020 2024-04-02T14:43:14.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.213+0000 7f6a9be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aa4109210 0x7f6aa419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.213+0000 7f6a9be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aa4109210 0x7f6aa419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41202/0 (socket says 172.21.15.62:41202) 2024-04-02T14:43:14.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.213+0000 7f6a9be00640 1 -- 172.21.15.62:0/2840892412 learned_addr learned my addr 172.21.15.62:0/2840892412 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.213+0000 7f6aa3400640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aa4109bd0 0x7f6aa41a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.213+0000 7f6aa2a00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6aa4108020 0x7f6aa419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.213+0000 7f6a9be00640 1 -- 172.21.15.62:0/2840892412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6aa4108020 msgr2=0x7f6aa419e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.213+0000 7f6a9be00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6aa4108020 0x7f6aa419e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.213+0000 7f6a9be00640 1 -- 172.21.15.62:0/2840892412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aa4109bd0 msgr2=0x7f6aa41a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.213+0000 7f6a9be00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aa4109bd0 0x7f6aa41a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.213+0000 7f6a9be00640 1 -- 172.21.15.62:0/2840892412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa41a5f70 con 0x7f6aa4109210 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.213+0000 7f6a9be00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aa4109210 0x7f6aa419e890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00e9f0 tx=0x7f6a8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.214+0000 7f6a9b400640 1 -- 172.21.15.62:0/2840892412 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00cd80 con 0x7f6aa4109210 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.214+0000 7f6a9b400640 1 -- 172.21.15.62:0/2840892412 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6a8c010930 con 0x7f6aa4109210 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.214+0000 7f6a9b400640 1 -- 172.21.15.62:0/2840892412 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c018b00 con 0x7f6aa4109210 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.214+0000 7f6aa3e00640 1 -- 172.21.15.62:0/2840892412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa41a6260 con 0x7f6aa4109210 2024-04-02T14:43:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.214+0000 7f6aa3e00640 1 -- 172.21.15.62:0/2840892412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa41a67b0 con 0x7f6aa4109210 2024-04-02T14:43:14.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.217+0000 7f6a9b400640 1 -- 172.21.15.62:0/2840892412 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6a8c018ca0 con 0x7f6aa4109210 2024-04-02T14:43:14.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.217+0000 7f6aac25a740 1 Processor -- start 2024-04-02T14:43:14.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.217+0000 7f6aac25a740 1 -- start start 2024-04-02T14:43:14.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6036f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d603c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 0x5596d603eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aac25a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5596d5f9c060 con 0x5596d603c670 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aac25a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5596d5f44560 con 0x5596d6036b10 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aac25a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5596d5f98010 con 0x5596d6037500 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6a9be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 0x5596d603eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aa3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6036f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aa3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6036f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:43672/0 (socket says 172.21.15.62:43672) 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aa3400640 1 -- 172.21.15.62:0/617499563 learned_addr learned my addr 172.21.15.62:0/617499563 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aa2a00640 1 --2- 172.21.15.62:0/617499563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d603c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aa3400640 1 -- 172.21.15.62:0/617499563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 msgr2=0x5596d603c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aa3400640 1 --2- 172.21.15.62:0/617499563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d603c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aa3400640 1 -- 172.21.15.62:0/617499563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 msgr2=0x5596d603eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.218+0000 7f6aa3400640 1 --2- 172.21.15.62:0/617499563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 0x5596d603eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.219+0000 7f6aa3400640 1 -- 172.21.15.62:0/617499563 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d5f66fd0 con 0x5596d6036b10 2024-04-02T14:43:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.219+0000 7f6aa3400640 1 --2- 172.21.15.62:0/617499563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6036f30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c02c710 tx=0x7f6a9c005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9a49a358481534a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.219+0000 7f6a9b400640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6a880a3bb0 0x7f6a880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.219+0000 7f6a93400640 1 -- 172.21.15.62:0/617499563 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c043070 con 0x5596d6036b10 2024-04-02T14:43:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.219+0000 7f6a9b400640 1 -- 172.21.15.62:0/2840892412 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f6a8c0d07a0 con 0x7f6aa4109210 2024-04-02T14:43:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.219+0000 7f6a93400640 1 -- 172.21.15.62:0/617499563 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6a9c032d40 con 0x5596d6036b10 2024-04-02T14:43:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.219+0000 7f6a93400640 1 -- 172.21.15.62:0/617499563 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c039040 con 0x5596d6036b10 2024-04-02T14:43:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.219+0000 7f6aa2a00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6a880a3bb0 0x7f6a880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.220+0000 7f6aac25a740 1 -- 172.21.15.62:0/617499563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 msgr2=0x5596d6036f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.220+0000 7f6aac25a740 1 --2- 172.21.15.62:0/617499563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6036f30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c02c710 tx=0x7f6a9c005f80 comp rx=0 tx=0).stop 2024-04-02T14:43:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.220+0000 7f6aa2a00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6a880a3bb0 0x7f6a880a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5596d6037900 tx=0x7f6a94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.220+0000 7f6aac25a740 1 -- 172.21.15.62:0/617499563 shutdown_connections 2024-04-02T14:43:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.220+0000 7f6aac25a740 1 --2- 172.21.15.62:0/617499563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 0x5596d603eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.220+0000 7f6aac25a740 1 --2- 172.21.15.62:0/617499563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d603c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.220+0000 7f6aac25a740 1 --2- 172.21.15.62:0/617499563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6036f30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.220+0000 7f6aac25a740 1 -- 172.21.15.62:0/617499563 >> 172.21.15.62:0/617499563 conn(0x5596d6032080 msgr2=0x5596d6032480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.220+0000 7f6aac25a740 1 -- 172.21.15.62:0/617499563 shutdown_connections 2024-04-02T14:43:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.220+0000 7f6aac25a740 1 -- 172.21.15.62:0/617499563 wait complete. 2024-04-02T14:43:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.221+0000 7f6aac25a740 1 -- start start 2024-04-02T14:43:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.221+0000 7f6aac25a740 1 Processor -- start 2024-04-02T14:43:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.221+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6041000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.221+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d6037500 0x5596d6041540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.221+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d603c670 0x5596d6048580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.221+0000 7f6aac25a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5596d603f5c0 con 0x5596d6037500 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.221+0000 7f6aac25a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5596d5f57920 con 0x5596d6036b10 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.221+0000 7f6aac25a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5596d603f740 con 0x5596d603c670 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.221+0000 7f6aac25a740 10 client.0 _ll_register_callbacks cb 0x5596d5f57310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6aa2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6041000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6a9be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d6037500 0x5596d6041540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6aa2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6041000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:43674/0 (socket says 172.21.15.62:43674) 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6aa2a00640 1 -- 172.21.15.62:0/3125575102 learned_addr learned my addr 172.21.15.62:0/3125575102 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6aa3400640 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d603c670 0x5596d6048580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6aa2a00640 1 -- 172.21.15.62:0/3125575102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d603c670 msgr2=0x5596d6048580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6aa2a00640 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d603c670 0x5596d6048580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6aa2a00640 1 -- 172.21.15.62:0/3125575102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d6037500 msgr2=0x5596d6041540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6aa2a00640 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d6037500 0x5596d6041540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6aa2a00640 1 -- 172.21.15.62:0/3125575102 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d5cb02d0 con 0x5596d6036b10 2024-04-02T14:43:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6a9be00640 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d6037500 0x5596d6041540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6aa3400640 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d603c670 0x5596d6048580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6aa2a00640 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6041000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6a94004600 tx=0x7f6a94004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0cd18296ea8a8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.222+0000 7f6a87e00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.223+0000 7f6a87e00640 1 -- 172.21.15.62:0/3125575102 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9401f630 con 0x5596d6036b10 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.223+0000 7f6aac25a740 1 -- 172.21.15.62:0/3125575102 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a9c032d40 con 0x5596d6036b10 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.223+0000 7f6a87e00640 1 -- 172.21.15.62:0/3125575102 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f6a9401f7d0 con 0x5596d6036b10 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.223+0000 7f6a87e00640 1 -- 172.21.15.62:0/3125575102 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a9401f9d0 con 0x5596d6036b10 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.223+0000 7f6aac25a740 10 client.24995 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.223+0000 7f6aac25a740 1 -- 172.21.15.62:0/3125575102 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596d5d7a6b0 con 0x5596d6036b10 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.223+0000 7f6a87e00640 1 -- 172.21.15.62:0/3125575102 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6a9c032d40 con 0x5596d6036b10 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.223+0000 7f6a87e00640 1 -- 172.21.15.62:0/3125575102 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a94027070 con 0x5596d6036b10 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.224+0000 7f6aac25a740 10 client.24995 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.224+0000 7f6aac25a740 10 client.24995 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T14:43:14.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.224+0000 7f6aac25a740 20 client.24995 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:14.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.224+0000 7f6aac25a740 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5596d604de90 0x5596d604f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.224+0000 7f6aac25a740 4 client.24995 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:43:14.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.224+0000 7f6aac25a740 1 -- 172.21.15.62:0/3125575102 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596d5fa95a0 con 0x5596d604de90 2024-04-02T14:43:14.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.224+0000 7f6a9be00640 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5596d604de90 0x5596d604f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.225+0000 7f6a9be00640 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5596d604de90 0x5596d604f300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=136df45e61d55466 server_cookie=e9e447526c7f9766 in_seq=0 out_seq=0 2024-04-02T14:43:14.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.225+0000 7f6a87e00640 10 client.24995 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:43:14.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.225+0000 7f6a87e00640 1 -- 172.21.15.62:0/3125575102 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596d5fa95a0 con 0x5596d604de90 2024-04-02T14:43:14.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.225+0000 7f6a87e00640 10 client.24995 handle_command_reply: tid=0 2024-04-02T14:43:14.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 1 client.24995 shutdown 2024-04-02T14:43:14.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 1 -- 172.21.15.62:0/3125575102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 msgr2=0x5596d6041000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6041000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6a94004600 tx=0x7f6a94004730 comp rx=0 tx=0).stop 2024-04-02T14:43:14.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 1 -- 172.21.15.62:0/3125575102 shutdown_connections 2024-04-02T14:43:14.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5596d604de90 0x5596d604f300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d603c670 0x5596d6048580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d6037500 0x5596d6041540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 1 --2- 172.21.15.62:0/3125575102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d6041000 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6a87e00640 0 client.24995 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:43:14.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 1 -- 172.21.15.62:0/3125575102 >> 172.21.15.62:0/3125575102 conn(0x5596d6032080 msgr2=0x5596d60311c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 1 -- 172.21.15.62:0/3125575102 shutdown_connections 2024-04-02T14:43:14.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 1 -- 172.21.15.62:0/3125575102 wait complete. 2024-04-02T14:43:14.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.226+0000 7f6aac25a740 20 client.24995 trim_cache size 0 max 16384 2024-04-02T14:43:14.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.246+0000 7f6aac25a740 1 Processor -- start 2024-04-02T14:43:14.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.246+0000 7f6aac25a740 1 -- start start 2024-04-02T14:43:14.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.246+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d603ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.246+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d6048540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.247+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 0x5596d603fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.247+0000 7f6aac25a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5596d604d640 con 0x5596d603c670 2024-04-02T14:43:14.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.247+0000 7f6aac25a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5596d604dd10 con 0x5596d6036b10 2024-04-02T14:43:14.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.247+0000 7f6aac25a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5596d604e3e0 con 0x5596d6037500 2024-04-02T14:43:14.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.247+0000 7f6aa3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 0x5596d603fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.247+0000 7f6aa2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d603ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.247+0000 7f6a9be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d6048540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.247+0000 7f6aa3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 0x5596d603fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41224/0 (socket says 172.21.15.62:41224) 2024-04-02T14:43:14.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.247+0000 7f6aa3400640 1 -- 172.21.15.62:0/1936597938 learned_addr learned my addr 172.21.15.62:0/1936597938 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.247+0000 7f6a9be00640 1 -- 172.21.15.62:0/1936597938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 msgr2=0x5596d603ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.247+0000 7f6a9be00640 1 --2- 172.21.15.62:0/1936597938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d603ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.248+0000 7f6a9be00640 1 -- 172.21.15.62:0/1936597938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 msgr2=0x5596d603fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.248+0000 7f6a9be00640 1 --2- 172.21.15.62:0/1936597938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 0x5596d603fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.248+0000 7f6a9be00640 1 -- 172.21.15.62:0/1936597938 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d60306c0 con 0x5596d6037500 2024-04-02T14:43:14.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.248+0000 7f6aa3400640 1 --2- 172.21.15.62:0/1936597938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 0x5596d603fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:14.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.248+0000 7f6a9be00640 1 --2- 172.21.15.62:0/1936597938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d6048540 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00c6a0 tx=0x7f6a8c0bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=327ae8087c442309 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.248+0000 7f6a92a00640 1 -- 172.21.15.62:0/1936597938 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c09c460 con 0x5596d6037500 2024-04-02T14:43:14.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.248+0000 7f6a92a00640 1 -- 172.21.15.62:0/1936597938 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6a8c0bd0b0 con 0x5596d6037500 2024-04-02T14:43:14.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.248+0000 7f6a92a00640 1 -- 172.21.15.62:0/1936597938 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c0d0b90 con 0x5596d6037500 2024-04-02T14:43:14.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.249+0000 7f6aac25a740 1 -- 172.21.15.62:0/1936597938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 msgr2=0x5596d6048540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.249+0000 7f6aac25a740 1 --2- 172.21.15.62:0/1936597938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d6048540 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00c6a0 tx=0x7f6a8c0bd040 comp rx=0 tx=0).stop 2024-04-02T14:43:14.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.249+0000 7f6aac25a740 1 -- 172.21.15.62:0/1936597938 shutdown_connections 2024-04-02T14:43:14.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.249+0000 7f6aac25a740 1 --2- 172.21.15.62:0/1936597938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d603c670 0x5596d603fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.249+0000 7f6aac25a740 1 --2- 172.21.15.62:0/1936597938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d6048540 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.249+0000 7f6aac25a740 1 --2- 172.21.15.62:0/1936597938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d6036b10 0x5596d603ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.249+0000 7f6aac25a740 1 -- 172.21.15.62:0/1936597938 >> 172.21.15.62:0/1936597938 conn(0x5596d6032080 msgr2=0x5596d6037f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.249+0000 7f6aac25a740 1 -- 172.21.15.62:0/1936597938 shutdown_connections 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.249+0000 7f6aac25a740 1 -- 172.21.15.62:0/1936597938 wait complete. 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.249+0000 7f6aac25a740 1 -- start start 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.249+0000 7f6aac25a740 1 Processor -- start 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d6036b10 0x5596d6046530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d60afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6aac25a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d603c670 0x5596d60b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6aac25a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5596d604f180 con 0x5596d6036b10 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6aac25a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5596d604eab0 con 0x5596d603c670 2024-04-02T14:43:14.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6aac25a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5596d604f850 con 0x5596d6037500 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6aac25a740 10 client.0 _ll_register_callbacks cb 0x7f6a940044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6a9be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d60afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6aa2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d6036b10 0x5596d6046530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6a9be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d60afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:36172/0 (socket says 172.21.15.62:36172) 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6a9be00640 1 -- 172.21.15.62:0/1369903475 learned_addr learned my addr 172.21.15.62:0/1369903475 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6aa3400640 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d603c670 0x5596d60b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6a9be00640 1 -- 172.21.15.62:0/1369903475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d603c670 msgr2=0x5596d60b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6a9be00640 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d603c670 0x5596d60b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6a9be00640 1 -- 172.21.15.62:0/1369903475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d6036b10 msgr2=0x5596d6046530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6a9be00640 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d6036b10 0x5596d6046530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.250+0000 7f6a9be00640 1 -- 172.21.15.62:0/1369903475 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d60b2a30 con 0x5596d6037500 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.251+0000 7f6aa3400640 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d603c670 0x5596d60b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.251+0000 7f6a9be00640 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d60afe20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009830 tx=0x7f6a8c0d6530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdef6a706ab0f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.251+0000 7f6a87e00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.251+0000 7f6a87e00640 1 -- 172.21.15.62:0/1369903475 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c0b5070 con 0x5596d6037500 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.251+0000 7f6aac25a740 1 -- 172.21.15.62:0/1369903475 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596d5fa38f0 con 0x5596d6037500 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.251+0000 7f6a87e00640 1 -- 172.21.15.62:0/1369903475 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f6a8c0d0b90 con 0x5596d6037500 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.251+0000 7f6a87e00640 1 -- 172.21.15.62:0/1369903475 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a8c0ac050 con 0x5596d6037500 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.251+0000 7f6aac25a740 10 client.24988 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.251+0000 7f6aac25a740 1 -- 172.21.15.62:0/1369903475 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596d60b3d30 con 0x5596d6037500 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.251+0000 7f6a87e00640 1 -- 172.21.15.62:0/1369903475 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x5596d5fa38f0 con 0x5596d6037500 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.252+0000 7f6a87e00640 1 -- 172.21.15.62:0/1369903475 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a8c09bf50 con 0x5596d6037500 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.252+0000 7f6aac25a740 10 client.24988 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.252+0000 7f6aac25a740 10 client.24988 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.252+0000 7f6aac25a740 20 client.24988 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.252+0000 7f6aac25a740 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5596d60b8460 0x5596d60b7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.252+0000 7f6aac25a740 4 client.24988 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T14:43:14.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.252+0000 7f6aac25a740 1 -- 172.21.15.62:0/1369903475 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "status"}) -- 0x5596d5f217d0 con 0x5596d60b8460 2024-04-02T14:43:14.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.252+0000 7f6aa2a00640 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5596d60b8460 0x5596d60b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6a87e00640 10 client.24988 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:43:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aa2a00640 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5596d60b8460 0x5596d60b7170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6bd0ec49134f5ef0 server_cookie=d3126f49f24a7263 in_seq=0 out_seq=0 2024-04-02T14:43:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6a87e00640 1 -- 172.21.15.62:0/1369903475 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5596d5f217d0 con 0x5596d60b8460 2024-04-02T14:43:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6a87e00640 10 client.24988 handle_command_reply: tid=0 2024-04-02T14:43:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aac25a740 1 client.24988 shutdown 2024-04-02T14:43:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aac25a740 1 -- 172.21.15.62:0/1369903475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 msgr2=0x5596d60afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aac25a740 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d60afe20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009830 tx=0x7f6a8c0d6530 comp rx=0 tx=0).stop 2024-04-02T14:43:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aac25a740 1 -- 172.21.15.62:0/1369903475 shutdown_connections 2024-04-02T14:43:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aac25a740 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5596d60b8460 0x5596d60b7170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aac25a740 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596d603c670 0x5596d60b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6a87e00640 0 client.24988 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:43:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aac25a740 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d6037500 0x5596d60afe20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aac25a740 1 --2- 172.21.15.62:0/1369903475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5596d6036b10 0x5596d6046530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aac25a740 1 -- 172.21.15.62:0/1369903475 >> 172.21.15.62:0/1369903475 conn(0x5596d6032080 msgr2=0x5596d6032460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aac25a740 1 -- 172.21.15.62:0/1369903475 shutdown_connections 2024-04-02T14:43:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.253+0000 7f6aac25a740 1 -- 172.21.15.62:0/1369903475 wait complete. 2024-04-02T14:43:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aac25a740 20 client.24988 trim_cache size 0 max 16384 2024-04-02T14:43:14.256 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:43:14.256 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:43:14.256 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 4, 2024-04-02T14:43:14.256 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24533, 2024-04-02T14:43:14.257 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-02T14:43:14.257 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-02T14:43:14.257 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-02T14:43:14.257 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 105.172040942, 2024-04-02T14:43:14.257 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-04-02T14:43:14.257 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-04-02T14:43:14.257 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-04-02T14:43:14.257 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 132.518342612 2024-04-02T14:43:14.257 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:43:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 -- 172.21.15.62:0/2840892412 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6a880a3bb0 msgr2=0x7f6a880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6a880a3bb0 0x7f6a880a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5596d6037900 tx=0x7f6a94005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 -- 172.21.15.62:0/2840892412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aa4109210 msgr2=0x7f6aa419e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aa4109210 0x7f6aa419e890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00e9f0 tx=0x7f6a8c00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 -- 172.21.15.62:0/2840892412 shutdown_connections 2024-04-02T14:43:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6a880a3bb0 0x7f6a880a6070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aa4109bd0 0x7f6aa41a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aa4109210 0x7f6aa419e890 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 --2- 172.21.15.62:0/2840892412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6aa4108020 0x7f6aa419e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 -- 172.21.15.62:0/2840892412 >> 172.21.15.62:0/2840892412 conn(0x7f6aa4076fd0 msgr2=0x7f6aa410c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 -- 172.21.15.62:0/2840892412 shutdown_connections 2024-04-02T14:43:14.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.254+0000 7f6aa3e00640 1 -- 172.21.15.62:0/2840892412 wait complete. 2024-04-02T14:43:14.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:14 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2694422097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:14.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:14 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2784594622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:14.268 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-02T14:43:14.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.484+0000 7fa25be00640 1 -- 172.21.15.62:0/3901369761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa25c108020 msgr2=0x7fa25c108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.484+0000 7fa25be00640 1 --2- 172.21.15.62:0/3901369761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa25c108020 0x7fa25c108440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009860 tx=0x7fa24c02bb70 comp rx=0 tx=0).stop 2024-04-02T14:43:14.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.484+0000 7fa25be00640 1 -- 172.21.15.62:0/3901369761 shutdown_connections 2024-04-02T14:43:14.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.484+0000 7fa25be00640 1 --2- 172.21.15.62:0/3901369761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa25c109bd0 0x7fa25c110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.484+0000 7fa25be00640 1 --2- 172.21.15.62:0/3901369761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c109210 0x7fa25c109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.484+0000 7fa25be00640 1 --2- 172.21.15.62:0/3901369761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa25c108020 0x7fa25c108440 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.484+0000 7fa25be00640 1 -- 172.21.15.62:0/3901369761 >> 172.21.15.62:0/3901369761 conn(0x7fa25c076fd0 msgr2=0x7fa25c0793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.485+0000 7fa25be00640 1 -- 172.21.15.62:0/3901369761 shutdown_connections 2024-04-02T14:43:14.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.485+0000 7fa25be00640 1 -- 172.21.15.62:0/3901369761 wait complete. 2024-04-02T14:43:14.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.486+0000 7fa25be00640 1 Processor -- start 2024-04-02T14:43:14.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.486+0000 7fa25be00640 1 -- start start 2024-04-02T14:43:14.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.486+0000 7fa25be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa25c108020 0x7fa25c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.486+0000 7fa25be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa25c109210 0x7fa25c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c109bd0 0x7fa25c1a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c113dd0 con 0x7fa25c109bd0 2024-04-02T14:43:14.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c113c50 con 0x7fa25c109210 2024-04-02T14:43:14.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c113f50 con 0x7fa25c108020 2024-04-02T14:43:14.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa25c108020 0x7fa25c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c109bd0 0x7fa25c1a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa25c108020 0x7fa25c19e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:36182/0 (socket says 172.21.15.62:36182) 2024-04-02T14:43:14.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25aa00640 1 -- 172.21.15.62:0/1993476748 learned_addr learned my addr 172.21.15.62:0/1993476748 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa252000640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa25c109210 0x7fa25c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25aa00640 1 -- 172.21.15.62:0/1993476748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa25c109210 msgr2=0x7fa25c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25aa00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa25c109210 0x7fa25c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25aa00640 1 -- 172.21.15.62:0/1993476748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c109bd0 msgr2=0x7fa25c1a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25aa00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c109bd0 0x7fa25c1a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.487+0000 7fa25aa00640 1 -- 172.21.15.62:0/1993476748 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c1a6000 con 0x7fa25c108020 2024-04-02T14:43:14.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.488+0000 7fa25aa00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa25c108020 0x7fa25c19e3e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa24c034a80 tx=0x7fa24c035f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.488+0000 7fa253e00640 1 -- 172.21.15.62:0/1993476748 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c041020 con 0x7fa25c108020 2024-04-02T14:43:14.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.488+0000 7fa253e00640 1 -- 172.21.15.62:0/1993476748 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa24c02ca10 con 0x7fa25c108020 2024-04-02T14:43:14.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.488+0000 7fa253e00640 1 -- 172.21.15.62:0/1993476748 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c039040 con 0x7fa25c108020 2024-04-02T14:43:14.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.488+0000 7fa25be00640 1 -- 172.21.15.62:0/1993476748 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25c1a6290 con 0x7fa25c108020 2024-04-02T14:43:14.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.488+0000 7fa25be00640 1 -- 172.21.15.62:0/1993476748 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25c1a6780 con 0x7fa25c108020 2024-04-02T14:43:14.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.491+0000 7fa264453740 1 Processor -- start 2024-04-02T14:43:14.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.491+0000 7fa253e00640 1 -- 172.21.15.62:0/1993476748 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa24c044020 con 0x7fa25c108020 2024-04-02T14:43:14.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.491+0000 7fa264453740 1 -- start start 2024-04-02T14:43:14.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.491+0000 7fa264453740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d31698bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa264453740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 0x55d316991000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa264453740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d316993b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa264453740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d3168f1060 con 0x55d31698c500 2024-04-02T14:43:14.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa264453740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d316899560 con 0x55d31698bb10 2024-04-02T14:43:14.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa264453740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d3168ed010 con 0x55d316991670 2024-04-02T14:43:14.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa25aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 0x55d316991000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa25aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 0x55d316991000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41264/0 (socket says 172.21.15.62:41264) 2024-04-02T14:43:14.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa25aa00640 1 -- 172.21.15.62:0/1120570427 learned_addr learned my addr 172.21.15.62:0/1120570427 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa25b400640 1 --2- 172.21.15.62:0/1120570427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d31698bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa252000640 1 --2- 172.21.15.62:0/1120570427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d316993b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa25b400640 1 -- 172.21.15.62:0/1120570427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 msgr2=0x55d316993b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa25b400640 1 --2- 172.21.15.62:0/1120570427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d316993b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa25b400640 1 -- 172.21.15.62:0/1120570427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 msgr2=0x55d316991000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa25b400640 1 --2- 172.21.15.62:0/1120570427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 0x55d316991000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.492+0000 7fa25b400640 1 -- 172.21.15.62:0/1120570427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3168bbfd0 con 0x55d31698bb10 2024-04-02T14:43:14.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.493+0000 7fa25b400640 1 --2- 172.21.15.62:0/1120570427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d31698bf30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa254009f00 tx=0x7fa254011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c2850f205168d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.493+0000 7fa253e00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400a3bb0 0x7fa2400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.493+0000 7fa24be00640 1 -- 172.21.15.62:0/1120570427 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254004d20 con 0x55d31698bb10 2024-04-02T14:43:14.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.493+0000 7fa24be00640 1 -- 172.21.15.62:0/1120570427 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa254004030 con 0x55d31698bb10 2024-04-02T14:43:14.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.493+0000 7fa24be00640 1 -- 172.21.15.62:0/1120570427 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254005280 con 0x55d31698bb10 2024-04-02T14:43:14.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.493+0000 7fa253e00640 1 -- 172.21.15.62:0/1993476748 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fa24c0f1c50 con 0x7fa25c108020 2024-04-02T14:43:14.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.493+0000 7fa25aa00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400a3bb0 0x7fa2400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.494+0000 7fa264453740 1 -- 172.21.15.62:0/1120570427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 msgr2=0x55d31698bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.494+0000 7fa264453740 1 --2- 172.21.15.62:0/1120570427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d31698bf30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa254009f00 tx=0x7fa254011810 comp rx=0 tx=0).stop 2024-04-02T14:43:14.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.494+0000 7fa25aa00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400a3bb0 0x7fa2400a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0bdef0 tx=0x7fa24c0f5720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.494+0000 7fa264453740 1 -- 172.21.15.62:0/1120570427 shutdown_connections 2024-04-02T14:43:14.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.494+0000 7fa264453740 1 --2- 172.21.15.62:0/1120570427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d316993b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.494+0000 7fa264453740 1 --2- 172.21.15.62:0/1120570427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 0x55d316991000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.494+0000 7fa264453740 1 --2- 172.21.15.62:0/1120570427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d31698bf30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.494+0000 7fa264453740 1 -- 172.21.15.62:0/1120570427 >> 172.21.15.62:0/1120570427 conn(0x55d316987080 msgr2=0x55d316987480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.494+0000 7fa264453740 1 -- 172.21.15.62:0/1120570427 shutdown_connections 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.494+0000 7fa264453740 1 -- 172.21.15.62:0/1120570427 wait complete. 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.494+0000 7fa264453740 1 -- start start 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa264453740 1 Processor -- start 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa264453740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698bb10 0x55d316996020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa264453740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698c500 0x55d316996560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa264453740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d31699d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa264453740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d316994620 con 0x55d31698bb10 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa264453740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d3168ac920 con 0x55d31698c500 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa264453740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d3169947a0 con 0x55d316991670 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa264453740 10 client.0 _ll_register_callbacks cb 0x55d3168ac310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa25b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698bb10 0x55d316996020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa25b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698bb10 0x55d316996020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41280/0 (socket says 172.21.15.62:41280) 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa25b400640 1 -- 172.21.15.62:0/2151626208 learned_addr learned my addr 172.21.15.62:0/2151626208 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.495+0000 7fa252000640 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698c500 0x55d316996560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa25b400640 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d31699d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa252000640 1 -- 172.21.15.62:0/2151626208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 msgr2=0x55d31699d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa252000640 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d31699d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa252000640 1 -- 172.21.15.62:0/2151626208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698bb10 msgr2=0x55d316996020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa252000640 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698bb10 0x55d316996020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa252000640 1 -- 172.21.15.62:0/2151626208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3166052d0 con 0x55d31698c500 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa25b400640 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698bb10 0x55d316996020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa252000640 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698c500 0x55d316996560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa25c19f8f0 tx=0x7fa244011b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b02bbb534d82a289 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa248c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa248c00640 1 -- 172.21.15.62:0/2151626208 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24401e060 con 0x55d31698c500 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa248c00640 1 -- 172.21.15.62:0/2151626208 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fa24400fbc0 con 0x55d31698c500 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.496+0000 7fa264453740 1 -- 172.21.15.62:0/2151626208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa254004030 con 0x55d31698c500 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.497+0000 7fa248c00640 1 -- 172.21.15.62:0/2151626208 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa24400fe00 con 0x55d31698c500 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.497+0000 7fa264453740 10 client.25013 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.497+0000 7fa264453740 1 -- 172.21.15.62:0/2151626208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d3166cf6b0 con 0x55d31698c500 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.497+0000 7fa248c00640 1 -- 172.21.15.62:0/2151626208 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa254004030 con 0x55d31698c500 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.497+0000 7fa248c00640 1 -- 172.21.15.62:0/2151626208 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa244019070 con 0x55d31698c500 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.497+0000 7fa264453740 10 client.25013 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.497+0000 7fa264453740 10 client.25013 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.497+0000 7fa264453740 20 client.25013 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.497+0000 7fa264453740 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55d3169a2df0 0x55d3169a4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.497+0000 7fa264453740 4 client.25013 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.497+0000 7fa264453740 1 -- 172.21.15.62:0/2151626208 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3168fe5a0 con 0x55d3169a2df0 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.498+0000 7fa25b400640 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55d3169a2df0 0x55d3169a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.498+0000 7fa25b400640 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55d3169a2df0 0x55d3169a4260 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6676e7bf76f7ad34 server_cookie=99676cbb448bc999 in_seq=0 out_seq=0 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.498+0000 7fa248c00640 10 client.25013 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa248c00640 1 -- 172.21.15.62:0/2151626208 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3168fe5a0 con 0x55d3169a2df0 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa248c00640 10 client.25013 handle_command_reply: tid=0 2024-04-02T14:43:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa264453740 1 client.25013 shutdown 2024-04-02T14:43:14.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa264453740 1 -- 172.21.15.62:0/2151626208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698c500 msgr2=0x55d316996560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa264453740 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698c500 0x55d316996560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa25c19f8f0 tx=0x7fa244011b30 comp rx=0 tx=0).stop 2024-04-02T14:43:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa264453740 1 -- 172.21.15.62:0/2151626208 shutdown_connections 2024-04-02T14:43:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa264453740 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55d3169a2df0 0x55d3169a4260 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa264453740 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d31699d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa264453740 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698c500 0x55d316996560 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa248c00640 0 client.25013 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:43:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa264453740 1 --2- 172.21.15.62:0/2151626208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698bb10 0x55d316996020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.499+0000 7fa264453740 1 -- 172.21.15.62:0/2151626208 >> 172.21.15.62:0/2151626208 conn(0x55d316987080 msgr2=0x55d3169861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.500+0000 7fa264453740 1 -- 172.21.15.62:0/2151626208 shutdown_connections 2024-04-02T14:43:14.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.500+0000 7fa264453740 1 -- 172.21.15.62:0/2151626208 wait complete. 2024-04-02T14:43:14.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.500+0000 7fa264453740 20 client.25013 trim_cache size 0 max 16384 2024-04-02T14:43:14.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:14 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2694422097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:14.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:14 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2784594622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:14.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.519+0000 7fa264453740 1 Processor -- start 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa264453740 1 -- start start 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa264453740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d316994500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa264453740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d31698c500 0x55d31699d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa264453740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d316991670 0x55d316987460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa264453740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d3169a25a0 con 0x55d316991670 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa264453740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d3169a2c70 con 0x55d31698bb10 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa264453740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d3169a3340 con 0x55d31698c500 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa252000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d31698c500 0x55d31699d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa252000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d31698c500 0x55d31699d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:36200/0 (socket says 172.21.15.62:36200) 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa252000640 1 -- 172.21.15.62:0/3447754176 learned_addr learned my addr 172.21.15.62:0/3447754176 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa25b400640 1 --2- 172.21.15.62:0/3447754176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d316994500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa25b400640 1 --2- 172.21.15.62:0/3447754176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d316991670 0x55d316987460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa252000640 1 -- 172.21.15.62:0/3447754176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 msgr2=0x55d316994500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.520+0000 7fa252000640 1 --2- 172.21.15.62:0/3447754176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d316994500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.521+0000 7fa252000640 1 -- 172.21.15.62:0/3447754176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d316991670 msgr2=0x55d316987460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.521+0000 7fa252000640 1 --2- 172.21.15.62:0/3447754176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d316991670 0x55d316987460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.521+0000 7fa252000640 1 -- 172.21.15.62:0/3447754176 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d316987df0 con 0x55d31698c500 2024-04-02T14:43:14.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.521+0000 7fa252000640 1 --2- 172.21.15.62:0/3447754176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d31698c500 0x55d31699d560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa244006010 tx=0x7fa24400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc52df7a58541dde server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.521+0000 7fa24b400640 1 -- 172.21.15.62:0/3447754176 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244029070 con 0x55d31698c500 2024-04-02T14:43:14.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.521+0000 7fa24b400640 1 -- 172.21.15.62:0/3447754176 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa24401a630 con 0x55d31698c500 2024-04-02T14:43:14.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.521+0000 7fa24b400640 1 -- 172.21.15.62:0/3447754176 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24401e040 con 0x55d31698c500 2024-04-02T14:43:14.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.522+0000 7fa264453740 1 -- 172.21.15.62:0/3447754176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d31698c500 msgr2=0x55d31699d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.522+0000 7fa264453740 1 --2- 172.21.15.62:0/3447754176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d31698c500 0x55d31699d560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa244006010 tx=0x7fa24400f790 comp rx=0 tx=0).stop 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.522+0000 7fa264453740 1 -- 172.21.15.62:0/3447754176 shutdown_connections 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.522+0000 7fa264453740 1 --2- 172.21.15.62:0/3447754176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d316991670 0x55d316987460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.522+0000 7fa264453740 1 --2- 172.21.15.62:0/3447754176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d31698c500 0x55d31699d560 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.522+0000 7fa264453740 1 --2- 172.21.15.62:0/3447754176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d316994500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.522+0000 7fa264453740 1 -- 172.21.15.62:0/3447754176 >> 172.21.15.62:0/3447754176 conn(0x55d316987080 msgr2=0x55d316981de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.522+0000 7fa264453740 1 -- 172.21.15.62:0/3447754176 shutdown_connections 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.522+0000 7fa264453740 1 -- 172.21.15.62:0/3447754176 wait complete. 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.522+0000 7fa264453740 1 -- start start 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.522+0000 7fa264453740 1 Processor -- start 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa264453740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d316999050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa264453740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 0x55d316999590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa264453740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d31699c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa264453740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d3169a40e0 con 0x55d31698c500 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa264453740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d3169a3a10 con 0x55d31698bb10 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa264453740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d3169a47b0 con 0x55d316991670 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa264453740 10 client.0 _ll_register_callbacks cb 0x7fa244009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa252000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 0x55d316999590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa252000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 0x55d316999590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41308/0 (socket says 172.21.15.62:41308) 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa252000640 1 -- 172.21.15.62:0/3161352260 learned_addr learned my addr 172.21.15.62:0/3161352260 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa25b400640 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d31699c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa25b400640 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d316999050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.523+0000 7fa252000640 1 -- 172.21.15.62:0/3161352260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 msgr2=0x55d31699c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa252000640 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d31699c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa252000640 1 -- 172.21.15.62:0/3161352260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 msgr2=0x55d316999050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa252000640 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d316999050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa252000640 1 -- 172.21.15.62:0/3161352260 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d31699cae0 con 0x55d31698c500 2024-04-02T14:43:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa25b400640 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d316999050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa252000640 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 0x55d316999590 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa244004340 tx=0x7fa244004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7514006f143d66f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa248c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa248c00640 1 -- 172.21.15.62:0/3161352260 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244029070 con 0x55d31698c500 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa248c00640 1 -- 172.21.15.62:0/3161352260 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fa24401a9f0 con 0x55d31698c500 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa264453740 1 -- 172.21.15.62:0/3161352260 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa24401a630 con 0x55d31698c500 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa248c00640 1 -- 172.21.15.62:0/3161352260 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa24403e050 con 0x55d31698c500 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa264453740 10 client.14901 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.524+0000 7fa264453740 1 -- 172.21.15.62:0/3161352260 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d316a090e0 con 0x55d31698c500 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.525+0000 7fa248c00640 1 -- 172.21.15.62:0/3161352260 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa24401a630 con 0x55d31698c500 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.525+0000 7fa248c00640 1 -- 172.21.15.62:0/3161352260 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa244019070 con 0x55d31698c500 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.525+0000 7fa264453740 10 client.14901 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.525+0000 7fa264453740 10 client.14901 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.525+0000 7fa264453740 20 client.14901 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.525+0000 7fa264453740 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55d316a0dad0 0x55d316a0c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.525+0000 7fa264453740 4 client.14901 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T14:43:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.525+0000 7fa264453740 1 -- 172.21.15.62:0/3161352260 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "status"}) -- 0x55d3168f88f0 con 0x55d316a0dad0 2024-04-02T14:43:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.525+0000 7fa25b400640 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55d316a0dad0 0x55d316a0c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.526+0000 7fa25b400640 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55d316a0dad0 0x55d316a0c7e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8b3bc2f3540b4f40 server_cookie=162e347b0e3674f5 in_seq=0 out_seq=0 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.526+0000 7fa248c00640 10 client.14901 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.526+0000 7fa248c00640 1 -- 172.21.15.62:0/3161352260 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d3168f88f0 con 0x55d316a0dad0 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.526+0000 7fa248c00640 10 client.14901 handle_command_reply: tid=0 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.526+0000 7fa264453740 1 client.14901 shutdown 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.526+0000 7fa264453740 1 -- 172.21.15.62:0/3161352260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 msgr2=0x55d316999590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.526+0000 7fa264453740 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 0x55d316999590 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa244004340 tx=0x7fa244004540 comp rx=0 tx=0).stop 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa264453740 1 -- 172.21.15.62:0/3161352260 shutdown_connections 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa264453740 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55d316a0dad0 0x55d316a0c7e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa264453740 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d316991670 0x55d31699c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa264453740 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d31698c500 0x55d316999590 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa264453740 1 --2- 172.21.15.62:0/3161352260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d31698bb10 0x55d316999050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa248c00640 0 client.14901 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:43:14.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa264453740 1 -- 172.21.15.62:0/3161352260 >> 172.21.15.62:0/3161352260 conn(0x55d316987080 msgr2=0x55d316987460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa264453740 1 -- 172.21.15.62:0/3161352260 shutdown_connections 2024-04-02T14:43:14.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa264453740 1 -- 172.21.15.62:0/3161352260 wait complete. 2024-04-02T14:43:14.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa264453740 20 client.14901 trim_cache size 0 max 16384 2024-04-02T14:43:14.529 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:43:14.529 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:43:14.529 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 3, 2024-04-02T14:43:14.529 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24535, 2024-04-02T14:43:14.529 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-02T14:43:14.529 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-02T14:43:14.530 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-02T14:43:14.530 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 106.462569778, 2024-04-02T14:43:14.530 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-04-02T14:43:14.530 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-04-02T14:43:14.530 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-04-02T14:43:14.530 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 129.94841313800001 2024-04-02T14:43:14.530 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa25be00640 1 -- 172.21.15.62:0/1993476748 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400a3bb0 msgr2=0x7fa2400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa25be00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400a3bb0 0x7fa2400a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0bdef0 tx=0x7fa24c0f5720 comp rx=0 tx=0).stop 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa25be00640 1 -- 172.21.15.62:0/1993476748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa25c108020 msgr2=0x7fa25c19e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.527+0000 7fa25be00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa25c108020 0x7fa25c19e3e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa24c034a80 tx=0x7fa24c035f10 comp rx=0 tx=0).stop 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.528+0000 7fa25be00640 1 -- 172.21.15.62:0/1993476748 shutdown_connections 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.528+0000 7fa25be00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2400a3bb0 0x7fa2400a6070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.528+0000 7fa25be00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c109bd0 0x7fa25c1a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.528+0000 7fa25be00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa25c109210 0x7fa25c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.528+0000 7fa25be00640 1 --2- 172.21.15.62:0/1993476748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa25c108020 0x7fa25c19e3e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.528+0000 7fa25be00640 1 -- 172.21.15.62:0/1993476748 >> 172.21.15.62:0/1993476748 conn(0x7fa25c076fd0 msgr2=0x7fa25c10c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.528+0000 7fa25be00640 1 -- 172.21.15.62:0/1993476748 shutdown_connections 2024-04-02T14:43:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.528+0000 7fa25be00640 1 -- 172.21.15.62:0/1993476748 wait complete. 2024-04-02T14:43:14.544 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-02T14:43:14.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:14 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2694422097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:14.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:14 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2784594622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:14.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.750+0000 7fcd06e00640 1 -- 172.21.15.62:0/3717180907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00109210 msgr2=0x7fcd00109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.750+0000 7fcd06e00640 1 --2- 172.21.15.62:0/3717180907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00109210 0x7fcd00109690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fccec009900 tx=0x7fccec02bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.750+0000 7fcd06e00640 1 -- 172.21.15.62:0/3717180907 shutdown_connections 2024-04-02T14:43:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.750+0000 7fcd06e00640 1 --2- 172.21.15.62:0/3717180907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd00109bd0 0x7fcd00110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.750+0000 7fcd06e00640 1 --2- 172.21.15.62:0/3717180907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00109210 0x7fcd00109690 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.750+0000 7fcd06e00640 1 --2- 172.21.15.62:0/3717180907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd00108020 0x7fcd00108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.750+0000 7fcd06e00640 1 -- 172.21.15.62:0/3717180907 >> 172.21.15.62:0/3717180907 conn(0x7fcd00076fd0 msgr2=0x7fcd000793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.750+0000 7fcd06e00640 1 -- 172.21.15.62:0/3717180907 shutdown_connections 2024-04-02T14:43:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.750+0000 7fcd06e00640 1 -- 172.21.15.62:0/3717180907 wait complete. 2024-04-02T14:43:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.751+0000 7fcd06e00640 1 Processor -- start 2024-04-02T14:43:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.751+0000 7fcd06e00640 1 -- start start 2024-04-02T14:43:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd06e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00108020 0x7fcd0019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd06e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd00109210 0x7fcd0019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd06e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd00109bd0 0x7fcd001a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd06e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00113d80 con 0x7fcd00108020 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd06e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00113c00 con 0x7fcd00109bd0 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd06e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00113f00 con 0x7fcd00109210 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd05a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00108020 0x7fcd0019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd05a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00108020 0x7fcd0019e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41322/0 (socket says 172.21.15.62:41322) 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd05a00640 1 -- 172.21.15.62:0/671059300 learned_addr learned my addr 172.21.15.62:0/671059300 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd06400640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd00109bd0 0x7fcd001a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd05000640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd00109210 0x7fcd0019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd05a00640 1 -- 172.21.15.62:0/671059300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd00109210 msgr2=0x7fcd0019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd05a00640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd00109210 0x7fcd0019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd05a00640 1 -- 172.21.15.62:0/671059300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd00109bd0 msgr2=0x7fcd001a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd05a00640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd00109bd0 0x7fcd001a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.752+0000 7fcd05a00640 1 -- 172.21.15.62:0/671059300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd001a5fa0 con 0x7fcd00108020 2024-04-02T14:43:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.753+0000 7fcd05a00640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00108020 0x7fcd0019e380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fccf000ebc0 tx=0x7fccf000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.753+0000 7fccfe000640 1 -- 172.21.15.62:0/671059300 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0003b70 con 0x7fcd00108020 2024-04-02T14:43:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.753+0000 7fcd06e00640 1 -- 172.21.15.62:0/671059300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd001a6290 con 0x7fcd00108020 2024-04-02T14:43:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.753+0000 7fccfe000640 1 -- 172.21.15.62:0/671059300 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fccf0003d10 con 0x7fcd00108020 2024-04-02T14:43:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.753+0000 7fccfe000640 1 -- 172.21.15.62:0/671059300 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0019980 con 0x7fcd00108020 2024-04-02T14:43:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.753+0000 7fcd06e00640 1 -- 172.21.15.62:0/671059300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd001a67b0 con 0x7fcd00108020 2024-04-02T14:43:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.757+0000 7fcd0b1b3740 1 Processor -- start 2024-04-02T14:43:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fcd0b1b3740 1 -- start start 2024-04-02T14:43:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e7b10 0x56118c8e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8e8500 0x56118c8ed000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8ed670 0x56118c8efb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56118c84d060 con 0x56118c8ed670 2024-04-02T14:43:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56118c7f5560 con 0x56118c8e8500 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56118c849010 con 0x56118c8e7b10 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fccfe000640 1 -- 172.21.15.62:0/671059300 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fccf0019b20 con 0x7fcd00108020 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fcd06400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e7b10 0x56118c8e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fcd06400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e7b10 0x56118c8e7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:36234/0 (socket says 172.21.15.62:36234) 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fcd06400640 1 -- 172.21.15.62:0/1352145353 learned_addr learned my addr 172.21.15.62:0/1352145353 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.758+0000 7fcd05000640 1 --2- 172.21.15.62:0/1352145353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8ed670 0x56118c8efb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fcd05a00640 1 --2- 172.21.15.62:0/1352145353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8e8500 0x56118c8ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fcd06400640 1 -- 172.21.15.62:0/1352145353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8e8500 msgr2=0x56118c8ed000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fcd06400640 1 --2- 172.21.15.62:0/1352145353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8e8500 0x56118c8ed000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fcd06400640 1 -- 172.21.15.62:0/1352145353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8ed670 msgr2=0x56118c8efb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fcd06400640 1 --2- 172.21.15.62:0/1352145353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8ed670 0x56118c8efb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fcd06400640 1 -- 172.21.15.62:0/1352145353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118c817fd0 con 0x56118c8e7b10 2024-04-02T14:43:14.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fcd05a00640 1 --2- 172.21.15.62:0/1352145353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8e8500 0x56118c8ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:43:14.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fcd05000640 1 --2- 172.21.15.62:0/1352145353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8ed670 0x56118c8efb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:14.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fcd06400640 1 --2- 172.21.15.62:0/1352145353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e7b10 0x56118c8e7f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009ca0 tx=0x7fccf8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7053edc16aee79e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fccf6a00640 1 -- 172.21.15.62:0/1352145353 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf801e070 con 0x56118c8e7b10 2024-04-02T14:43:14.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fccf6a00640 1 -- 172.21.15.62:0/1352145353 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fccf8009e50 con 0x56118c8e7b10 2024-04-02T14:43:14.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.759+0000 7fccf6a00640 1 -- 172.21.15.62:0/1352145353 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf800ebd0 con 0x56118c8e7b10 2024-04-02T14:43:14.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.760+0000 7fccfe000640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcce40a3bb0 0x7fcce40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.760+0000 7fccfe000640 1 -- 172.21.15.62:0/671059300 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fccf00d0cc0 con 0x7fcd00108020 2024-04-02T14:43:14.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.760+0000 7fcd05a00640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcce40a3bb0 0x7fcce40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.760+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1352145353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e7b10 msgr2=0x56118c8e7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.760+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1352145353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e7b10 0x56118c8e7f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009ca0 tx=0x7fccf8011810 comp rx=0 tx=0).stop 2024-04-02T14:43:14.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.760+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1352145353 shutdown_connections 2024-04-02T14:43:14.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.760+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1352145353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8ed670 0x56118c8efb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.760+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1352145353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8e8500 0x56118c8ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.760+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1352145353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e7b10 0x56118c8e7f30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.760+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1352145353 >> 172.21.15.62:0/1352145353 conn(0x56118c8e3080 msgr2=0x56118c8e3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.761+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1352145353 shutdown_connections 2024-04-02T14:43:14.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.761+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1352145353 wait complete. 2024-04-02T14:43:14.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.761+0000 7fcd05a00640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcce40a3bb0 0x7fcce40a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009270 tx=0x7fccf000cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.761+0000 7fcd0b1b3740 1 -- start start 2024-04-02T14:43:14.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.761+0000 7fcd0b1b3740 1 Processor -- start 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8e8500 0x56118c8f2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8ed670 0x56118c8f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56118c8f0620 con 0x56118c8e7b10 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56118c808920 con 0x56118c8e8500 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56118c8f07a0 con 0x56118c8ed670 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd0b1b3740 10 client.0 _ll_register_callbacks cb 0x56118c808310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd06400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd06400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41334/0 (socket says 172.21.15.62:41334) 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd06400640 1 -- 172.21.15.62:0/1158484197 learned_addr learned my addr 172.21.15.62:0/1158484197 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd05000640 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8e8500 0x56118c8f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.762+0000 7fcd06400640 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8ed670 0x56118c8f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.763+0000 7fcd06400640 1 -- 172.21.15.62:0/1158484197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8ed670 msgr2=0x56118c8f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.763+0000 7fcd06400640 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8ed670 0x56118c8f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.763+0000 7fcd06400640 1 -- 172.21.15.62:0/1158484197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8e8500 msgr2=0x56118c8f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.763+0000 7fcd06400640 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8e8500 0x56118c8f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.763+0000 7fcd06400640 1 -- 172.21.15.62:0/1158484197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118c5612d0 con 0x56118c8e7b10 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.763+0000 7fcd06400640 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f2030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fccf8011d10 tx=0x7fccf8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553b090dd8b42612 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.763+0000 7fcceb400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.763+0000 7fcceb400640 1 -- 172.21.15.62:0/1158484197 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf802c060 con 0x56118c8e7b10 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.763+0000 7fcceb400640 1 -- 172.21.15.62:0/1158484197 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fccf800ed40 con 0x56118c8e7b10 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.763+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1158484197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccf8009e50 con 0x56118c8e7b10 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.763+0000 7fcceb400640 1 -- 172.21.15.62:0/1158484197 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fccf801e040 con 0x56118c8e7b10 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.764+0000 7fcd0b1b3740 10 client.14913 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.764+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1158484197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56118c62b6b0 con 0x56118c8e7b10 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.764+0000 7fcceb400640 1 -- 172.21.15.62:0/1158484197 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fccf8009e50 con 0x56118c8e7b10 2024-04-02T14:43:14.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.764+0000 7fcceb400640 1 -- 172.21.15.62:0/1158484197 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fccf80336b0 con 0x56118c8e7b10 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.764+0000 7fcd0b1b3740 10 client.14913 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.764+0000 7fcd0b1b3740 10 client.14913 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.764+0000 7fcd0b1b3740 20 client.14913 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.764+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56118c8fe9e0 0x56118c8ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.764+0000 7fcd0b1b3740 4 client.14913 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.764+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1158484197 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56118c85a5a0 con 0x56118c8fe9e0 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.765+0000 7fcd05000640 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56118c8fe9e0 0x56118c8ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.765+0000 7fcd05000640 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56118c8fe9e0 0x56118c8ffe50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c8cacd9bd1ff124 server_cookie=740baca184b84a76 in_seq=0 out_seq=0 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.765+0000 7fcceb400640 10 client.14913 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcceb400640 1 -- 172.21.15.62:0/1158484197 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56118c85a5a0 con 0x56118c8fe9e0 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcceb400640 10 client.14913 handle_command_reply: tid=0 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcd0b1b3740 1 client.14913 shutdown 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1158484197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 msgr2=0x56118c8f2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f2030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fccf8011d10 tx=0x7fccf8005850 comp rx=0 tx=0).stop 2024-04-02T14:43:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1158484197 shutdown_connections 2024-04-02T14:43:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56118c8fe9e0 0x56118c8ffe50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8ed670 0x56118c8f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8e8500 0x56118c8f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1158484197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f2030 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcceb400640 0 client.14913 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:43:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.766+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1158484197 >> 172.21.15.62:0/1158484197 conn(0x56118c8e3080 msgr2=0x56118c8e21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.767+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1158484197 shutdown_connections 2024-04-02T14:43:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.767+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1158484197 wait complete. 2024-04-02T14:43:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.767+0000 7fcd0b1b3740 20 client.14913 trim_cache size 0 max 16384 2024-04-02T14:43:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd0b1b3740 1 Processor -- start 2024-04-02T14:43:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd0b1b3740 1 -- start start 2024-04-02T14:43:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8e3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e8500 0x56118c8f9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8ed670 0x56118c8f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56118c8fe190 con 0x56118c8e7b10 2024-04-02T14:43:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56118c8fe860 con 0x56118c8ed670 2024-04-02T14:43:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56118c8fef30 con 0x56118c8e8500 2024-04-02T14:43:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd06400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd06400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8e3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41346/0 (socket says 172.21.15.62:41346) 2024-04-02T14:43:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd06400640 1 -- 172.21.15.62:0/1056507318 learned_addr learned my addr 172.21.15.62:0/1056507318 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.787+0000 7fcd05000640 1 --2- 172.21.15.62:0/1056507318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e8500 0x56118c8f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.788+0000 7fcd06400640 1 --2- 172.21.15.62:0/1056507318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8ed670 0x56118c8f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.788+0000 7fcd06400640 1 -- 172.21.15.62:0/1056507318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e8500 msgr2=0x56118c8f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.788+0000 7fcd06400640 1 --2- 172.21.15.62:0/1056507318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e8500 0x56118c8f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.788+0000 7fcd06400640 1 -- 172.21.15.62:0/1056507318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8ed670 msgr2=0x56118c8f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.788+0000 7fcd06400640 1 --2- 172.21.15.62:0/1056507318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8ed670 0x56118c8f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.788+0000 7fcd06400640 1 -- 172.21.15.62:0/1056507318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118c8f3020 con 0x56118c8e7b10 2024-04-02T14:43:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.788+0000 7fcd06400640 1 --2- 172.21.15.62:0/1056507318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8e3460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009ca0 tx=0x7fccf8005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd012bb4dbcc280 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.788+0000 7fccf6000640 1 -- 172.21.15.62:0/1056507318 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf802c070 con 0x56118c8e7b10 2024-04-02T14:43:14.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.788+0000 7fccf6000640 1 -- 172.21.15.62:0/1056507318 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fccf8009780 con 0x56118c8e7b10 2024-04-02T14:43:14.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.788+0000 7fccf6000640 1 -- 172.21.15.62:0/1056507318 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8033880 con 0x56118c8e7b10 2024-04-02T14:43:14.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.789+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1056507318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 msgr2=0x56118c8e3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.789+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1056507318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8e3460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009ca0 tx=0x7fccf8005e00 comp rx=0 tx=0).stop 2024-04-02T14:43:14.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.789+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1056507318 shutdown_connections 2024-04-02T14:43:14.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.789+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1056507318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8ed670 0x56118c8f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.789+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1056507318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e8500 0x56118c8f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.789+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/1056507318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8e3460 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.789+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1056507318 >> 172.21.15.62:0/1056507318 conn(0x56118c8e3080 msgr2=0x56118c8ddde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.789+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1056507318 shutdown_connections 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.789+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/1056507318 wait complete. 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.789+0000 7fcd0b1b3740 1 -- start start 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.790+0000 7fcd0b1b3740 1 Processor -- start 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.790+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.790+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e8500 0x56118c8f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.790+0000 7fcd0b1b3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8ed670 0x56118c8f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.790+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56118c8ffcd0 con 0x56118c8e7b10 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.790+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56118c8ff600 con 0x56118c8ed670 2024-04-02T14:43:14.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.790+0000 7fcd0b1b3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56118c9003a0 con 0x56118c8e8500 2024-04-02T14:43:14.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.790+0000 7fcd0b1b3740 10 client.0 _ll_register_callbacks cb 0x7fccf8009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:14.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.790+0000 7fcd06400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.790+0000 7fcd06400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41350/0 (socket says 172.21.15.62:41350) 2024-04-02T14:43:14.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.790+0000 7fcd06400640 1 -- 172.21.15.62:0/3119786894 learned_addr learned my addr 172.21.15.62:0/3119786894 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:14.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.791+0000 7fcd05000640 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e8500 0x56118c8f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.791+0000 7fcd06400640 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8ed670 0x56118c8f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.791+0000 7fcd06400640 1 -- 172.21.15.62:0/3119786894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e8500 msgr2=0x56118c8f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.791+0000 7fcd06400640 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e8500 0x56118c8f57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.791+0000 7fcd06400640 1 -- 172.21.15.62:0/3119786894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8ed670 msgr2=0x56118c8f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:14.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.791+0000 7fcd06400640 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8ed670 0x56118c8f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.791+0000 7fcd06400640 1 -- 172.21.15.62:0/3119786894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118c8f8db0 con 0x56118c8e7b10 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.791+0000 7fcd05000640 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e8500 0x56118c8f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.791+0000 7fcd06400640 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f5260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fccf8005520 tx=0x7fccf8027470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a0a19a95222183 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.791+0000 7fcceb400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.791+0000 7fcceb400640 1 -- 172.21.15.62:0/3119786894 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf802c070 con 0x56118c8e7b10 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcceb400640 1 -- 172.21.15.62:0/3119786894 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fccf800eb20 con 0x56118c8e7b10 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/3119786894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccf8009780 con 0x56118c8e7b10 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcceb400640 1 -- 172.21.15.62:0/3119786894 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fccf800ed40 con 0x56118c8e7b10 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcd0b1b3740 10 client.14919 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/3119786894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56118c964ea0 con 0x56118c8e7b10 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcceb400640 1 -- 172.21.15.62:0/3119786894 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fccf8009780 con 0x56118c8e7b10 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcceb400640 1 -- 172.21.15.62:0/3119786894 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fccf8019550 con 0x56118c8e7b10 2024-04-02T14:43:14.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcd0b1b3740 10 client.14919 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcd0b1b3740 10 client.14919 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcd0b1b3740 20 client.14919 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56118c969610 0x56118c968320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcd0b1b3740 4 client.14919 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.792+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/3119786894 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "status"}) -- 0x56118c8548f0 con 0x56118c969610 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.793+0000 7fcd05000640 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56118c969610 0x56118c968320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.793+0000 7fcceb400640 10 client.14919 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.793+0000 7fcd05000640 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56118c969610 0x56118c968320 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76bca47c5ab50387 server_cookie=36cecde32b23a9c8 in_seq=0 out_seq=0 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.793+0000 7fcceb400640 1 -- 172.21.15.62:0/3119786894 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56118c8548f0 con 0x56118c969610 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.793+0000 7fcceb400640 10 client.14919 handle_command_reply: tid=0 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.793+0000 7fcd0b1b3740 1 client.14919 shutdown 2024-04-02T14:43:14.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/3119786894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 msgr2=0x56118c8f5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f5260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fccf8005520 tx=0x7fccf8027470 comp rx=0 tx=0).stop 2024-04-02T14:43:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/3119786894 shutdown_connections 2024-04-02T14:43:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56118c969610 0x56118c968320 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56118c8ed670 0x56118c8f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118c8e8500 0x56118c8f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcd0b1b3740 1 --2- 172.21.15.62:0/3119786894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56118c8e7b10 0x56118c8f5260 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcceb400640 0 client.14919 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:43:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/3119786894 >> 172.21.15.62:0/3119786894 conn(0x56118c8e3080 msgr2=0x56118c8e3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/3119786894 shutdown_connections 2024-04-02T14:43:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcd0b1b3740 1 -- 172.21.15.62:0/3119786894 wait complete. 2024-04-02T14:43:14.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.794+0000 7fcd0b1b3740 20 client.14919 trim_cache size 0 max 16384 2024-04-02T14:43:14.799 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:43:14.799 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:43:14.799 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-04-02T14:43:14.799 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24553, 2024-04-02T14:43:14.799 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-02T14:43:14.800 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-02T14:43:14.800 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-02T14:43:14.800 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 107.74697717700001, 2024-04-02T14:43:14.800 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-04-02T14:43:14.800 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-04-02T14:43:14.800 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-04-02T14:43:14.800 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 126.111464707 2024-04-02T14:43:14.800 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:43:14.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 -- 172.21.15.62:0/671059300 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcce40a3bb0 msgr2=0x7fcce40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcce40a3bb0 0x7fcce40a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009270 tx=0x7fccf000cf90 comp rx=0 tx=0).stop 2024-04-02T14:43:14.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 -- 172.21.15.62:0/671059300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00108020 msgr2=0x7fcd0019e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:14.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00108020 0x7fcd0019e380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fccf000ebc0 tx=0x7fccf000c6a0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 -- 172.21.15.62:0/671059300 shutdown_connections 2024-04-02T14:43:14.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fcce40a3bb0 0x7fcce40a6070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd00109bd0 0x7fcd001a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd00109210 0x7fcd0019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 --2- 172.21.15.62:0/671059300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00108020 0x7fcd0019e380 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:14.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 -- 172.21.15.62:0/671059300 >> 172.21.15.62:0/671059300 conn(0x7fcd00076fd0 msgr2=0x7fcd0010c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:14.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 -- 172.21.15.62:0/671059300 shutdown_connections 2024-04-02T14:43:14.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:14.795+0000 7fcd06e00640 1 -- 172.21.15.62:0/671059300 wait complete. 2024-04-02T14:43:14.812 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-02T14:43:15.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.036+0000 7f90d7200640 1 -- 172.21.15.62:0/55884463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90d8104df0 msgr2=0x7f90d8105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.036+0000 7f90d7200640 1 --2- 172.21.15.62:0/55884463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90d8104df0 0x7f90d8105270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009900 tx=0x7f90c002bc50 comp rx=0 tx=0).stop 2024-04-02T14:43:15.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.036+0000 7f90d7200640 1 -- 172.21.15.62:0/55884463 shutdown_connections 2024-04-02T14:43:15.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.036+0000 7f90d7200640 1 --2- 172.21.15.62:0/55884463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d81057b0 0x7f90d810c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.036+0000 7f90d7200640 1 --2- 172.21.15.62:0/55884463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90d8104df0 0x7f90d8105270 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.036+0000 7f90d7200640 1 --2- 172.21.15.62:0/55884463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d8103c00 0x7f90d8104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.036+0000 7f90d7200640 1 -- 172.21.15.62:0/55884463 >> 172.21.15.62:0/55884463 conn(0x7f90d80ff170 msgr2=0x7f90d81015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.036+0000 7f90d7200640 1 -- 172.21.15.62:0/55884463 shutdown_connections 2024-04-02T14:43:15.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.036+0000 7f90d7200640 1 -- 172.21.15.62:0/55884463 wait complete. 2024-04-02T14:43:15.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.037+0000 7f90d7200640 1 Processor -- start 2024-04-02T14:43:15.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.037+0000 7f90d7200640 1 -- start start 2024-04-02T14:43:15.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90d7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d8103c00 0x7f90d8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90d7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d8104df0 0x7f90d819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90d7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90d81057b0 0x7f90d81a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90d7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90d810f9b0 con 0x7f90d8104df0 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90d7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f90d810f830 con 0x7f90d8103c00 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90d7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f90d810fb30 con 0x7f90d81057b0 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90cd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d8104df0 0x7f90d819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90cd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d8104df0 0x7f90d819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41370/0 (socket says 172.21.15.62:41370) 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90cd400640 1 -- 172.21.15.62:0/342740048 learned_addr learned my addr 172.21.15.62:0/342740048 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90d5e00640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d8103c00 0x7f90d8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90d6800640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90d81057b0 0x7f90d81a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90cd400640 1 -- 172.21.15.62:0/342740048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90d81057b0 msgr2=0x7f90d81a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90cd400640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90d81057b0 0x7f90d81a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.038+0000 7f90cd400640 1 -- 172.21.15.62:0/342740048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d8103c00 msgr2=0x7f90d8199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.039+0000 7f90cd400640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d8103c00 0x7f90d8199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.039+0000 7f90cd400640 1 -- 172.21.15.62:0/342740048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d81a1bc0 con 0x7f90d8104df0 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.039+0000 7f90cd400640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d8104df0 0x7f90d819a4e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009230 tx=0x7f90c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.039+0000 7f90cf400640 1 -- 172.21.15.62:0/342740048 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0043020 con 0x7f90d8104df0 2024-04-02T14:43:15.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.039+0000 7f90cf400640 1 -- 172.21.15.62:0/342740048 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f90c0004250 con 0x7f90d8104df0 2024-04-02T14:43:15.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.039+0000 7f90d7200640 1 -- 172.21.15.62:0/342740048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d81a1e50 con 0x7f90d8104df0 2024-04-02T14:43:15.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.039+0000 7f90cf400640 1 -- 172.21.15.62:0/342740048 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0003b90 con 0x7f90d8104df0 2024-04-02T14:43:15.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.040+0000 7f90d7200640 1 -- 172.21.15.62:0/342740048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d81a2310 con 0x7f90d8104df0 2024-04-02T14:43:15.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.043+0000 7f90df7d8740 1 Processor -- start 2024-04-02T14:43:15.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.043+0000 7f90df7d8740 1 -- start start 2024-04-02T14:43:15.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.043+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x559953573f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.043+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x559953579000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.043+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 0x55995357bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.043+0000 7f90df7d8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5599534d9060 con 0x559953573b10 2024-04-02T14:43:15.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.043+0000 7f90df7d8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559953481560 con 0x559953574500 2024-04-02T14:43:15.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.043+0000 7f90df7d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5599534d5010 con 0x559953579670 2024-04-02T14:43:15.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.044+0000 7f90d6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x559953573f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.044+0000 7f90d6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x559953573f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41376/0 (socket says 172.21.15.62:41376) 2024-04-02T14:43:15.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.044+0000 7f90d6800640 1 -- 172.21.15.62:0/887763788 learned_addr learned my addr 172.21.15.62:0/887763788 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:15.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.044+0000 7f90d5e00640 1 --2- 172.21.15.62:0/887763788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x559953579000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.044+0000 7f90cf400640 1 -- 172.21.15.62:0/342740048 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f90c0046020 con 0x7f90d8104df0 2024-04-02T14:43:15.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.044+0000 7f90d6800640 1 -- 172.21.15.62:0/887763788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 msgr2=0x55995357bb00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T14:43:15.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.044+0000 7f90d6800640 1 --2- 172.21.15.62:0/887763788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 0x55995357bb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.044+0000 7f90d6800640 1 -- 172.21.15.62:0/887763788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 msgr2=0x559953579000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.044+0000 7f90d6800640 1 --2- 172.21.15.62:0/887763788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x559953579000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.044+0000 7f90d6800640 1 -- 172.21.15.62:0/887763788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599534a3fd0 con 0x559953573b10 2024-04-02T14:43:15.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.044+0000 7f90d6800640 1 --2- 172.21.15.62:0/887763788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x559953573f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009f50 tx=0x7f90d00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5391cdf6448c0913 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.045+0000 7f90c7000640 1 -- 172.21.15.62:0/887763788 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0004d20 con 0x559953573b10 2024-04-02T14:43:15.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.045+0000 7f90c7000640 1 -- 172.21.15.62:0/887763788 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f90d0004050 con 0x559953573b10 2024-04-02T14:43:15.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.045+0000 7f90c7000640 1 -- 172.21.15.62:0/887763788 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0005260 con 0x559953573b10 2024-04-02T14:43:15.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.045+0000 7f90cf400640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f90b80a3bb0 0x7f90b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.045+0000 7f90df7d8740 1 -- 172.21.15.62:0/887763788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 msgr2=0x559953573f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.045+0000 7f90df7d8740 1 --2- 172.21.15.62:0/887763788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x559953573f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009f50 tx=0x7f90d00118d0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.046+0000 7f90cf400640 1 -- 172.21.15.62:0/342740048 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f90c00f2540 con 0x7f90d8104df0 2024-04-02T14:43:15.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.046+0000 7f90df7d8740 1 -- 172.21.15.62:0/887763788 shutdown_connections 2024-04-02T14:43:15.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.046+0000 7f90df7d8740 1 --2- 172.21.15.62:0/887763788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 0x55995357bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.046+0000 7f90df7d8740 1 --2- 172.21.15.62:0/887763788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x559953579000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.046+0000 7f90df7d8740 1 --2- 172.21.15.62:0/887763788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x559953573f30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.046+0000 7f90df7d8740 1 -- 172.21.15.62:0/887763788 >> 172.21.15.62:0/887763788 conn(0x55995356f080 msgr2=0x55995356f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.046+0000 7f90d5e00640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f90b80a3bb0 0x7f90b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.046+0000 7f90df7d8740 1 -- 172.21.15.62:0/887763788 shutdown_connections 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.046+0000 7f90df7d8740 1 -- 172.21.15.62:0/887763788 wait complete. 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.046+0000 7f90df7d8740 1 -- start start 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.046+0000 7f90d5e00640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f90b80a3bb0 0x7f90b80a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55995356fff0 tx=0x7f90c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.047+0000 7f90df7d8740 1 Processor -- start 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.047+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953573b10 0x55995357e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.047+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953574500 0x55995357e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.047+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 0x559953585610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.047+0000 7f90df7d8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55995357c620 con 0x559953574500 2024-04-02T14:43:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.047+0000 7f90df7d8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559953494920 con 0x559953573b10 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.047+0000 7f90df7d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55995357c7a0 con 0x559953579670 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.047+0000 7f90df7d8740 10 client.0 _ll_register_callbacks cb 0x559953494310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.047+0000 7f90cd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953574500 0x55995357e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.047+0000 7f90cd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953574500 0x55995357e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41386/0 (socket says 172.21.15.62:41386) 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.047+0000 7f90cd400640 1 -- 172.21.15.62:0/2667075886 learned_addr learned my addr 172.21.15.62:0/2667075886 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90d5e00640 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953573b10 0x55995357e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90d6800640 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 0x559953585610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90cd400640 1 -- 172.21.15.62:0/2667075886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 msgr2=0x559953585610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90cd400640 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 0x559953585610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90cd400640 1 -- 172.21.15.62:0/2667075886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953573b10 msgr2=0x55995357e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90cd400640 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953573b10 0x55995357e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90cd400640 1 -- 172.21.15.62:0/2667075886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599531ed2d0 con 0x559953574500 2024-04-02T14:43:15.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90d5e00640 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953573b10 0x55995357e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90cd400640 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953574500 0x55995357e5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009a20 tx=0x7f90c00be410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb251723f3ca7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90bf400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90bf400640 1 -- 172.21.15.62:0/2667075886 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00c2c00 con 0x559953574500 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90bf400640 1 -- 172.21.15.62:0/2667075886 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f90c0003d30 con 0x559953574500 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.048+0000 7f90df7d8740 1 -- 172.21.15.62:0/2667075886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d0004050 con 0x559953574500 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.049+0000 7f90bf400640 1 -- 172.21.15.62:0/2667075886 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f90c00c74c0 con 0x559953574500 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.049+0000 7f90df7d8740 10 client.14931 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.049+0000 7f90df7d8740 1 -- 172.21.15.62:0/2667075886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599532b76b0 con 0x559953574500 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.049+0000 7f90bf400640 1 -- 172.21.15.62:0/2667075886 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f90d0004050 con 0x559953574500 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.049+0000 7f90bf400640 1 -- 172.21.15.62:0/2667075886 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f90c00d4060 con 0x559953574500 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.049+0000 7f90df7d8740 10 client.14931 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.049+0000 7f90df7d8740 10 client.14931 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.049+0000 7f90df7d8740 20 client.14931 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:15.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.049+0000 7f90df7d8740 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55995358ae20 0x55995358c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.049+0000 7f90df7d8740 4 client.14931 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:43:15.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.049+0000 7f90df7d8740 1 -- 172.21.15.62:0/2667075886 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599534e65a0 con 0x55995358ae20 2024-04-02T14:43:15.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.050+0000 7f90d5e00640 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55995358ae20 0x55995358c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.050+0000 7f90d5e00640 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55995358ae20 0x55995358c290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a861812a916c67b server_cookie=fec412627a198f6f in_seq=0 out_seq=0 2024-04-02T14:43:15.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.050+0000 7f90bf400640 10 client.14931 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:43:15.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.051+0000 7f90bf400640 1 -- 172.21.15.62:0/2667075886 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599534e65a0 con 0x55995358ae20 2024-04-02T14:43:15.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.051+0000 7f90bf400640 10 client.14931 handle_command_reply: tid=0 2024-04-02T14:43:15.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.051+0000 7f90df7d8740 1 client.14931 shutdown 2024-04-02T14:43:15.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90df7d8740 1 -- 172.21.15.62:0/2667075886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953574500 msgr2=0x55995357e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90df7d8740 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953574500 0x55995357e5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009a20 tx=0x7f90c00be410 comp rx=0 tx=0).stop 2024-04-02T14:43:15.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90df7d8740 1 -- 172.21.15.62:0/2667075886 shutdown_connections 2024-04-02T14:43:15.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90df7d8740 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55995358ae20 0x55995358c290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90df7d8740 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 0x559953585610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90df7d8740 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953574500 0x55995357e5d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90df7d8740 1 --2- 172.21.15.62:0/2667075886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953573b10 0x55995357e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90bf400640 0 client.14931 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:43:15.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90df7d8740 1 -- 172.21.15.62:0/2667075886 >> 172.21.15.62:0/2667075886 conn(0x55995356f080 msgr2=0x55995356e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90df7d8740 1 -- 172.21.15.62:0/2667075886 shutdown_connections 2024-04-02T14:43:15.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90df7d8740 1 -- 172.21.15.62:0/2667075886 wait complete. 2024-04-02T14:43:15.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.052+0000 7f90df7d8740 20 client.14931 trim_cache size 0 max 16384 2024-04-02T14:43:15.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.072+0000 7f90df7d8740 1 Processor -- start 2024-04-02T14:43:15.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90df7d8740 1 -- start start 2024-04-02T14:43:15.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x55995356f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x5599535855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 0x559953569de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90df7d8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55995358a5d0 con 0x559953573b10 2024-04-02T14:43:15.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90df7d8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55995358aca0 con 0x559953574500 2024-04-02T14:43:15.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90df7d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55995358b370 con 0x559953579670 2024-04-02T14:43:15.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90d5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x55995356f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90d5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x55995356f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41390/0 (socket says 172.21.15.62:41390) 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90d5e00640 1 -- 172.21.15.62:0/1634933015 learned_addr learned my addr 172.21.15.62:0/1634933015 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90cd400640 1 --2- 172.21.15.62:0/1634933015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x5599535855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.073+0000 7f90d6800640 1 --2- 172.21.15.62:0/1634933015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 0x559953569de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.074+0000 7f90cd400640 1 -- 172.21.15.62:0/1634933015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 msgr2=0x559953569de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.074+0000 7f90cd400640 1 --2- 172.21.15.62:0/1634933015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 0x559953569de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.074+0000 7f90cd400640 1 -- 172.21.15.62:0/1634933015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 msgr2=0x55995356f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.074+0000 7f90cd400640 1 --2- 172.21.15.62:0/1634933015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x55995356f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.074+0000 7f90cd400640 1 -- 172.21.15.62:0/1634933015 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55995356d6c0 con 0x559953574500 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.074+0000 7f90d5e00640 1 --2- 172.21.15.62:0/1634933015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x55995356f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.074+0000 7f90cd400640 1 --2- 172.21.15.62:0/1634933015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x5599535855d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f90c0004650 tx=0x7f90c002bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c03286ef69b3e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.074+0000 7f90c6600640 1 -- 172.21.15.62:0/1634933015 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00baaf0 con 0x559953574500 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.074+0000 7f90c6600640 1 -- 172.21.15.62:0/1634933015 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f90c00c2c00 con 0x559953574500 2024-04-02T14:43:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.074+0000 7f90c6600640 1 -- 172.21.15.62:0/1634933015 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00a90b0 con 0x559953574500 2024-04-02T14:43:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.075+0000 7f90df7d8740 1 -- 172.21.15.62:0/1634933015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 msgr2=0x5599535855d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.075+0000 7f90df7d8740 1 --2- 172.21.15.62:0/1634933015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x5599535855d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f90c0004650 tx=0x7f90c002bc50 comp rx=0 tx=0).stop 2024-04-02T14:43:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.075+0000 7f90df7d8740 1 -- 172.21.15.62:0/1634933015 shutdown_connections 2024-04-02T14:43:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.075+0000 7f90df7d8740 1 --2- 172.21.15.62:0/1634933015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953579670 0x559953569de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.075+0000 7f90df7d8740 1 --2- 172.21.15.62:0/1634933015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x5599535855d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.075+0000 7f90df7d8740 1 --2- 172.21.15.62:0/1634933015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953573b10 0x55995356f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.075+0000 7f90df7d8740 1 -- 172.21.15.62:0/1634933015 >> 172.21.15.62:0/1634933015 conn(0x55995356f080 msgr2=0x55995357cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.075+0000 7f90df7d8740 1 -- 172.21.15.62:0/1634933015 shutdown_connections 2024-04-02T14:43:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.075+0000 7f90df7d8740 1 -- 172.21.15.62:0/1634933015 wait complete. 2024-04-02T14:43:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.076+0000 7f90df7d8740 1 -- start start 2024-04-02T14:43:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.076+0000 7f90df7d8740 1 Processor -- start 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.076+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953573b10 0x559953577ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.076+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x559953578020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.076+0000 7f90df7d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953579670 0x559953583bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.076+0000 7f90df7d8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55995358c110 con 0x559953579670 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.076+0000 7f90df7d8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55995358ba40 con 0x559953574500 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.076+0000 7f90df7d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55995358c7e0 con 0x559953573b10 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.076+0000 7f90df7d8740 10 client.0 _ll_register_callbacks cb 0x7f90c00be620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.076+0000 7f90d6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953579670 0x559953583bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.076+0000 7f90cd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x559953578020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90cd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x559953578020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:43856/0 (socket says 172.21.15.62:43856) 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90cd400640 1 -- 172.21.15.62:0/70308957 learned_addr learned my addr 172.21.15.62:0/70308957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90d5e00640 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953573b10 0x559953577ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90d6800640 1 -- 172.21.15.62:0/70308957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953573b10 msgr2=0x559953577ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90d6800640 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953573b10 0x559953577ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90d6800640 1 -- 172.21.15.62:0/70308957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 msgr2=0x559953578020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90d6800640 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x559953578020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90d6800640 1 -- 172.21.15.62:0/70308957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599535841b0 con 0x559953579670 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90cd400640 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x559953578020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90d5e00640 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953573b10 0x559953577ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90d6800640 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953579670 0x559953583bc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f90d0012fd0 tx=0x7f90d0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba50bded2cbecd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.077+0000 7f90bf400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.078+0000 7f90bf400640 1 -- 172.21.15.62:0/70308957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0009be0 con 0x559953579670 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.078+0000 7f90bf400640 1 -- 172.21.15.62:0/70308957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f90d0004050 con 0x559953579670 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.078+0000 7f90df7d8740 1 -- 172.21.15.62:0/70308957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c00c2c00 con 0x559953579670 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.078+0000 7f90bf400640 1 -- 172.21.15.62:0/70308957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f90d0034070 con 0x559953579670 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.078+0000 7f90df7d8740 10 client.14937 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.078+0000 7f90df7d8740 1 -- 172.21.15.62:0/70308957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599535854b0 con 0x559953579670 2024-04-02T14:43:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.078+0000 7f90bf400640 1 -- 172.21.15.62:0/70308957 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f90c00c2c00 con 0x559953579670 2024-04-02T14:43:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.078+0000 7f90bf400640 1 -- 172.21.15.62:0/70308957 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f90d0005640 con 0x559953579670 2024-04-02T14:43:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.079+0000 7f90df7d8740 10 client.14937 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.079+0000 7f90df7d8740 10 client.14937 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-02T14:43:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.079+0000 7f90df7d8740 20 client.14937 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.079+0000 7f90df7d8740 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5599535f5160 0x5599535f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.079+0000 7f90df7d8740 4 client.14937 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T14:43:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.079+0000 7f90df7d8740 1 -- 172.21.15.62:0/70308957 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "status"}) -- 0x5599534e08f0 con 0x5599535f5160 2024-04-02T14:43:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.079+0000 7f90d5e00640 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5599535f5160 0x5599535f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.079+0000 7f90d5e00640 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5599535f5160 0x5599535f3e70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=daef2671a9d0e404 server_cookie=196a1c502529bc7e in_seq=0 out_seq=0 2024-04-02T14:43:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.079+0000 7f90bf400640 10 client.14937 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:43:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90bf400640 1 -- 172.21.15.62:0/70308957 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x5599534e08f0 con 0x5599535f5160 2024-04-02T14:43:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90bf400640 10 client.14937 handle_command_reply: tid=0 2024-04-02T14:43:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 1 client.14937 shutdown 2024-04-02T14:43:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 1 -- 172.21.15.62:0/70308957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953579670 msgr2=0x559953583bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953579670 0x559953583bc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f90d0012fd0 tx=0x7f90d0004ac0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 1 -- 172.21.15.62:0/70308957 shutdown_connections 2024-04-02T14:43:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5599535f5160 0x5599535f3e70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559953579670 0x559953583bc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559953574500 0x559953578020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 1 --2- 172.21.15.62:0/70308957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559953573b10 0x559953577ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90bf400640 0 client.14937 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:43:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 1 -- 172.21.15.62:0/70308957 >> 172.21.15.62:0/70308957 conn(0x55995356f080 msgr2=0x559953569de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 1 -- 172.21.15.62:0/70308957 shutdown_connections 2024-04-02T14:43:15.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 1 -- 172.21.15.62:0/70308957 wait complete. 2024-04-02T14:43:15.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.080+0000 7f90df7d8740 20 client.14937 trim_cache size 0 max 16384 2024-04-02T14:43:15.083 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:43:15.083 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:43:15.083 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-04-02T14:43:15.083 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24565, 2024-04-02T14:43:15.083 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-02T14:43:15.083 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-02T14:43:15.083 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-02T14:43:15.084 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 109.046033403, 2024-04-02T14:43:15.084 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-04-02T14:43:15.084 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-04-02T14:43:15.084 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-04-02T14:43:15.084 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 122.3140518 2024-04-02T14:43:15.084 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:43:15.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 -- 172.21.15.62:0/342740048 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f90b80a3bb0 msgr2=0x7f90b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f90b80a3bb0 0x7f90b80a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55995356fff0 tx=0x7f90c800b410 comp rx=0 tx=0).stop 2024-04-02T14:43:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 -- 172.21.15.62:0/342740048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d8104df0 msgr2=0x7f90d819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d8104df0 0x7f90d819a4e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009230 tx=0x7f90c0004ab0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 -- 172.21.15.62:0/342740048 shutdown_connections 2024-04-02T14:43:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f90b80a3bb0 0x7f90b80a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90d81057b0 0x7f90d81a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d8104df0 0x7f90d819a4e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 --2- 172.21.15.62:0/342740048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d8103c00 0x7f90d8199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 -- 172.21.15.62:0/342740048 >> 172.21.15.62:0/342740048 conn(0x7f90d80ff170 msgr2=0x7f90d81080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 -- 172.21.15.62:0/342740048 shutdown_connections 2024-04-02T14:43:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.081+0000 7f90d7200640 1 -- 172.21.15.62:0/342740048 wait complete. 2024-04-02T14:43:15.096 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T14:43:15.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.305+0000 7f53af400640 1 -- 172.21.15.62:0/2239771840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53a8103d40 msgr2=0x7f53a8104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.305+0000 7f53af400640 1 --2- 172.21.15.62:0/2239771840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53a8103d40 0x7f53a8104160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5390009900 tx=0x7f539002bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.305+0000 7f53af400640 1 -- 172.21.15.62:0/2239771840 shutdown_connections 2024-04-02T14:43:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.305+0000 7f53af400640 1 --2- 172.21.15.62:0/2239771840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a81058f0 0x7f53a810c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.305+0000 7f53af400640 1 --2- 172.21.15.62:0/2239771840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a8104f30 0x7f53a81053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.305+0000 7f53af400640 1 --2- 172.21.15.62:0/2239771840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53a8103d40 0x7f53a8104160 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.305+0000 7f53af400640 1 -- 172.21.15.62:0/2239771840 >> 172.21.15.62:0/2239771840 conn(0x7f53a80ff2b0 msgr2=0x7f53a81016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.305+0000 7f53af400640 1 -- 172.21.15.62:0/2239771840 shutdown_connections 2024-04-02T14:43:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.305+0000 7f53af400640 1 -- 172.21.15.62:0/2239771840 wait complete. 2024-04-02T14:43:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.306+0000 7f53af400640 1 Processor -- start 2024-04-02T14:43:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.306+0000 7f53af400640 1 -- start start 2024-04-02T14:43:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a8103d40 0x7f53a819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53af400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a8104f30 0x7f53a819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53af400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53a81058f0 0x7f53a81a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53af400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f53a810faa0 con 0x7f53a8103d40 2024-04-02T14:43:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53af400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f53a810f920 con 0x7f53a81058f0 2024-04-02T14:43:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53af400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f53a810fc20 con 0x7f53a8104f30 2024-04-02T14:43:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53aca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a8103d40 0x7f53a819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f539fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a8104f30 0x7f53a819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53aca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a8103d40 0x7f53a819a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41408/0 (socket says 172.21.15.62:41408) 2024-04-02T14:43:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53aca00640 1 -- 172.21.15.62:0/537964387 learned_addr learned my addr 172.21.15.62:0/537964387 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53ad400640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53a81058f0 0x7f53a81a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53aca00640 1 -- 172.21.15.62:0/537964387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a8104f30 msgr2=0x7f53a819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53aca00640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a8104f30 0x7f53a819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53aca00640 1 -- 172.21.15.62:0/537964387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53a81058f0 msgr2=0x7f53a81a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53aca00640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53a81058f0 0x7f53a81a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.307+0000 7f53aca00640 1 -- 172.21.15.62:0/537964387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a81a1d90 con 0x7f53a8103d40 2024-04-02T14:43:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.308+0000 7f53aca00640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a8103d40 0x7f53a819a170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5390000c00 tx=0x7f5390004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.308+0000 7f53a6000640 1 -- 172.21.15.62:0/537964387 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390043020 con 0x7f53a8103d40 2024-04-02T14:43:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.308+0000 7f53a6000640 1 -- 172.21.15.62:0/537964387 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5390004250 con 0x7f53a8103d40 2024-04-02T14:43:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.308+0000 7f53af400640 1 -- 172.21.15.62:0/537964387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a81a2020 con 0x7f53a8103d40 2024-04-02T14:43:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.308+0000 7f53a6000640 1 -- 172.21.15.62:0/537964387 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390003b90 con 0x7f53a8103d40 2024-04-02T14:43:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.308+0000 7f53af400640 1 -- 172.21.15.62:0/537964387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a81a2570 con 0x7f53a8103d40 2024-04-02T14:43:15.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.311+0000 7f53b2013740 1 Processor -- start 2024-04-02T14:43:15.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53b2013740 1 -- start start 2024-04-02T14:43:15.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53a6000640 1 -- 172.21.15.62:0/537964387 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5390046020 con 0x7f53a8103d40 2024-04-02T14:43:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53b2013740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c2498f0 0x55ab7c249d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53b2013740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c24a2e0 0x55ab7c24ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53b2013740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c2518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53b2013740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c1af060 con 0x55ab7c24f450 2024-04-02T14:43:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53b2013740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c157560 con 0x55ab7c2498f0 2024-04-02T14:43:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53b2013740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c1ab010 con 0x55ab7c24a2e0 2024-04-02T14:43:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53ad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c2498f0 0x55ab7c249d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53ad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c2498f0 0x55ab7c249d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:43874/0 (socket says 172.21.15.62:43874) 2024-04-02T14:43:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53ad400640 1 -- 172.21.15.62:0/1524276245 learned_addr learned my addr 172.21.15.62:0/1524276245 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f539fe00640 1 --2- 172.21.15.62:0/1524276245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c2518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.312+0000 7f53aca00640 1 --2- 172.21.15.62:0/1524276245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c24a2e0 0x55ab7c24ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.313+0000 7f53ad400640 1 -- 172.21.15.62:0/1524276245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c24a2e0 msgr2=0x55ab7c24ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.313+0000 7f53ad400640 1 --2- 172.21.15.62:0/1524276245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c24a2e0 0x55ab7c24ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.313+0000 7f53ad400640 1 -- 172.21.15.62:0/1524276245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 msgr2=0x55ab7c2518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.313+0000 7f53ad400640 1 --2- 172.21.15.62:0/1524276245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c2518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.313+0000 7f53ad400640 1 -- 172.21.15.62:0/1524276245 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab7c179fd0 con 0x55ab7c2498f0 2024-04-02T14:43:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.313+0000 7f53ad400640 1 --2- 172.21.15.62:0/1524276245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c2498f0 0x55ab7c249d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009fd0 tx=0x7f53a00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba4820cb41a46998 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.313+0000 7f539dc00640 1 -- 172.21.15.62:0/1524276245 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0004d20 con 0x55ab7c2498f0 2024-04-02T14:43:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.313+0000 7f539dc00640 1 -- 172.21.15.62:0/1524276245 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53a0004050 con 0x55ab7c2498f0 2024-04-02T14:43:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.313+0000 7f539dc00640 1 -- 172.21.15.62:0/1524276245 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0005260 con 0x55ab7c2498f0 2024-04-02T14:43:15.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.313+0000 7f53a6000640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f53880a3bb0 0x7f53880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53a6000640 1 -- 172.21.15.62:0/537964387 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f53900f2420 con 0x7f53a8103d40 2024-04-02T14:43:15.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53aca00640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f53880a3bb0 0x7f53880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53b2013740 1 -- 172.21.15.62:0/1524276245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c2498f0 msgr2=0x55ab7c249d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53b2013740 1 --2- 172.21.15.62:0/1524276245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c2498f0 0x55ab7c249d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009fd0 tx=0x7f53a00118d0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53b2013740 1 -- 172.21.15.62:0/1524276245 shutdown_connections 2024-04-02T14:43:15.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53b2013740 1 --2- 172.21.15.62:0/1524276245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c2518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53b2013740 1 --2- 172.21.15.62:0/1524276245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c24a2e0 0x55ab7c24ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53b2013740 1 --2- 172.21.15.62:0/1524276245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c2498f0 0x55ab7c249d10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53b2013740 1 -- 172.21.15.62:0/1524276245 >> 172.21.15.62:0/1524276245 conn(0x55ab7c244e60 msgr2=0x55ab7c245260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53b2013740 1 -- 172.21.15.62:0/1524276245 shutdown_connections 2024-04-02T14:43:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53b2013740 1 -- 172.21.15.62:0/1524276245 wait complete. 2024-04-02T14:43:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.314+0000 7f53aca00640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f53880a3bb0 0x7f53880a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55ab7c24a6e0 tx=0x7f539002bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.315+0000 7f53b2013740 1 -- start start 2024-04-02T14:43:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.315+0000 7f53b2013740 1 Processor -- start 2024-04-02T14:43:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.315+0000 7f53b2013740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c2498f0 0x55ab7c253e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53b2013740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c2543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53b2013740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c24f450 0x55ab7c25b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53b2013740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c252400 con 0x55ab7c2498f0 2024-04-02T14:43:15.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53b2013740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c16a920 con 0x55ab7c24a2e0 2024-04-02T14:43:15.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53b2013740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c252580 con 0x55ab7c24f450 2024-04-02T14:43:15.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53b2013740 10 client.0 _ll_register_callbacks cb 0x55ab7c16a310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:15.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53ad400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c2498f0 0x55ab7c253e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53ad400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c2498f0 0x55ab7c253e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41424/0 (socket says 172.21.15.62:41424) 2024-04-02T14:43:15.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53ad400640 1 -- 172.21.15.62:0/991826248 learned_addr learned my addr 172.21.15.62:0/991826248 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:15.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f539fe00640 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c2543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53ad400640 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c24f450 0x55ab7c25b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53ad400640 1 -- 172.21.15.62:0/991826248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c24f450 msgr2=0x55ab7c25b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53ad400640 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c24f450 0x55ab7c25b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53ad400640 1 -- 172.21.15.62:0/991826248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 msgr2=0x55ab7c2543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53ad400640 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c2543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.316+0000 7f53ad400640 1 -- 172.21.15.62:0/991826248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab7bec32d0 con 0x55ab7c2498f0 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.317+0000 7f53ad400640 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c2498f0 0x55ab7c253e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f53a0005c10 tx=0x7f53a0004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc0dd95bf24e61e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.317+0000 7f5396000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.317+0000 7f5396000640 1 -- 172.21.15.62:0/991826248 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0004610 con 0x55ab7c2498f0 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.317+0000 7f5396000640 1 -- 172.21.15.62:0/991826248 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f53a000f4e0 con 0x55ab7c2498f0 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.317+0000 7f53b2013740 1 -- 172.21.15.62:0/991826248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53a0004050 con 0x55ab7c2498f0 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.317+0000 7f5396000640 1 -- 172.21.15.62:0/991826248 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f53a0003d40 con 0x55ab7c2498f0 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.317+0000 7f53b2013740 10 client.14949 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.317+0000 7f53b2013740 1 -- 172.21.15.62:0/991826248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab7bf8d6b0 con 0x55ab7c2498f0 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.318+0000 7f5396000640 1 -- 172.21.15.62:0/991826248 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53a0004050 con 0x55ab7c2498f0 2024-04-02T14:43:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.318+0000 7f5396000640 1 -- 172.21.15.62:0/991826248 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f53a0028810 con 0x55ab7c2498f0 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.318+0000 7f53b2013740 10 client.14949 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.318+0000 7f53b2013740 10 client.14949 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.318+0000 7f53b2013740 20 client.14949 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.318+0000 7f53b2013740 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ab7c2609d0 0x55ab7c261e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.318+0000 7f53b2013740 4 client.14949 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.318+0000 7f53b2013740 1 -- 172.21.15.62:0/991826248 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab7c1bc5a0 con 0x55ab7c2609d0 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.318+0000 7f539fe00640 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ab7c2609d0 0x55ab7c261e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.319+0000 7f539fe00640 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ab7c2609d0 0x55ab7c261e40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a67ae367020050e server_cookie=c8d482924c41d2b1 in_seq=0 out_seq=0 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.319+0000 7f5396000640 10 client.14949 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f5396000640 1 -- 172.21.15.62:0/991826248 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab7c1bc5a0 con 0x55ab7c2609d0 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f5396000640 10 client.14949 handle_command_reply: tid=0 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 1 client.14949 shutdown 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 1 -- 172.21.15.62:0/991826248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c2498f0 msgr2=0x55ab7c253e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c2498f0 0x55ab7c253e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f53a0005c10 tx=0x7f53a0004940 comp rx=0 tx=0).stop 2024-04-02T14:43:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 1 -- 172.21.15.62:0/991826248 shutdown_connections 2024-04-02T14:43:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ab7c2609d0 0x55ab7c261e40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c24f450 0x55ab7c25b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c2543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f5396000640 0 client.14949 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:43:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 1 --2- 172.21.15.62:0/991826248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c2498f0 0x55ab7c253e60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 1 -- 172.21.15.62:0/991826248 >> 172.21.15.62:0/991826248 conn(0x55ab7c244e60 msgr2=0x55ab7c244080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 1 -- 172.21.15.62:0/991826248 shutdown_connections 2024-04-02T14:43:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 1 -- 172.21.15.62:0/991826248 wait complete. 2024-04-02T14:43:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.320+0000 7f53b2013740 20 client.14949 trim_cache size 0 max 16384 2024-04-02T14:43:15.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.340+0000 7f53b2013740 1 Processor -- start 2024-04-02T14:43:15.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f53b2013740 1 -- start start 2024-04-02T14:43:15.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f53b2013740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c2498f0 0x55ab7c252950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f53b2013740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c25b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f53b2013740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c23fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f53b2013740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c260180 con 0x55ab7c24f450 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f53b2013740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c260850 con 0x55ab7c24a2e0 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f53b2013740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c260f20 con 0x55ab7c2498f0 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f53ad400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c2498f0 0x55ab7c252950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f53ad400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c23fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f53ad400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c23fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41436/0 (socket says 172.21.15.62:41436) 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f53ad400640 1 -- 172.21.15.62:0/2416944989 learned_addr learned my addr 172.21.15.62:0/2416944989 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.341+0000 7f539fe00640 1 --2- 172.21.15.62:0/2416944989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c25b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.342+0000 7f539fe00640 1 -- 172.21.15.62:0/2416944989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c2498f0 msgr2=0x55ab7c252950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.342+0000 7f539fe00640 1 --2- 172.21.15.62:0/2416944989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c2498f0 0x55ab7c252950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.342+0000 7f539fe00640 1 -- 172.21.15.62:0/2416944989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 msgr2=0x55ab7c23fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.342+0000 7f539fe00640 1 --2- 172.21.15.62:0/2416944989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c23fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.342+0000 7f539fe00640 1 -- 172.21.15.62:0/2416944989 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab7c24eae0 con 0x55ab7c24a2e0 2024-04-02T14:43:15.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.342+0000 7f539fe00640 1 --2- 172.21.15.62:0/2416944989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c25b370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55ab7c262580 tx=0x7f5398005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a148da5ae6e5765 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.342+0000 7f539d200640 1 -- 172.21.15.62:0/2416944989 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398023070 con 0x55ab7c24a2e0 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.342+0000 7f539d200640 1 -- 172.21.15.62:0/2416944989 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53980047a0 con 0x55ab7c24a2e0 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.342+0000 7f539d200640 1 -- 172.21.15.62:0/2416944989 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539801f5d0 con 0x55ab7c24a2e0 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.343+0000 7f53b2013740 1 -- 172.21.15.62:0/2416944989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 msgr2=0x55ab7c25b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.343+0000 7f53b2013740 1 --2- 172.21.15.62:0/2416944989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c25b370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55ab7c262580 tx=0x7f5398005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.343+0000 7f53b2013740 1 -- 172.21.15.62:0/2416944989 shutdown_connections 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.343+0000 7f53b2013740 1 --2- 172.21.15.62:0/2416944989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c23fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.343+0000 7f53b2013740 1 --2- 172.21.15.62:0/2416944989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c25b370 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.343+0000 7f53b2013740 1 --2- 172.21.15.62:0/2416944989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c2498f0 0x55ab7c252950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.343+0000 7f53b2013740 1 -- 172.21.15.62:0/2416944989 >> 172.21.15.62:0/2416944989 conn(0x55ab7c244e60 msgr2=0x55ab7c245240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.343+0000 7f53b2013740 1 -- 172.21.15.62:0/2416944989 shutdown_connections 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.343+0000 7f53b2013740 1 -- 172.21.15.62:0/2416944989 wait complete. 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.344+0000 7f53b2013740 1 -- start start 2024-04-02T14:43:15.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.344+0000 7f53b2013740 1 Processor -- start 2024-04-02T14:43:15.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.344+0000 7f53b2013740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c2498f0 0x55ab7c256520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.344+0000 7f53b2013740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c256a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.344+0000 7f53b2013740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c259a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.344+0000 7f53b2013740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c261cc0 con 0x55ab7c24f450 2024-04-02T14:43:15.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.344+0000 7f53b2013740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c2615f0 con 0x55ab7c24a2e0 2024-04-02T14:43:15.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.344+0000 7f53b2013740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c262390 con 0x55ab7c2498f0 2024-04-02T14:43:15.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.344+0000 7f53b2013740 10 client.0 _ll_register_callbacks cb 0x7f53a0004d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:15.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f539fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c256a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f53ad400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c2498f0 0x55ab7c256520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f539fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c256a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:43896/0 (socket says 172.21.15.62:43896) 2024-04-02T14:43:15.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f53ad400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c2498f0 0x55ab7c256520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:36358/0 (socket says 172.21.15.62:36358) 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f539fe00640 1 -- 172.21.15.62:0/520635793 learned_addr learned my addr 172.21.15.62:0/520635793 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f53ad400640 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c259a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f539fe00640 1 -- 172.21.15.62:0/520635793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c2498f0 msgr2=0x55ab7c256520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f539fe00640 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c2498f0 0x55ab7c256520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f539fe00640 1 -- 172.21.15.62:0/520635793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 msgr2=0x55ab7c259a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f539fe00640 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c259a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f539fe00640 1 -- 172.21.15.62:0/520635793 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab7c25a070 con 0x55ab7c24a2e0 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f53ad400640 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c259a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f539fe00640 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c256a60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55ab7c262580 tx=0x7f53980049e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8ab14e5cd28350a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.345+0000 7f5396000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.346+0000 7f5396000640 1 -- 172.21.15.62:0/520635793 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539802c030 con 0x55ab7c24a2e0 2024-04-02T14:43:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.346+0000 7f53b2013740 1 -- 172.21.15.62:0/520635793 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53980047a0 con 0x55ab7c24a2e0 2024-04-02T14:43:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.346+0000 7f5396000640 1 -- 172.21.15.62:0/520635793 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f539801f350 con 0x55ab7c24a2e0 2024-04-02T14:43:15.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.346+0000 7f5396000640 1 -- 172.21.15.62:0/520635793 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5398023050 con 0x55ab7c24a2e0 2024-04-02T14:43:15.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.346+0000 7f53b2013740 10 client.25070 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:15.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.346+0000 7f53b2013740 1 -- 172.21.15.62:0/520635793 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab7c25b380 con 0x55ab7c24a2e0 2024-04-02T14:43:15.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.346+0000 7f5396000640 1 -- 172.21.15.62:0/520635793 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53980047a0 con 0x55ab7c24a2e0 2024-04-02T14:43:15.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.346+0000 7f5396000640 1 -- 172.21.15.62:0/520635793 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f53980334d0 con 0x55ab7c24a2e0 2024-04-02T14:43:15.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.347+0000 7f53b2013740 10 client.25070 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.347+0000 7f53b2013740 10 client.25070 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.347+0000 7f53b2013740 20 client.25070 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.347+0000 7f53b2013740 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ab7c2cad10 0x55ab7c2c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.347+0000 7f53b2013740 4 client.25070 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.347+0000 7f53b2013740 1 -- 172.21.15.62:0/520635793 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "status"}) -- 0x55ab7c1b68f0 con 0x55ab7c2cad10 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.347+0000 7f53ad400640 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ab7c2cad10 0x55ab7c2c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.347+0000 7f53ad400640 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ab7c2cad10 0x55ab7c2c9a20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7306efbdcce2401 server_cookie=d590b6ef4974db49 in_seq=0 out_seq=0 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.347+0000 7f5396000640 10 client.25070 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f5396000640 1 -- 172.21.15.62:0/520635793 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ab7c1b68f0 con 0x55ab7c2cad10 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f5396000640 10 client.25070 handle_command_reply: tid=0 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 1 client.25070 shutdown 2024-04-02T14:43:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 1 -- 172.21.15.62:0/520635793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 msgr2=0x55ab7c256a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c256a60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55ab7c262580 tx=0x7f53980049e0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 1 -- 172.21.15.62:0/520635793 shutdown_connections 2024-04-02T14:43:15.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ab7c2cad10 0x55ab7c2c9a20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab7c24f450 0x55ab7c259a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab7c24a2e0 0x55ab7c256a60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f5396000640 0 client.25070 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:43:15.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 1 --2- 172.21.15.62:0/520635793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7c2498f0 0x55ab7c256520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 1 -- 172.21.15.62:0/520635793 >> 172.21.15.62:0/520635793 conn(0x55ab7c244e60 msgr2=0x55ab7c253130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 1 -- 172.21.15.62:0/520635793 shutdown_connections 2024-04-02T14:43:15.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 1 -- 172.21.15.62:0/520635793 wait complete. 2024-04-02T14:43:15.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.348+0000 7f53b2013740 20 client.25070 trim_cache size 0 max 16384 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24584, 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 112.34306266999999, 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 119.993985466 2024-04-02T14:43:15.352 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:43:15.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 -- 172.21.15.62:0/537964387 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f53880a3bb0 msgr2=0x7f53880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f53880a3bb0 0x7f53880a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55ab7c24a6e0 tx=0x7f539002bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:15.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 -- 172.21.15.62:0/537964387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a8103d40 msgr2=0x7f53a819a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:15.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a8103d40 0x7f53a819a170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5390000c00 tx=0x7f5390004a30 comp rx=0 tx=0).stop 2024-04-02T14:43:15.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 -- 172.21.15.62:0/537964387 shutdown_connections 2024-04-02T14:43:15.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f53880a3bb0 0x7f53880a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53a81058f0 0x7f53a81a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a8104f30 0x7f53a819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 --2- 172.21.15.62:0/537964387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a8103d40 0x7f53a819a170 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:15.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 -- 172.21.15.62:0/537964387 >> 172.21.15.62:0/537964387 conn(0x7f53a80ff2b0 msgr2=0x7f53a8108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:15.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 -- 172.21.15.62:0/537964387 shutdown_connections 2024-04-02T14:43:15.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:15.349+0000 7f53af400640 1 -- 172.21.15.62:0/537964387 wait complete. 2024-04-02T14:43:15.362 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:15 smithi062 ceph-mon[32116]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:15.366 INFO:tasks.cephadm:Setup complete, yielding 2024-04-02T14:43:15.366 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T14:43:15.376 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-04-02T14:43:15.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch status' 2024-04-02T14:43:15.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:15 smithi204 ceph-mon[35948]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:15.523 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:15 smithi113 ceph-mon[36135]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:43:17.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:17 smithi062 ceph-mon[32116]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:17.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:17 smithi204 ceph-mon[35948]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:17.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:17 smithi113 ceph-mon[36135]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.324+0000 7f3f83200700 1 -- 172.21.15.62:0/2836135269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c10a500 msgr2=0x7f3f7c10c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.324+0000 7f3f83200700 1 --2- 172.21.15.62:0/2836135269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c10a500 0x7f3f7c10c990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009f90 tx=0x7f3f7400b330 comp rx=0 tx=0).stop 2024-04-02T14:43:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.324+0000 7f3f83200700 1 -- 172.21.15.62:0/2836135269 shutdown_connections 2024-04-02T14:43:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.324+0000 7f3f83200700 1 --2- 172.21.15.62:0/2836135269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c10a500 0x7f3f7c10c990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.324+0000 7f3f83200700 1 --2- 172.21.15.62:0/2836135269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f7c076620 0x7f3f7c074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.324+0000 7f3f83200700 1 --2- 172.21.15.62:0/2836135269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f7c075050 0x7f3f7c0754d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.324+0000 7f3f83200700 1 -- 172.21.15.62:0/2836135269 >> 172.21.15.62:0/2836135269 conn(0x7f3f7c0fdb90 msgr2=0x7f3f7c100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.324+0000 7f3f83200700 1 -- 172.21.15.62:0/2836135269 shutdown_connections 2024-04-02T14:43:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.324+0000 7f3f83200700 1 -- 172.21.15.62:0/2836135269 wait complete. 2024-04-02T14:43:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.326+0000 7f3f83200700 1 Processor -- start 2024-04-02T14:43:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.326+0000 7f3f83200700 1 -- start start 2024-04-02T14:43:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.326+0000 7f3f83200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f7c075050 0x7f3f7c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.326+0000 7f3f83200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c076620 0x7f3f7c19b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.326+0000 7f3f83200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f7c10a500 0x7f3f7c1a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.326+0000 7f3f83200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c1105a0 con 0x7f3f7c076620 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.326+0000 7f3f83200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c110420 con 0x7f3f7c075050 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.327+0000 7f3f83200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c110720 con 0x7f3f7c10a500 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.327+0000 7f3f7be00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f7c075050 0x7f3f7c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.327+0000 7f3f81200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f7c10a500 0x7f3f7c1a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.327+0000 7f3f7be00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f7c075050 0x7f3f7c19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:43904/0 (socket says 172.21.15.62:43904) 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.327+0000 7f3f7b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c076620 0x7f3f7c19b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.327+0000 7f3f7be00700 1 -- 172.21.15.62:0/355170205 learned_addr learned my addr 172.21.15.62:0/355170205 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.327+0000 7f3f7be00700 1 -- 172.21.15.62:0/355170205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f7c10a500 msgr2=0x7f3f7c1a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.327+0000 7f3f7be00700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f7c10a500 0x7f3f7c1a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.327+0000 7f3f7be00700 1 -- 172.21.15.62:0/355170205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c076620 msgr2=0x7f3f7c19b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.327+0000 7f3f7be00700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c076620 0x7f3f7c19b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:18.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.327+0000 7f3f7be00700 1 -- 172.21.15.62:0/355170205 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c1a2750 con 0x7f3f7c075050 2024-04-02T14:43:18.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.328+0000 7f3f81200700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f7c10a500 0x7f3f7c1a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:18.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.328+0000 7f3f7be00700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f7c075050 0x7f3f7c19aaf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00d950 tx=0x7f3f6c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:18.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.328+0000 7f3f78c00700 1 -- 172.21.15.62:0/355170205 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c016070 con 0x7f3f7c075050 2024-04-02T14:43:18.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.328+0000 7f3f78c00700 1 -- 172.21.15.62:0/355170205 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3f6c004d30 con 0x7f3f7c075050 2024-04-02T14:43:18.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.328+0000 7f3f83200700 1 -- 172.21.15.62:0/355170205 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c1a2a40 con 0x7f3f7c075050 2024-04-02T14:43:18.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.328+0000 7f3f78c00700 1 -- 172.21.15.62:0/355170205 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c005520 con 0x7f3f7c075050 2024-04-02T14:43:18.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.328+0000 7f3f83200700 1 -- 172.21.15.62:0/355170205 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c1a2fd0 con 0x7f3f7c075050 2024-04-02T14:43:18.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.329+0000 7f3f83200700 1 -- 172.21.15.62:0/355170205 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7c0692b0 con 0x7f3f7c075050 2024-04-02T14:43:18.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.331+0000 7f3f78c00700 1 -- 172.21.15.62:0/355170205 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3f6c004750 con 0x7f3f7c075050 2024-04-02T14:43:18.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.334+0000 7f3f78c00700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3f600a3cc0 0x7f3f600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:18.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.334+0000 7f3f78c00700 1 -- 172.21.15.62:0/355170205 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f3f6c0d00f0 con 0x7f3f7c075050 2024-04-02T14:43:18.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.334+0000 7f3f7b400700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3f600a3cc0 0x7f3f600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:18.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.335+0000 7f3f7b400700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3f600a3cc0 0x7f3f600a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c076350 tx=0x7f3f68009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:18.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.337+0000 7f3f78c00700 1 -- 172.21.15.62:0/355170205 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3f6c01f020 con 0x7f3f7c075050 2024-04-02T14:43:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.595+0000 7f3f83200700 1 -- 172.21.15.62:0/355170205 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3f7c062a30 con 0x7f3f600a3cc0 2024-04-02T14:43:18.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.598+0000 7f3f78c00700 1 -- 172.21.15.62:0/355170205 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3f7c062a30 con 0x7f3f600a3cc0 2024-04-02T14:43:18.596 INFO:teuthology.orchestra.run.smithi062.stdout:Backend: cephadm 2024-04-02T14:43:18.597 INFO:teuthology.orchestra.run.smithi062.stdout:Available: Yes 2024-04-02T14:43:18.597 INFO:teuthology.orchestra.run.smithi062.stdout:Paused: No 2024-04-02T14:43:18.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 -- 172.21.15.62:0/355170205 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3f600a3cc0 msgr2=0x7f3f600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:18.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3f600a3cc0 0x7f3f600a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c076350 tx=0x7f3f68009310 comp rx=0 tx=0).stop 2024-04-02T14:43:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 -- 172.21.15.62:0/355170205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f7c075050 msgr2=0x7f3f7c19aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f7c075050 0x7f3f7c19aaf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00d950 tx=0x7f3f6c00dc60 comp rx=0 tx=0).stop 2024-04-02T14:43:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 -- 172.21.15.62:0/355170205 shutdown_connections 2024-04-02T14:43:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3f600a3cc0 0x7f3f600a6180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f7c075050 0x7f3f7c19aaf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f7c10a500 0x7f3f7c1a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 --2- 172.21.15.62:0/355170205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c076620 0x7f3f7c19b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 -- 172.21.15.62:0/355170205 >> 172.21.15.62:0/355170205 conn(0x7f3f7c0fdb90 msgr2=0x7f3f7c106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 -- 172.21.15.62:0/355170205 shutdown_connections 2024-04-02T14:43:18.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:18.601+0000 7f3f83200700 1 -- 172.21.15.62:0/355170205 wait complete. 2024-04-02T14:43:19.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:19 smithi062 ceph-mon[32116]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:19.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ps' 2024-04-02T14:43:19.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:19 smithi204 ceph-mon[35948]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:19.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:19 smithi113 ceph-mon[36135]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:19.872 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:43:20.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:20 smithi062 ceph-mon[32116]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:20.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:20 smithi204 ceph-mon[35948]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:20.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:20 smithi113 ceph-mon[36135]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:21.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:21 smithi204 ceph-mon[35948]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:43:21.511 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:21 smithi062 ceph-mon[32116]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:43:21.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:21 smithi113 ceph-mon[36135]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:43:22.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.057+0000 7f2569c00700 1 -- 172.21.15.62:0/4281571734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2564104470 msgr2=0x7f25641048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.057+0000 7f2569c00700 1 --2- 172.21.15.62:0/4281571734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2564104470 0x7f25641048d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f254c009920 tx=0x7f254c009c30 comp rx=0 tx=0).stop 2024-04-02T14:43:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.057+0000 7f2569c00700 1 -- 172.21.15.62:0/4281571734 shutdown_connections 2024-04-02T14:43:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.057+0000 7f2569c00700 1 --2- 172.21.15.62:0/4281571734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2564104e10 0x7f256410cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.057+0000 7f2569c00700 1 --2- 172.21.15.62:0/4281571734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2564105f30 0x7f2564106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.057+0000 7f2569c00700 1 --2- 172.21.15.62:0/4281571734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2564104470 0x7f25641048d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.057+0000 7f2569c00700 1 -- 172.21.15.62:0/4281571734 >> 172.21.15.62:0/4281571734 conn(0x7f25640ffe10 msgr2=0x7f2564102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:22.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.058+0000 7f2569c00700 1 -- 172.21.15.62:0/4281571734 shutdown_connections 2024-04-02T14:43:22.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.058+0000 7f2569c00700 1 -- 172.21.15.62:0/4281571734 wait complete. 2024-04-02T14:43:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.059+0000 7f2569c00700 1 Processor -- start 2024-04-02T14:43:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.059+0000 7f2569c00700 1 -- start start 2024-04-02T14:43:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.059+0000 7f2569c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2564104470 0x7f256419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.059+0000 7f2569c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2564104e10 0x7f256419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.059+0000 7f2569c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2564105f30 0x7f25641a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.059+0000 7f2569c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f25641106a0 con 0x7f2564104470 2024-04-02T14:43:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.059+0000 7f2569c00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2564110520 con 0x7f2564104e10 2024-04-02T14:43:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.059+0000 7f2569c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2564110820 con 0x7f2564105f30 2024-04-02T14:43:22.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f2562a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2564104470 0x7f256419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:22.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f2562a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2564104470 0x7f256419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41474/0 (socket says 172.21.15.62:41474) 2024-04-02T14:43:22.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f2562a00700 1 -- 172.21.15.62:0/3813588180 learned_addr learned my addr 172.21.15.62:0/3813588180 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:22.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f255a000700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2564104e10 0x7f256419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:22.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f2563400700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2564105f30 0x7f25641a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:22.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f2562a00700 1 -- 172.21.15.62:0/3813588180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2564105f30 msgr2=0x7f25641a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f2562a00700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2564105f30 0x7f25641a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f2562a00700 1 -- 172.21.15.62:0/3813588180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2564104e10 msgr2=0x7f256419b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f2562a00700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2564104e10 0x7f256419b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f2562a00700 1 -- 172.21.15.62:0/3813588180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25641a2900 con 0x7f2564104470 2024-04-02T14:43:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f255a000700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2564104e10 0x7f256419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.060+0000 7f2562a00700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2564104470 0x7f256419ac80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f255400e3f0 tx=0x7f255400e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.061+0000 7f255be00700 1 -- 172.21.15.62:0/3813588180 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25540092d0 con 0x7f2564104470 2024-04-02T14:43:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.061+0000 7f255be00700 1 -- 172.21.15.62:0/3813588180 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2554019070 con 0x7f2564104470 2024-04-02T14:43:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.061+0000 7f2569c00700 1 -- 172.21.15.62:0/3813588180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25641a2b90 con 0x7f2564104470 2024-04-02T14:43:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.061+0000 7f255be00700 1 -- 172.21.15.62:0/3813588180 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554014800 con 0x7f2564104470 2024-04-02T14:43:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.061+0000 7f2569c00700 1 -- 172.21.15.62:0/3813588180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25641a30a0 con 0x7f2564104470 2024-04-02T14:43:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.062+0000 7f2569c00700 1 -- 172.21.15.62:0/3813588180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25640692b0 con 0x7f2564104470 2024-04-02T14:43:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.064+0000 7f255be00700 1 -- 172.21.15.62:0/3813588180 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2554012050 con 0x7f2564104470 2024-04-02T14:43:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.066+0000 7f255be00700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25480a3c40 0x7f25480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:22.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.066+0000 7f255be00700 1 -- 172.21.15.62:0/3813588180 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f25540cfd70 con 0x7f2564104470 2024-04-02T14:43:22.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.066+0000 7f255a000700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25480a3c40 0x7f25480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:22.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.067+0000 7f255a000700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25480a3c40 0x7f25480a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f254c00fc60 tx=0x7f254c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:22.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.071+0000 7f255be00700 1 -- 172.21.15.62:0/3813588180 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2554022020 con 0x7f2564104470 2024-04-02T14:43:22.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.313+0000 7f2569c00700 1 -- 172.21.15.62:0/3813588180 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2564062a30 con 0x7f25480a3c40 2024-04-02T14:43:22.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.357+0000 7f255be00700 1 -- 172.21.15.62:0/3813588180 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f2564062a30 con 0x7f25480a3c40 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.a smithi062 running (2m) 26s ago 2m 17.0M - 19.0.0-2609-g191d963a f7301e070545 e5aee3247ea0 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.b smithi113 running (2m) 58s ago 2m 26.1M - 19.0.0-2609-g191d963a f7301e070545 0e462871d7d4 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.c smithi204 running (2m) 58s ago 2m 18.1M - 19.0.0-2609-g191d963a f7301e070545 b1021335a446 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.d smithi062 running (2m) 26s ago 2m 15.8M - 19.0.0-2609-g191d963a f7301e070545 dab721609ae7 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.e smithi113 running (2m) 58s ago 2m 19.0M - 19.0.0-2609-g191d963a f7301e070545 e9fd3363bc17 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.f smithi204 running (2m) 58s ago 2m 16.4M - 19.0.0-2609-g191d963a f7301e070545 0719b437b19e 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.g smithi062 running (2m) 26s ago 2m 16.5M - 19.0.0-2609-g191d963a f7301e070545 3fc8af3cdc2c 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.h smithi113 running (2m) 58s ago 2m 16.3M - 19.0.0-2609-g191d963a f7301e070545 16a6adee0d7a 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.i smithi204 running (2m) 58s ago 2m 17.5M - 19.0.0-2609-g191d963a f7301e070545 781f554886b8 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.j smithi062 running (2m) 26s ago 2m 12.4M - 19.0.0-2609-g191d963a f7301e070545 e1c39c3c5078 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.k smithi113 running (2m) 58s ago 2m 17.0M - 19.0.0-2609-g191d963a f7301e070545 5be66e02d694 2024-04-02T14:43:22.355 INFO:teuthology.orchestra.run.smithi062.stdout:mds.l smithi204 running (2m) 58s ago 2m 13.7M - 19.0.0-2609-g191d963a f7301e070545 c1ddf547fb05 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi062 *:9283,8765 running (11m) 26s ago 11m 515M - 19.0.0-2609-g191d963a f7301e070545 4e5db03de101 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi113 *:8443,8765 running (8m) 58s ago 8m 454M - 19.0.0-2609-g191d963a f7301e070545 995bf8c68dae 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.z smithi204 *:8443,8765 running (8m) 58s ago 8m 452M - 19.0.0-2609-g191d963a f7301e070545 a846a5f3b4b8 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (11m) 26s ago 11m 47.1M 2048M 19.0.0-2609-g191d963a f7301e070545 6078bd3e7250 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi113 running (8m) 58s ago 8m 44.0M 2048M 19.0.0-2609-g191d963a f7301e070545 65380f09a8d9 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi204 running (9m) 58s ago 9m 40.1M 2048M 19.0.0-2609-g191d963a f7301e070545 b0510f1b34f1 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (8m) 26s ago 8m 55.5M 4272M 19.0.0-2609-g191d963a f7301e070545 815503bf6b67 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi113 running (7m) 58s ago 7m 56.9M 4270M 19.0.0-2609-g191d963a f7301e070545 daceecafcb8e 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi204 running (7m) 58s ago 7m 57.6M 4270M 19.0.0-2609-g191d963a f7301e070545 417c14fb1150 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (6m) 26s ago 6m 54.3M 4272M 19.0.0-2609-g191d963a f7301e070545 520201df4310 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi113 running (6m) 58s ago 6m 52.6M 4270M 19.0.0-2609-g191d963a f7301e070545 edb50252c156 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi204 running (5m) 58s ago 5m 49.2M 4270M 19.0.0-2609-g191d963a f7301e070545 a899c2bd035a 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi062 running (5m) 26s ago 5m 50.7M 4272M 19.0.0-2609-g191d963a f7301e070545 0b5b6f838dbd 2024-04-02T14:43:22.356 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi113 running (4m) 58s ago 4m 46.0M 4270M 19.0.0-2609-g191d963a f7301e070545 2947fee64d72 2024-04-02T14:43:22.357 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 smithi204 running (4m) 58s ago 4m 55.0M 4270M 19.0.0-2609-g191d963a f7301e070545 d021ed348433 2024-04-02T14:43:22.357 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 smithi062 running (3m) 26s ago 3m 49.3M 4272M 19.0.0-2609-g191d963a f7301e070545 8eccc4555a8b 2024-04-02T14:43:22.357 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 smithi113 running (3m) 58s ago 3m 45.9M 4270M 19.0.0-2609-g191d963a f7301e070545 d1a2a974f592 2024-04-02T14:43:22.357 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 smithi204 running (3m) 58s ago 3m 44.0M 4270M 19.0.0-2609-g191d963a f7301e070545 c1892443fdb3 2024-04-02T14:43:22.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.360+0000 7f2569c00700 1 -- 172.21.15.62:0/3813588180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25480a3c40 msgr2=0x7f25480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:22.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.360+0000 7f2569c00700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25480a3c40 0x7f25480a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f254c00fc60 tx=0x7f254c019040 comp rx=0 tx=0).stop 2024-04-02T14:43:22.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.360+0000 7f2569c00700 1 -- 172.21.15.62:0/3813588180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2564104470 msgr2=0x7f256419ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:22.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.360+0000 7f2569c00700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2564104470 0x7f256419ac80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f255400e3f0 tx=0x7f255400e700 comp rx=0 tx=0).stop 2024-04-02T14:43:22.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.360+0000 7f2569c00700 1 -- 172.21.15.62:0/3813588180 shutdown_connections 2024-04-02T14:43:22.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.360+0000 7f2569c00700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25480a3c40 0x7f25480a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:22.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.360+0000 7f2569c00700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2564105f30 0x7f25641a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:22.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.360+0000 7f2569c00700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2564104470 0x7f256419ac80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:22.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.360+0000 7f2569c00700 1 --2- 172.21.15.62:0/3813588180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2564104e10 0x7f256419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:22.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.360+0000 7f2569c00700 1 -- 172.21.15.62:0/3813588180 >> 172.21.15.62:0/3813588180 conn(0x7f25640ffe10 msgr2=0x7f2564108d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:22.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.361+0000 7f2569c00700 1 -- 172.21.15.62:0/3813588180 shutdown_connections 2024-04-02T14:43:22.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:22.361+0000 7f2569c00700 1 -- 172.21.15.62:0/3813588180 wait complete. 2024-04-02T14:43:23.303 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ls' 2024-04-02T14:43:23.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:23 smithi062 ceph-mon[32116]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:23.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:23 smithi204 ceph-mon[35948]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:23.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:23 smithi113 ceph-mon[36135]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:23.778 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:43:24.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:24 smithi062 ceph-mon[32116]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:24.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:24 smithi204 ceph-mon[35948]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:24.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:24 smithi113 ceph-mon[36135]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:25.499 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:25 smithi062 ceph-mon[32116]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:25.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:25 smithi204 ceph-mon[35948]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:25.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:25 smithi113 ceph-mon[36135]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:26.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.060+0000 7f140ac00700 1 -- 172.21.15.62:0/1451232242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14041059e0 msgr2=0x7f1404107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:26.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.060+0000 7f140ac00700 1 --2- 172.21.15.62:0/1451232242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14041059e0 0x7f1404107dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0098f0 tx=0x7f13ec009c00 comp rx=0 tx=0).stop 2024-04-02T14:43:26.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.061+0000 7f140ac00700 1 -- 172.21.15.62:0/1451232242 shutdown_connections 2024-04-02T14:43:26.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.061+0000 7f140ac00700 1 --2- 172.21.15.62:0/1451232242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14041030b0 0x7f14041054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:26.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.061+0000 7f140ac00700 1 --2- 172.21.15.62:0/1451232242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1404108310 0x7f140410a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:26.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.061+0000 7f140ac00700 1 --2- 172.21.15.62:0/1451232242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14041059e0 0x7f1404107dd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:26.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.061+0000 7f140ac00700 1 -- 172.21.15.62:0/1451232242 >> 172.21.15.62:0/1451232242 conn(0x7f14040fcac0 msgr2=0x7f14040fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:26.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.061+0000 7f140ac00700 1 -- 172.21.15.62:0/1451232242 shutdown_connections 2024-04-02T14:43:26.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.061+0000 7f140ac00700 1 -- 172.21.15.62:0/1451232242 wait complete. 2024-04-02T14:43:26.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.062+0000 7f140ac00700 1 Processor -- start 2024-04-02T14:43:26.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.062+0000 7f140ac00700 1 -- start start 2024-04-02T14:43:26.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f140ac00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14041030b0 0x7f140419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:26.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f140ac00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14041059e0 0x7f140419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:26.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f140ac00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1404108310 0x7f14041a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:26.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f140ac00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f140410e350 con 0x7f14041059e0 2024-04-02T14:43:26.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f140ac00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f140410e1d0 con 0x7f1404108310 2024-04-02T14:43:26.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f140ac00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f140410e4d0 con 0x7f14041030b0 2024-04-02T14:43:26.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f13fbe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14041059e0 0x7f140419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:26.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f1408c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1404108310 0x7f14041a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:26.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f1408c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1404108310 0x7f14041a2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:45384/0 (socket says 172.21.15.62:45384) 2024-04-02T14:43:26.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f1408c00700 1 -- 172.21.15.62:0/3236243209 learned_addr learned my addr 172.21.15.62:0/3236243209 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:26.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f1403e00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14041030b0 0x7f140419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:26.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f1408c00700 1 -- 172.21.15.62:0/3236243209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14041030b0 msgr2=0x7f140419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:26.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f1408c00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14041030b0 0x7f140419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:26.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f1408c00700 1 -- 172.21.15.62:0/3236243209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14041059e0 msgr2=0x7f140419b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:26.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f1408c00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14041059e0 0x7f140419b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:26.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.063+0000 7f1408c00700 1 -- 172.21.15.62:0/3236243209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14041a27f0 con 0x7f1404108310 2024-04-02T14:43:26.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.064+0000 7f1408c00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1404108310 0x7f14041a2150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00ef90 tx=0x7f13fc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:26.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.064+0000 7f1401600700 1 -- 172.21.15.62:0/3236243209 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0092e0 con 0x7f1404108310 2024-04-02T14:43:26.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.064+0000 7f140ac00700 1 -- 172.21.15.62:0/3236243209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14041a2a80 con 0x7f1404108310 2024-04-02T14:43:26.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.064+0000 7f1401600700 1 -- 172.21.15.62:0/3236243209 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f13fc004bb0 con 0x7f1404108310 2024-04-02T14:43:26.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.064+0000 7f1401600700 1 -- 172.21.15.62:0/3236243209 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc004280 con 0x7f1404108310 2024-04-02T14:43:26.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.064+0000 7f140ac00700 1 -- 172.21.15.62:0/3236243209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14041a2fc0 con 0x7f1404108310 2024-04-02T14:43:26.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.065+0000 7f140ac00700 1 -- 172.21.15.62:0/3236243209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14040692b0 con 0x7f1404108310 2024-04-02T14:43:26.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.067+0000 7f1401600700 1 -- 172.21.15.62:0/3236243209 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f13fc01b030 con 0x7f1404108310 2024-04-02T14:43:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.069+0000 7f1401600700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f13e80a3c40 0x7f13e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.069+0000 7f1401600700 1 -- 172.21.15.62:0/3236243209 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f13fc0cfea0 con 0x7f1404108310 2024-04-02T14:43:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.069+0000 7f1403e00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f13e80a3c40 0x7f13e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.070+0000 7f1403e00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f13e80a3c40 0x7f13e80a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f13f4005fd0 tx=0x7f13f4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:26.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.073+0000 7f1401600700 1 -- 172.21.15.62:0/3236243209 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f13fc022020 con 0x7f1404108310 2024-04-02T14:43:26.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.317+0000 7f140ac00700 1 -- 172.21.15.62:0/3236243209 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f1404062a30 con 0x7f13e80a3c40 2024-04-02T14:43:26.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.327+0000 7f1401600700 1 -- 172.21.15.62:0/3236243209 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f1404062a30 con 0x7f13e80a3c40 2024-04-02T14:43:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-02T14:43:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:mds.all 12/12 62s ago 2m smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi113=b;smithi113=e;smithi113=h;smithi113=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-04-02T14:43:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:mgr 3/3 62s ago 8m smithi062=x;smithi113=y;smithi204=z;count:3 2024-04-02T14:43:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:mon 3/3 62s ago 9m smithi062:172.21.15.62=a;smithi113:172.21.15.113=b;smithi204:172.21.15.204=c;count:3 2024-04-02T14:43:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:osd 12 62s ago - 2024-04-02T14:43:26.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.330+0000 7f140ac00700 1 -- 172.21.15.62:0/3236243209 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f13e80a3c40 msgr2=0x7f13e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:26.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.330+0000 7f140ac00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f13e80a3c40 0x7f13e80a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f13f4005fd0 tx=0x7f13f4009310 comp rx=0 tx=0).stop 2024-04-02T14:43:26.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.330+0000 7f140ac00700 1 -- 172.21.15.62:0/3236243209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1404108310 msgr2=0x7f14041a2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:26.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.330+0000 7f140ac00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1404108310 0x7f14041a2150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00ef90 tx=0x7f13fc00c3a0 comp rx=0 tx=0).stop 2024-04-02T14:43:26.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.330+0000 7f140ac00700 1 -- 172.21.15.62:0/3236243209 shutdown_connections 2024-04-02T14:43:26.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.330+0000 7f140ac00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f13e80a3c40 0x7f13e80a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.330+0000 7f140ac00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1404108310 0x7f14041a2150 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.330+0000 7f140ac00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14041030b0 0x7f140419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.330+0000 7f140ac00700 1 --2- 172.21.15.62:0/3236243209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14041059e0 0x7f140419b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.330+0000 7f140ac00700 1 -- 172.21.15.62:0/3236243209 >> 172.21.15.62:0/3236243209 conn(0x7f14040fcac0 msgr2=0x7f14041028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.331+0000 7f140ac00700 1 -- 172.21.15.62:0/3236243209 shutdown_connections 2024-04-02T14:43:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:26.331+0000 7f140ac00700 1 -- 172.21.15.62:0/3236243209 wait complete. 2024-04-02T14:43:27.123 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch host ls' 2024-04-02T14:43:27.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:27 smithi062 ceph-mon[32116]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:27.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:27 smithi204 ceph-mon[35948]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:27.611 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:43:27.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:27 smithi113 ceph-mon[36135]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:28.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:28 smithi204 ceph-mon[35948]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:28 smithi062 ceph-mon[32116]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:28.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:28 smithi113 ceph-mon[36135]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:29.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:29 smithi062 ceph-mon[32116]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:29.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:29 smithi204 ceph-mon[35948]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:29.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:29 smithi113 ceph-mon[36135]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:29.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.885+0000 7f5699c00700 1 -- 172.21.15.62:0/391160233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5694108350 msgr2=0x7f569410a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:29.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.885+0000 7f5699c00700 1 --2- 172.21.15.62:0/391160233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5694108350 0x7f569410a780 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f568c00b0a0 tx=0x7f568c00b3b0 comp rx=0 tx=0).stop 2024-04-02T14:43:29.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.885+0000 7f5699c00700 1 -- 172.21.15.62:0/391160233 shutdown_connections 2024-04-02T14:43:29.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.885+0000 7f5699c00700 1 --2- 172.21.15.62:0/391160233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5694108350 0x7f569410a780 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:29.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.885+0000 7f5699c00700 1 --2- 172.21.15.62:0/391160233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56941030f0 0x7f56941054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:29.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.886+0000 7f5699c00700 1 --2- 172.21.15.62:0/391160233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5694105a20 0x7f5694107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:29.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.886+0000 7f5699c00700 1 -- 172.21.15.62:0/391160233 >> 172.21.15.62:0/391160233 conn(0x7f56940fcac0 msgr2=0x7f56940fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:29.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.886+0000 7f5699c00700 1 -- 172.21.15.62:0/391160233 shutdown_connections 2024-04-02T14:43:29.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.886+0000 7f5699c00700 1 -- 172.21.15.62:0/391160233 wait complete. 2024-04-02T14:43:29.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.887+0000 7f5699c00700 1 Processor -- start 2024-04-02T14:43:29.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.887+0000 7f5699c00700 1 -- start start 2024-04-02T14:43:29.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.887+0000 7f5699c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56941030f0 0x7f569419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:29.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.887+0000 7f5699c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5694105a20 0x7f569419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:29.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.887+0000 7f5699c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5694108350 0x7f56941a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:29.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.887+0000 7f5699c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f569410e3a0 con 0x7f5694105a20 2024-04-02T14:43:29.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.887+0000 7f5699c00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f569410e220 con 0x7f5694108350 2024-04-02T14:43:29.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.887+0000 7f5699c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f569410e520 con 0x7f56941030f0 2024-04-02T14:43:29.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.888+0000 7f5692000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5694105a20 0x7f569419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:29.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.888+0000 7f5692a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56941030f0 0x7f569419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:29.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.888+0000 7f5692000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5694105a20 0x7f569419b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55140/0 (socket says 172.21.15.62:55140) 2024-04-02T14:43:29.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.888+0000 7f5692000700 1 -- 172.21.15.62:0/135248701 learned_addr learned my addr 172.21.15.62:0/135248701 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:29.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.888+0000 7f5693400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5694108350 0x7f56941a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:29.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.888+0000 7f5692a00700 1 -- 172.21.15.62:0/135248701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5694108350 msgr2=0x7f56941a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:29.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.888+0000 7f5692a00700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5694108350 0x7f56941a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:29.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.888+0000 7f5692a00700 1 -- 172.21.15.62:0/135248701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5694105a20 msgr2=0x7f569419b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:29.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.888+0000 7f5692a00700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5694105a20 0x7f569419b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:29.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.888+0000 7f5692a00700 1 -- 172.21.15.62:0/135248701 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56941a27e0 con 0x7f56941030f0 2024-04-02T14:43:29.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.888+0000 7f5693400700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5694108350 0x7f56941a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:29.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.889+0000 7f5692a00700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56941030f0 0x7f569419ab80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f568400ea20 tx=0x7f568400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:29.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.889+0000 7f568b400700 1 -- 172.21.15.62:0/135248701 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568400cbd0 con 0x7f56941030f0 2024-04-02T14:43:29.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.889+0000 7f5699c00700 1 -- 172.21.15.62:0/135248701 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56941a2ad0 con 0x7f56941030f0 2024-04-02T14:43:29.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.889+0000 7f568b400700 1 -- 172.21.15.62:0/135248701 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5684004520 con 0x7f56941030f0 2024-04-02T14:43:29.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.889+0000 7f568b400700 1 -- 172.21.15.62:0/135248701 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684005280 con 0x7f56941030f0 2024-04-02T14:43:29.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.889+0000 7f5699c00700 1 -- 172.21.15.62:0/135248701 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56941a3010 con 0x7f56941030f0 2024-04-02T14:43:29.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.890+0000 7f5699c00700 1 -- 172.21.15.62:0/135248701 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56940692b0 con 0x7f56941030f0 2024-04-02T14:43:29.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.892+0000 7f568b400700 1 -- 172.21.15.62:0/135248701 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5684005420 con 0x7f56941030f0 2024-04-02T14:43:29.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.894+0000 7f568b400700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f56780a3c40 0x7f56780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:29.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.894+0000 7f568b400700 1 -- 172.21.15.62:0/135248701 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f56840d04b0 con 0x7f56941030f0 2024-04-02T14:43:29.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.894+0000 7f5692000700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f56780a3c40 0x7f56780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:29.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.895+0000 7f5692000700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f56780a3c40 0x7f56780a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5680007c90 tx=0x7f5680007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:29.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:29.897+0000 7f568b400700 1 -- 172.21.15.62:0/135248701 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f568409ce60 con 0x7f56941030f0 2024-04-02T14:43:30.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.138+0000 7f5699c00700 1 -- 172.21.15.62:0/135248701 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5694062a30 con 0x7f56780a3c40 2024-04-02T14:43:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.142+0000 7f568b400700 1 -- 172.21.15.62:0/135248701 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5694062a30 con 0x7f56780a3c40 2024-04-02T14:43:30.139 INFO:teuthology.orchestra.run.smithi062.stdout:HOST ADDR LABELS STATUS 2024-04-02T14:43:30.139 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 172.21.15.62 2024-04-02T14:43:30.140 INFO:teuthology.orchestra.run.smithi062.stdout:smithi113 172.21.15.113 2024-04-02T14:43:30.140 INFO:teuthology.orchestra.run.smithi062.stdout:smithi204 172.21.15.204 2024-04-02T14:43:30.140 INFO:teuthology.orchestra.run.smithi062.stdout:3 hosts in cluster 2024-04-02T14:43:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 -- 172.21.15.62:0/135248701 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f56780a3c40 msgr2=0x7f56780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f56780a3c40 0x7f56780a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5680007c90 tx=0x7f5680007480 comp rx=0 tx=0).stop 2024-04-02T14:43:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 -- 172.21.15.62:0/135248701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56941030f0 msgr2=0x7f569419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56941030f0 0x7f569419ab80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f568400ea20 tx=0x7f568400ed30 comp rx=0 tx=0).stop 2024-04-02T14:43:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 -- 172.21.15.62:0/135248701 shutdown_connections 2024-04-02T14:43:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f56780a3c40 0x7f56780a6100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5694108350 0x7f56941a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56941030f0 0x7f569419ab80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 --2- 172.21.15.62:0/135248701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5694105a20 0x7f569419b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 -- 172.21.15.62:0/135248701 >> 172.21.15.62:0/135248701 conn(0x7f56940fcac0 msgr2=0x7f56941027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 -- 172.21.15.62:0/135248701 shutdown_connections 2024-04-02T14:43:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:30.146+0000 7f5699c00700 1 -- 172.21.15.62:0/135248701 wait complete. 2024-04-02T14:43:31.097 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch device ls' 2024-04-02T14:43:31.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:31 smithi062 ceph-mon[32116]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:43:31.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:31 smithi062 ceph-mon[32116]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:31.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:31 smithi204 ceph-mon[35948]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:43:31.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:31 smithi204 ceph-mon[35948]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:43:31.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:31 smithi113 ceph-mon[36135]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:43:31.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:31 smithi113 ceph-mon[36135]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:33.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:33 smithi062 ceph-mon[32116]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:33.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:33 smithi204 ceph-mon[35948]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:33.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:33 smithi113 ceph-mon[36135]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:33.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.932+0000 7f4f36800700 1 -- 172.21.15.62:0/390706185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f30108c70 msgr2=0x7f4f30109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:33.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.932+0000 7f4f36800700 1 --2- 172.21.15.62:0/390706185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f30108c70 0x7f4f30109090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f20009920 tx=0x7f4f20009c30 comp rx=0 tx=0).stop 2024-04-02T14:43:33.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.932+0000 7f4f36800700 1 -- 172.21.15.62:0/390706185 shutdown_connections 2024-04-02T14:43:33.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.932+0000 7f4f36800700 1 --2- 172.21.15.62:0/390706185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f30108c70 0x7f4f30109090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:33.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.932+0000 7f4f36800700 1 --2- 172.21.15.62:0/390706185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3010a810 0x7f4f301110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:33.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.932+0000 7f4f36800700 1 --2- 172.21.15.62:0/390706185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f30109e70 0x7f4f3010a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:33.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.932+0000 7f4f36800700 1 -- 172.21.15.62:0/390706185 >> 172.21.15.62:0/390706185 conn(0x7f4f30077c40 msgr2=0x7f4f3007a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:33.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.932+0000 7f4f36800700 1 -- 172.21.15.62:0/390706185 shutdown_connections 2024-04-02T14:43:33.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.932+0000 7f4f36800700 1 -- 172.21.15.62:0/390706185 wait complete. 2024-04-02T14:43:33.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.934+0000 7f4f36800700 1 Processor -- start 2024-04-02T14:43:33.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.934+0000 7f4f36800700 1 -- start start 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.934+0000 7f4f36800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f30108c70 0x7f4f3019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.934+0000 7f4f36800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f30109e70 0x7f4f3019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.934+0000 7f4f36800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3010a810 0x7f4f301a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.934+0000 7f4f36800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30114990 con 0x7f4f3010a810 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.934+0000 7f4f36800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30114810 con 0x7f4f30109e70 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.934+0000 7f4f36800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30114b10 con 0x7f4f30108c70 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.935+0000 7f4f2f400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f30108c70 0x7f4f3019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.935+0000 7f4f2fe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3010a810 0x7f4f301a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.935+0000 7f4f2f400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f30108c70 0x7f4f3019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:59556/0 (socket says 172.21.15.62:59556) 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.935+0000 7f4f2f400700 1 -- 172.21.15.62:0/2243931305 learned_addr learned my addr 172.21.15.62:0/2243931305 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.935+0000 7f4f2ea00700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f30109e70 0x7f4f3019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.935+0000 7f4f2f400700 1 -- 172.21.15.62:0/2243931305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f30109e70 msgr2=0x7f4f3019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:33.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.935+0000 7f4f2f400700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f30109e70 0x7f4f3019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:33.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.935+0000 7f4f2f400700 1 -- 172.21.15.62:0/2243931305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3010a810 msgr2=0x7f4f301a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:33.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.935+0000 7f4f2f400700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3010a810 0x7f4f301a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:33.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.935+0000 7f4f2f400700 1 -- 172.21.15.62:0/2243931305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f301a6c00 con 0x7f4f30108c70 2024-04-02T14:43:33.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.935+0000 7f4f2f400700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f30108c70 0x7f4f3019efa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4f20009230 tx=0x7f4f20005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:33.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.936+0000 7f4f27e00700 1 -- 172.21.15.62:0/2243931305 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f20027070 con 0x7f4f30108c70 2024-04-02T14:43:33.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.936+0000 7f4f27e00700 1 -- 172.21.15.62:0/2243931305 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4f2000b560 con 0x7f4f30108c70 2024-04-02T14:43:33.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.936+0000 7f4f36800700 1 -- 172.21.15.62:0/2243931305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f301a6e90 con 0x7f4f30108c70 2024-04-02T14:43:33.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.936+0000 7f4f27e00700 1 -- 172.21.15.62:0/2243931305 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f20004b90 con 0x7f4f30108c70 2024-04-02T14:43:33.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.936+0000 7f4f36800700 1 -- 172.21.15.62:0/2243931305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f301a7230 con 0x7f4f30108c70 2024-04-02T14:43:33.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.937+0000 7f4f36800700 1 -- 172.21.15.62:0/2243931305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3002dd30 con 0x7f4f30108c70 2024-04-02T14:43:33.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.939+0000 7f4f27e00700 1 -- 172.21.15.62:0/2243931305 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4f2002a070 con 0x7f4f30108c70 2024-04-02T14:43:33.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.940+0000 7f4f27e00700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4f140a3c40 0x7f4f140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:33.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.941+0000 7f4f27e00700 1 -- 172.21.15.62:0/2243931305 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f4f200d1150 con 0x7f4f30108c70 2024-04-02T14:43:33.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.941+0000 7f4f2ea00700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4f140a3c40 0x7f4f140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:33.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.941+0000 7f4f2ea00700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4f140a3c40 0x7f4f140a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009c60 tx=0x7f4f1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:33.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:33.943+0000 7f4f27e00700 1 -- 172.21.15.62:0/2243931305 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4f20013030 con 0x7f4f30108c70 2024-04-02T14:43:34.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.203+0000 7f4f36800700 1 -- 172.21.15.62:0/2243931305 --> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4f30062a30 con 0x7f4f140a3c40 2024-04-02T14:43:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.207+0000 7f4f27e00700 1 -- 172.21.15.62:0/2243931305 <== mgr.14152 v2:172.21.15.62:6800/1995176952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4f30062a30 con 0x7f4f140a3c40 2024-04-02T14:43:34.205 INFO:teuthology.orchestra.run.smithi062.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-02T14:43:34.205 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440094400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-02T14:43:34.205 INFO:teuthology.orchestra.run.smithi062.stdout:smithi113 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD5171000M2P0SGN 1863G No 3m ago Has a FileSystem, LVM detected 2024-04-02T14:43:34.205 INFO:teuthology.orchestra.run.smithi062.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-02T14:43:34.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.212+0000 7f4f36800700 1 -- 172.21.15.62:0/2243931305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4f140a3c40 msgr2=0x7f4f140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:34.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.212+0000 7f4f36800700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4f140a3c40 0x7f4f140a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009c60 tx=0x7f4f1c009260 comp rx=0 tx=0).stop 2024-04-02T14:43:34.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.212+0000 7f4f36800700 1 -- 172.21.15.62:0/2243931305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f30108c70 msgr2=0x7f4f3019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:34.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.212+0000 7f4f36800700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f30108c70 0x7f4f3019efa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4f20009230 tx=0x7f4f20005040 comp rx=0 tx=0).stop 2024-04-02T14:43:34.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.213+0000 7f4f36800700 1 -- 172.21.15.62:0/2243931305 shutdown_connections 2024-04-02T14:43:34.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.213+0000 7f4f36800700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f4f140a3c40 0x7f4f140a6100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:34.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.213+0000 7f4f36800700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3010a810 0x7f4f301a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:34.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.213+0000 7f4f36800700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f30108c70 0x7f4f3019efa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:34.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.213+0000 7f4f36800700 1 --2- 172.21.15.62:0/2243931305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f30109e70 0x7f4f3019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:34.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.213+0000 7f4f36800700 1 -- 172.21.15.62:0/2243931305 >> 172.21.15.62:0/2243931305 conn(0x7f4f30077c40 msgr2=0x7f4f3010d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:34.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.213+0000 7f4f36800700 1 -- 172.21.15.62:0/2243931305 shutdown_connections 2024-04-02T14:43:34.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:34.213+0000 7f4f36800700 1 -- 172.21.15.62:0/2243931305 wait complete. 2024-04-02T14:43:35.068 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T14:43:35.078 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-04-02T14:43:35.078 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-02T14:43:35.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:35 smithi204 ceph-mon[35948]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:35.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:35 smithi204 ceph-mon[35948]: from='client.25087 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:35.574 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:43:35.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:35 smithi062 ceph-mon[32116]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:35.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:35 smithi062 ceph-mon[32116]: from='client.25087 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:35.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:35 smithi113 ceph-mon[36135]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:35.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:35 smithi113 ceph-mon[36135]: from='client.25087 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:43:36.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:36 smithi062 ceph-mon[32116]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:36.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:36 smithi113 ceph-mon[36135]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:36.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:36 smithi204 ceph-mon[35948]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.873+0000 7f2238a00700 1 -- 172.21.15.62:0/3452178660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2234103820 msgr2=0x7f2234103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.873+0000 7f2238a00700 1 --2- 172.21.15.62:0/3452178660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2234103820 0x7f2234103c80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f221c0099c0 tx=0x7f221c009cd0 comp rx=0 tx=0).stop 2024-04-02T14:43:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.873+0000 7f2238a00700 1 -- 172.21.15.62:0/3452178660 shutdown_connections 2024-04-02T14:43:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.873+0000 7f2238a00700 1 --2- 172.21.15.62:0/3452178660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2234102620 0x7f2234102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.873+0000 7f2238a00700 1 --2- 172.21.15.62:0/3452178660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22341041c0 0x7f223410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.873+0000 7f2238a00700 1 --2- 172.21.15.62:0/3452178660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2234103820 0x7f2234103c80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.873+0000 7f2238a00700 1 -- 172.21.15.62:0/3452178660 >> 172.21.15.62:0/3452178660 conn(0x7f22340fdc00 msgr2=0x7f2234100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.873+0000 7f2238a00700 1 -- 172.21.15.62:0/3452178660 shutdown_connections 2024-04-02T14:43:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.873+0000 7f2238a00700 1 -- 172.21.15.62:0/3452178660 wait complete. 2024-04-02T14:43:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.875+0000 7f2238a00700 1 Processor -- start 2024-04-02T14:43:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.875+0000 7f2238a00700 1 -- start start 2024-04-02T14:43:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.875+0000 7f2238a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2234102620 0x7f223419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.875+0000 7f2238a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2234103820 0x7f223419b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.875+0000 7f2238a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22341041c0 0x7f22341a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.875+0000 7f2238a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f223410e2d0 con 0x7f2234102620 2024-04-02T14:43:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.875+0000 7f2238a00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f223410e150 con 0x7f2234103820 2024-04-02T14:43:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.875+0000 7f2238a00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f223410e450 con 0x7f22341041c0 2024-04-02T14:43:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.875+0000 7f2231e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2234102620 0x7f223419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f2231e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2234102620 0x7f223419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39508/0 (socket says 172.21.15.62:39508) 2024-04-02T14:43:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f2231e00700 1 -- 172.21.15.62:0/2904384715 learned_addr learned my addr 172.21.15.62:0/2904384715 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f222be00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2234103820 0x7f223419b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f2232800700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22341041c0 0x7f22341a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f2231e00700 1 -- 172.21.15.62:0/2904384715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22341041c0 msgr2=0x7f22341a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f2231e00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22341041c0 0x7f22341a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f2231e00700 1 -- 172.21.15.62:0/2904384715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2234103820 msgr2=0x7f223419b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f2231e00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2234103820 0x7f223419b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f2231e00700 1 -- 172.21.15.62:0/2904384715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22341a2730 con 0x7f2234102620 2024-04-02T14:43:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f2232800700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22341041c0 0x7f22341a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f222be00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2234103820 0x7f223419b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.876+0000 7f2231e00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2234102620 0x7f223419aad0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f222400ea80 tx=0x7f222400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.877+0000 7f222aa00700 1 -- 172.21.15.62:0/2904384715 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222400cbd0 con 0x7f2234102620 2024-04-02T14:43:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.877+0000 7f222aa00700 1 -- 172.21.15.62:0/2904384715 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f222400cd70 con 0x7f2234102620 2024-04-02T14:43:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.877+0000 7f2238a00700 1 -- 172.21.15.62:0/2904384715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22341a2a20 con 0x7f2234102620 2024-04-02T14:43:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.877+0000 7f222aa00700 1 -- 172.21.15.62:0/2904384715 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2224018860 con 0x7f2234102620 2024-04-02T14:43:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.877+0000 7f2238a00700 1 -- 172.21.15.62:0/2904384715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22341a2f60 con 0x7f2234102620 2024-04-02T14:43:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.878+0000 7f2238a00700 1 -- 172.21.15.62:0/2904384715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22340692b0 con 0x7f2234102620 2024-04-02T14:43:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.880+0000 7f222aa00700 1 -- 172.21.15.62:0/2904384715 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f222401f020 con 0x7f2234102620 2024-04-02T14:43:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.881+0000 7f222aa00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f22180a3c40 0x7f22180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.882+0000 7f222aa00700 1 -- 172.21.15.62:0/2904384715 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f22240d0470 con 0x7f2234102620 2024-04-02T14:43:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.882+0000 7f222be00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f22180a3c40 0x7f22180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.883+0000 7f222be00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f22180a3c40 0x7f22180a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f221c000c00 tx=0x7f221c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:37.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:37.886+0000 7f222aa00700 1 -- 172.21.15.62:0/2904384715 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f222409ce20 con 0x7f2234102620 2024-04-02T14:43:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.170+0000 7f2238a00700 1 -- 172.21.15.62:0/2904384715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f22341a3840 con 0x7f2234102620 2024-04-02T14:43:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.172+0000 7f222aa00700 1 -- 172.21.15.62:0/2904384715 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3092 (secure 0 0 0) 0x7f222409c720 con 0x7f2234102620 2024-04-02T14:43:38.170 INFO:teuthology.orchestra.run.smithi062.stdout:e27 2024-04-02T14:43:38.170 INFO:teuthology.orchestra.run.smithi062.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T14:43:38.170 INFO:teuthology.orchestra.run.smithi062.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T14:43:38.170 INFO:teuthology.orchestra.run.smithi062.stdout:legacy client fscid: 1 2024-04-02T14:43:38.170 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:38.170 INFO:teuthology.orchestra.run.smithi062.stdout:Filesystem 'cephfs' (1) 2024-04-02T14:43:38.170 INFO:teuthology.orchestra.run.smithi062.stdout:fs_name cephfs 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 27 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-04-02T14:41:22.009309+0000 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2024-04-02T14:41:30.090009+0000 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:tableserver 0 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:root 0 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:session_timeout 300 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:session_autoclose 300 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:max_file_size 1099511627776 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:max_xattr_size 65536 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:required_client_features {} 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure 0 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure_osd_epoch 0 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T14:43:38.171 INFO:teuthology.orchestra.run.smithi062.stdout:max_mds 5 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:in 0,1,2,3,4 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:up {0=24584,1=24565,2=24553,3=24535,4=24533} 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:failed 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:damaged 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:stopped 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:data_pools [3] 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:metadata_pool 2 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:inline_data disabled 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:balancer 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:bal_rank_mask -1 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:standby_count_wanted 0 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:qdb_cluster leader: 24584 members: 24553,24565,24533,24535,24584 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b{0:24584} state up:active seq 3 addr [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c{0:24511} state up:standby-replay seq 1 addr [v2:172.21.15.204:6836/3537379638,v1:172.21.15.204:6837/3537379638] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a{1:24565} state up:active seq 5 addr [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.172 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.e{1:24509} state up:standby-replay seq 1 addr [v2:172.21.15.113:6834/360641878,v1:172.21.15.113:6835/360641878] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.g{2:24553} state up:active seq 6 addr [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.f{2:24499} state up:standby-replay seq 1 addr [v2:172.21.15.204:6834/3756932584,v1:172.21.15.204:6835/3756932584] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.i{3:24535} state up:active seq 7 addr [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d{3:24481} state up:standby-replay seq 1 addr [v2:172.21.15.62:6836/3805165269,v1:172.21.15.62:6837/3805165269] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.h{4:24533} state up:active seq 8 addr [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.k{4:24461} state up:standby-replay seq 1 addr [v2:172.21.15.113:6832/1862883972,v1:172.21.15.113:6833/1862883972] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout:Standby daemons: 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.j{-1:14382} state up:standby seq 1 addr [v2:172.21.15.62:6834/1178681921,v1:172.21.15.62:6835/1178681921] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.173 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.l{-1:24436} state up:standby seq 1 addr [v2:172.21.15.204:6832/3512482618,v1:172.21.15.204:6833/3512482618] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T14:43:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.176+0000 7f2238a00700 1 -- 172.21.15.62:0/2904384715 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f22180a3c40 msgr2=0x7f22180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.176+0000 7f2238a00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f22180a3c40 0x7f22180a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f221c000c00 tx=0x7f221c005fb0 comp rx=0 tx=0).stop 2024-04-02T14:43:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.176+0000 7f2238a00700 1 -- 172.21.15.62:0/2904384715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2234102620 msgr2=0x7f223419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.176+0000 7f2238a00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2234102620 0x7f223419aad0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f222400ea80 tx=0x7f222400ed90 comp rx=0 tx=0).stop 2024-04-02T14:43:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.176+0000 7f2238a00700 1 -- 172.21.15.62:0/2904384715 shutdown_connections 2024-04-02T14:43:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.176+0000 7f2238a00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f22180a3c40 0x7f22180a6100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.176+0000 7f2238a00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22341041c0 0x7f22341a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.176+0000 7f2238a00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2234102620 0x7f223419aad0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.176+0000 7f2238a00700 1 --2- 172.21.15.62:0/2904384715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2234103820 0x7f223419b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.176+0000 7f2238a00700 1 -- 172.21.15.62:0/2904384715 >> 172.21.15.62:0/2904384715 conn(0x7f22340fdc00 msgr2=0x7f2234074820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.177+0000 7f2238a00700 1 -- 172.21.15.62:0/2904384715 shutdown_connections 2024-04-02T14:43:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:38.177+0000 7f2238a00700 1 -- 172.21.15.62:0/2904384715 wait complete. 2024-04-02T14:43:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-02T14:43:38.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:38 smithi204 ceph-mon[35948]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:38.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:38 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2904384715' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T14:43:38.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:38 smithi062 ceph-mon[32116]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:38.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:38 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2904384715' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T14:43:38.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:38 smithi113 ceph-mon[36135]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:38.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:38 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2904384715' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T14:43:39.006 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- bash -c 'ceph osd dump' 2024-04-02T14:43:39.486 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:43:40.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:40 smithi204 ceph-mon[35948]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T14:43:40.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:40 smithi062 ceph-mon[32116]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T14:43:40.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:40 smithi113 ceph-mon[36135]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T14:43:41.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.810+0000 7f31fbc00700 1 -- 172.21.15.62:0/650375034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f4102720 msgr2=0x7f31f4102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:41.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.810+0000 7f31fbc00700 1 --2- 172.21.15.62:0/650375034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f4102720 0x7f31f4102b80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009990 tx=0x7f31e0009ca0 comp rx=0 tx=0).stop 2024-04-02T14:43:41.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.811+0000 7f31fbc00700 1 -- 172.21.15.62:0/650375034 shutdown_connections 2024-04-02T14:43:41.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.811+0000 7f31fbc00700 1 --2- 172.21.15.62:0/650375034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f41030c0 0x7f31f4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:41.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.811+0000 7f31fbc00700 1 --2- 172.21.15.62:0/650375034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f4101520 0x7f31f4101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:41.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.811+0000 7f31fbc00700 1 --2- 172.21.15.62:0/650375034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f4102720 0x7f31f4102b80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:41.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.811+0000 7f31fbc00700 1 -- 172.21.15.62:0/650375034 >> 172.21.15.62:0/650375034 conn(0x7f31f40fcac0 msgr2=0x7f31f40fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:41.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.811+0000 7f31fbc00700 1 -- 172.21.15.62:0/650375034 shutdown_connections 2024-04-02T14:43:41.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.811+0000 7f31fbc00700 1 -- 172.21.15.62:0/650375034 wait complete. 2024-04-02T14:43:41.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.812+0000 7f31fbc00700 1 Processor -- start 2024-04-02T14:43:41.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.812+0000 7f31fbc00700 1 -- start start 2024-04-02T14:43:41.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.813+0000 7f31fbc00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f4101520 0x7f31f406ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:41.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.813+0000 7f31fbc00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f4102720 0x7f31f419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:41.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.813+0000 7f31fbc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f41030c0 0x7f31f419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:41.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.813+0000 7f31fbc00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f31f410d280 con 0x7f31f41030c0 2024-04-02T14:43:41.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.813+0000 7f31fbc00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f31f410d100 con 0x7f31f4102720 2024-04-02T14:43:41.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.813+0000 7f31fbc00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f31f410d400 con 0x7f31f4101520 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.813+0000 7f31f9200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f4101520 0x7f31f406ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.813+0000 7f31f3e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f4102720 0x7f31f419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.813+0000 7f31f9c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f41030c0 0x7f31f419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.813+0000 7f31f9200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f4101520 0x7f31f406ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:59582/0 (socket says 172.21.15.62:59582) 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.813+0000 7f31f9200700 1 -- 172.21.15.62:0/1292710098 learned_addr learned my addr 172.21.15.62:0/1292710098 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.814+0000 7f31f3e00700 1 -- 172.21.15.62:0/1292710098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f4101520 msgr2=0x7f31f406ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.814+0000 7f31f3e00700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f4101520 0x7f31f406ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.814+0000 7f31f3e00700 1 -- 172.21.15.62:0/1292710098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f41030c0 msgr2=0x7f31f419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.814+0000 7f31f3e00700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f41030c0 0x7f31f419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.814+0000 7f31f3e00700 1 -- 172.21.15.62:0/1292710098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f41a0650 con 0x7f31f4102720 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.814+0000 7f31f9200700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f4101520 0x7f31f406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.814+0000 7f31f3e00700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f4102720 0x7f31f419d6f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f31e00098e0 tx=0x7f31e000b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.814+0000 7f31f1600700 1 -- 172.21.15.62:0/1292710098 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0027070 con 0x7f31f4102720 2024-04-02T14:43:41.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.815+0000 7f31f1600700 1 -- 172.21.15.62:0/1292710098 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f31e0003d90 con 0x7f31f4102720 2024-04-02T14:43:41.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.815+0000 7f31fbc00700 1 -- 172.21.15.62:0/1292710098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f41a08e0 con 0x7f31f4102720 2024-04-02T14:43:41.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.815+0000 7f31f1600700 1 -- 172.21.15.62:0/1292710098 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e000f460 con 0x7f31f4102720 2024-04-02T14:43:41.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.815+0000 7f31fbc00700 1 -- 172.21.15.62:0/1292710098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f41a0d30 con 0x7f31f4102720 2024-04-02T14:43:41.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.816+0000 7f31fbc00700 1 -- 172.21.15.62:0/1292710098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f40692b0 con 0x7f31f4102720 2024-04-02T14:43:41.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.818+0000 7f31f1600700 1 -- 172.21.15.62:0/1292710098 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f31e002a070 con 0x7f31f4102720 2024-04-02T14:43:41.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.819+0000 7f31f1600700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f31dc0a3c40 0x7f31dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:41.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.820+0000 7f31f1600700 1 -- 172.21.15.62:0/1292710098 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f31e00d1820 con 0x7f31f4102720 2024-04-02T14:43:41.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.820+0000 7f31f9200700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f31dc0a3c40 0x7f31dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:41.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.820+0000 7f31f9200700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f31dc0a3c40 0x7f31dc0a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005950 tx=0x7f31e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:41.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:41.822+0000 7f31f1600700 1 -- 172.21.15.62:0/1292710098 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f31e0013070 con 0x7f31f4102720 2024-04-02T14:43:42.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.057+0000 7f31fbc00700 1 -- 172.21.15.62:0/1292710098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f31f402ddf0 con 0x7f31f4102720 2024-04-02T14:43:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.058+0000 7f31f1600700 1 -- 172.21.15.62:0/1292710098 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5251 (secure 0 0 0) 0x7f31e009e260 con 0x7f31f4102720 2024-04-02T14:43:42.056 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 90 2024-04-02T14:43:42.056 INFO:teuthology.orchestra.run.smithi062.stdout:fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:43:42.056 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-04-02T14:31:58.064401+0000 2024-04-02T14:43:42.056 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2024-04-02T14:42:21.228892+0000 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:crush_version 26 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:full_ratio 0.95 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:backfillfull_ratio 0.9 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:nearfull_ratio 0.85 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:require_min_compat_client luminous 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:min_compat_client luminous 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:require_osd_release squid 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:stretch_mode_enabled false 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-02T14:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-02T14:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout:max_osd 12 2024-04-02T14:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6802/2100120191,v1:172.21.15.62:6803/2100120191] [v2:172.21.15.62:6804/2100120191,v1:172.21.15.62:6805/2100120191] exists,up 4fd83a35-8d86-4932-adfb-5185155bec3b 2024-04-02T14:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6800/124870295,v1:172.21.15.113:6801/124870295] [v2:172.21.15.113:6802/124870295,v1:172.21.15.113:6803/124870295] exists,up 801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5 2024-04-02T14:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/3439625144,v1:172.21.15.204:6801/3439625144] [v2:172.21.15.204:6802/3439625144,v1:172.21.15.204:6803/3439625144] exists,up 26dd6e9a-9371-40b8-bdb7-909b89f97877 2024-04-02T14:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 up in weight 1 up_from 27 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6810/263141868,v1:172.21.15.62:6811/263141868] [v2:172.21.15.62:6812/263141868,v1:172.21.15.62:6813/263141868] exists,up 8017fd90-25ff-460d-8273-c49a205ae472 2024-04-02T14:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 up in weight 1 up_from 35 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6808/3019628927,v1:172.21.15.113:6809/3019628927] [v2:172.21.15.113:6810/3019628927,v1:172.21.15.113:6811/3019628927] exists,up 5b899f2e-b8bc-4dce-991e-e222816cd0d6 2024-04-02T14:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 up in weight 1 up_from 40 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/730785574,v1:172.21.15.204:6809/730785574] [v2:172.21.15.204:6810/730785574,v1:172.21.15.204:6811/730785574] exists,up 65e6ae99-58a3-4ca6-b090-62db34e993b6 2024-04-02T14:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 up in weight 1 up_from 48 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6818/2905631892,v1:172.21.15.62:6819/2905631892] [v2:172.21.15.62:6820/2905631892,v1:172.21.15.62:6821/2905631892] exists,up 6dfddadf-9e19-4690-aa90-b9bf9044e6c9 2024-04-02T14:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6816/3862122396,v1:172.21.15.113:6817/3862122396] [v2:172.21.15.113:6818/3862122396,v1:172.21.15.113:6819/3862122396] exists,up 3097ff07-ddef-4070-88b8-d3002833f1fb 2024-04-02T14:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 up in weight 1 up_from 62 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/499713626,v1:172.21.15.204:6817/499713626] [v2:172.21.15.204:6818/499713626,v1:172.21.15.204:6819/499713626] exists,up 12ef880a-ec50-4a20-9edd-f25fcf4ac28e 2024-04-02T14:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 up in weight 1 up_from 70 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6826/465866320,v1:172.21.15.62:6827/465866320] [v2:172.21.15.62:6828/465866320,v1:172.21.15.62:6829/465866320] exists,up 2f2258db-f7e1-432b-be11-a2299b46c30f 2024-04-02T14:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 up in weight 1 up_from 75 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6824/4261348940,v1:172.21.15.113:6825/4261348940] [v2:172.21.15.113:6826/4261348940,v1:172.21.15.113:6827/4261348940] exists,up 1ccb9dfd-397e-4996-a37d-1161181a34cd 2024-04-02T14:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 up in weight 1 up_from 83 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/1914333038,v1:172.21.15.204:6825/1914333038] [v2:172.21.15.204:6826/1914333038,v1:172.21.15.204:6827/1914333038] exists,up d6f10568-eebc-4be6-92eb-942cc4afa1f6 2024-04-02T14:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.4 [2,11] 2024-04-02T14:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.b [8,11] 2024-04-02T14:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.1b [2,4] 2024-04-02T14:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.3 [4,9] 2024-04-02T14:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.1b [5,9] 2024-04-02T14:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.27 [4,8] 2024-04-02T14:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.3c [4,9] 2024-04-02T14:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/1356430694 expires 2024-04-03T14:33:19.211822+0000 2024-04-02T14:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/1356430694 expires 2024-04-03T14:33:19.211822+0000 2024-04-02T14:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/2679742291 expires 2024-04-03T14:33:19.211822+0000 2024-04-02T14:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/3147101239 expires 2024-04-03T14:32:43.373728+0000 2024-04-02T14:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/3831753172 expires 2024-04-03T14:33:19.211822+0000 2024-04-02T14:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/2106382100 expires 2024-04-03T14:32:43.373728+0000 2024-04-02T14:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/3623171244 expires 2024-04-03T14:32:43.373728+0000 2024-04-02T14:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/3623171244 expires 2024-04-03T14:32:43.373728+0000 2024-04-02T14:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/3824719077 expires 2024-04-03T14:32:43.373728+0000 2024-04-02T14:43:42.061 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/2038939412 expires 2024-04-03T14:33:19.211822+0000 2024-04-02T14:43:42.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 -- 172.21.15.62:0/1292710098 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f31dc0a3c40 msgr2=0x7f31dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f31dc0a3c40 0x7f31dc0a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005950 tx=0x7f31e8009260 comp rx=0 tx=0).stop 2024-04-02T14:43:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 -- 172.21.15.62:0/1292710098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f4102720 msgr2=0x7f31f419d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f4102720 0x7f31f419d6f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f31e00098e0 tx=0x7f31e000b9f0 comp rx=0 tx=0).stop 2024-04-02T14:43:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 -- 172.21.15.62:0/1292710098 shutdown_connections 2024-04-02T14:43:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f31dc0a3c40 0x7f31dc0a6100 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f41030c0 0x7f31f419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f4101520 0x7f31f406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:42.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 --2- 172.21.15.62:0/1292710098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f4102720 0x7f31f419d6f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:42.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 -- 172.21.15.62:0/1292710098 >> 172.21.15.62:0/1292710098 conn(0x7f31f40fcac0 msgr2=0x7f31f4105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:42.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 -- 172.21.15.62:0/1292710098 shutdown_connections 2024-04-02T14:43:42.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:42.062+0000 7f31fbc00700 1 -- 172.21.15.62:0/1292710098 wait complete. 2024-04-02T14:43:42.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:42 smithi204 ceph-mon[35948]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:42.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:42 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1292710098' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T14:43:42.832 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:42 smithi062 ceph-mon[32116]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:42.832 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:42 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1292710098' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T14:43:42.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:42 smithi113 ceph-mon[36135]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T14:43:42.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:42 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1292710098' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T14:43:42.894 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-02T14:43:42.909 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:43:43.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.154+0000 7f85aaa00640 1 -- 172.21.15.62:0/4010211468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a4107f10 msgr2=0x7f85a4108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:43.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.154+0000 7f85aaa00640 1 --2- 172.21.15.62:0/4010211468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a4107f10 0x7f85a4108350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8598009900 tx=0x7f859802bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:43.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.154+0000 7f85aaa00640 1 -- 172.21.15.62:0/4010211468 shutdown_connections 2024-04-02T14:43:43.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.154+0000 7f85aaa00640 1 --2- 172.21.15.62:0/4010211468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4109aa0 0x7f85a41103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.154+0000 7f85aaa00640 1 --2- 172.21.15.62:0/4010211468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a41090e0 0x7f85a4109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.154+0000 7f85aaa00640 1 --2- 172.21.15.62:0/4010211468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a4107f10 0x7f85a4108350 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.154+0000 7f85aaa00640 1 -- 172.21.15.62:0/4010211468 >> 172.21.15.62:0/4010211468 conn(0x7f85a4076f10 msgr2=0x7f85a4079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:43.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.155+0000 7f85aaa00640 1 -- 172.21.15.62:0/4010211468 shutdown_connections 2024-04-02T14:43:43.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.155+0000 7f85aaa00640 1 -- 172.21.15.62:0/4010211468 wait complete. 2024-04-02T14:43:43.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.155+0000 7f85aaa00640 1 Processor -- start 2024-04-02T14:43:43.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85aaa00640 1 -- start start 2024-04-02T14:43:43.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85aaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a4107f10 0x7f85a419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:43.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85aaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a41090e0 0x7f85a419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85aaa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4109aa0 0x7f85a41a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85aaa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4113cd0 con 0x7f85a41090e0 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85aaa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4113b50 con 0x7f85a4107f10 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85aaa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4113e50 con 0x7f85a4109aa0 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85a8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a41090e0 0x7f85a419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85a8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a41090e0 0x7f85a419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46366/0 (socket says 172.21.15.62:46366) 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85a8c00640 1 -- 172.21.15.62:0/3945242151 learned_addr learned my addr 172.21.15.62:0/3945242151 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85a9600640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a4107f10 0x7f85a419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.156+0000 7f85aa000640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4109aa0 0x7f85a41a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.157+0000 7f85a9600640 1 -- 172.21.15.62:0/3945242151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4109aa0 msgr2=0x7f85a41a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.157+0000 7f85a9600640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4109aa0 0x7f85a41a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.157+0000 7f85a9600640 1 -- 172.21.15.62:0/3945242151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a41090e0 msgr2=0x7f85a419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.157+0000 7f85a9600640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a41090e0 0x7f85a419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.157+0000 7f85a9600640 1 -- 172.21.15.62:0/3945242151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a41a5f90 con 0x7f85a4107f10 2024-04-02T14:43:43.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.157+0000 7f85a8c00640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a41090e0 0x7f85a419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:43.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.157+0000 7f85a9600640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a4107f10 0x7f85a419e370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8598036040 tx=0x7f8598004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:43.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.157+0000 7f85a2000640 1 -- 172.21.15.62:0/3945242151 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598043020 con 0x7f85a4107f10 2024-04-02T14:43:43.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.157+0000 7f85a2000640 1 -- 172.21.15.62:0/3945242151 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8598004310 con 0x7f85a4107f10 2024-04-02T14:43:43.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.157+0000 7f85aaa00640 1 -- 172.21.15.62:0/3945242151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a41a6220 con 0x7f85a4107f10 2024-04-02T14:43:43.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.157+0000 7f85a2000640 1 -- 172.21.15.62:0/3945242151 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598003af0 con 0x7f85a4107f10 2024-04-02T14:43:43.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.158+0000 7f85aaa00640 1 -- 172.21.15.62:0/3945242151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a41a6740 con 0x7f85a4107f10 2024-04-02T14:43:43.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.159+0000 7f85aaa00640 1 -- 172.21.15.62:0/3945242151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a410f2f0 con 0x7f85a4107f10 2024-04-02T14:43:43.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.161+0000 7f85a2000640 1 -- 172.21.15.62:0/3945242151 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8598046020 con 0x7f85a4107f10 2024-04-02T14:43:43.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.162+0000 7f85a2000640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3bb0 0x7f858c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:43.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.162+0000 7f85a2000640 1 -- 172.21.15.62:0/3945242151 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f85980f2a70 con 0x7f85a4107f10 2024-04-02T14:43:43.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.162+0000 7f85a8c00640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3bb0 0x7f858c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:43.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.163+0000 7f85a8c00640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3bb0 0x7f858c0a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f85a419f880 tx=0x7f8590005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:43.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.164+0000 7f85a2000640 1 -- 172.21.15.62:0/3945242151 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f859804d020 con 0x7f85a4107f10 2024-04-02T14:43:43.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.372+0000 7f85aaa00640 1 -- 172.21.15.62:0/3945242151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85a4062700 con 0x7f85a4107f10 2024-04-02T14:43:43.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.374+0000 7f85a2000640 1 -- 172.21.15.62:0/3945242151 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22583 (secure 0 0 0) 0x7f85980bf420 con 0x7f85a4107f10 2024-04-02T14:43:43.372 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:43.372 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:42:21.228892+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:43:43.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.377+0000 7f85aaa00640 1 -- 172.21.15.62:0/3945242151 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3bb0 msgr2=0x7f858c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:43.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.377+0000 7f85aaa00640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3bb0 0x7f858c0a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f85a419f880 tx=0x7f8590005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:43.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.377+0000 7f85aaa00640 1 -- 172.21.15.62:0/3945242151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a4107f10 msgr2=0x7f85a419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:43.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.377+0000 7f85aaa00640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a4107f10 0x7f85a419e370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8598036040 tx=0x7f8598004a30 comp rx=0 tx=0).stop 2024-04-02T14:43:43.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.377+0000 7f85aaa00640 1 -- 172.21.15.62:0/3945242151 shutdown_connections 2024-04-02T14:43:43.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.377+0000 7f85aaa00640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f858c0a3bb0 0x7f858c0a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.377+0000 7f85aaa00640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4109aa0 0x7f85a41a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.377+0000 7f85aaa00640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a41090e0 0x7f85a419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.377+0000 7f85aaa00640 1 --2- 172.21.15.62:0/3945242151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a4107f10 0x7f85a419e370 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.377+0000 7f85aaa00640 1 -- 172.21.15.62:0/3945242151 >> 172.21.15.62:0/3945242151 conn(0x7f85a4076f10 msgr2=0x7f85a410c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:43.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.377+0000 7f85aaa00640 1 -- 172.21.15.62:0/3945242151 shutdown_connections 2024-04-02T14:43:43.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.378+0000 7f85aaa00640 1 -- 172.21.15.62:0/3945242151 wait complete. 2024-04-02T14:43:43.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T14:41:20.030393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T14:41:20.917405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:43:43.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:43:43.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.628+0000 7f3c1a600640 1 -- 172.21.15.62:0/486645134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c14107f20 msgr2=0x7f3c14108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:43.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.628+0000 7f3c1a600640 1 --2- 172.21.15.62:0/486645134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c14107f20 0x7f3c14108340 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009860 tx=0x7f3c0802bb70 comp rx=0 tx=0).stop 2024-04-02T14:43:43.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.628+0000 7f3c1a600640 1 -- 172.21.15.62:0/486645134 shutdown_connections 2024-04-02T14:43:43.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.628+0000 7f3c1a600640 1 --2- 172.21.15.62:0/486645134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c14109a30 0x7f3c14110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.628+0000 7f3c1a600640 1 --2- 172.21.15.62:0/486645134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c14109070 0x7f3c141094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.628+0000 7f3c1a600640 1 --2- 172.21.15.62:0/486645134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c14107f20 0x7f3c14108340 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.628+0000 7f3c1a600640 1 -- 172.21.15.62:0/486645134 >> 172.21.15.62:0/486645134 conn(0x7f3c14076ec0 msgr2=0x7f3c14079300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:43.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.628+0000 7f3c1a600640 1 -- 172.21.15.62:0/486645134 shutdown_connections 2024-04-02T14:43:43.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.628+0000 7f3c1a600640 1 -- 172.21.15.62:0/486645134 wait complete. 2024-04-02T14:43:43.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.629+0000 7f3c1a600640 1 Processor -- start 2024-04-02T14:43:43.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.629+0000 7f3c1a600640 1 -- start start 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.629+0000 7f3c1a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c14107f20 0x7f3c1419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.629+0000 7f3c1a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c14109070 0x7f3c1419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.629+0000 7f3c1a600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c14109a30 0x7f3c141a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.629+0000 7f3c1a600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14113d00 con 0x7f3c14109a30 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.629+0000 7f3c1a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14113b80 con 0x7f3c14107f20 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.629+0000 7f3c1a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14113e80 con 0x7f3c14109070 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c19c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c14109a30 0x7f3c141a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c19200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c14107f20 0x7f3c1419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c19c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c14109a30 0x7f3c141a5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46382/0 (socket says 172.21.15.62:46382) 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c19200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c14107f20 0x7f3c1419e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:32774/0 (socket says 172.21.15.62:32774) 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c19c00640 1 -- 172.21.15.62:0/1009891172 learned_addr learned my addr 172.21.15.62:0/1009891172 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c13e00640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c14109070 0x7f3c1419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c19200640 1 -- 172.21.15.62:0/1009891172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c14109070 msgr2=0x7f3c1419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c19200640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c14109070 0x7f3c1419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c19200640 1 -- 172.21.15.62:0/1009891172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c14109a30 msgr2=0x7f3c141a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c19200640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c14109a30 0x7f3c141a5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c19200640 1 -- 172.21.15.62:0/1009891172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c141a5f00 con 0x7f3c14107f20 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.630+0000 7f3c19200640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c14107f20 0x7f3c1419e2e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3c0802c0b0 tx=0x7f3c08005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.631+0000 7f3c11600640 1 -- 172.21.15.62:0/1009891172 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08041020 con 0x7f3c14107f20 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.631+0000 7f3c11600640 1 -- 172.21.15.62:0/1009891172 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3c0802ca10 con 0x7f3c14107f20 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.631+0000 7f3c1a600640 1 -- 172.21.15.62:0/1009891172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c141a6190 con 0x7f3c14107f20 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.631+0000 7f3c11600640 1 -- 172.21.15.62:0/1009891172 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08039040 con 0x7f3c14107f20 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.631+0000 7f3c1a600640 1 -- 172.21.15.62:0/1009891172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c141a6650 con 0x7f3c14107f20 2024-04-02T14:43:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.632+0000 7f3c1a600640 1 -- 172.21.15.62:0/1009891172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1410f230 con 0x7f3c14107f20 2024-04-02T14:43:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.634+0000 7f3c11600640 1 -- 172.21.15.62:0/1009891172 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3c08044020 con 0x7f3c14107f20 2024-04-02T14:43:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.635+0000 7f3c11600640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3bfc0a3bb0 0x7f3bfc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.635+0000 7f3c11600640 1 -- 172.21.15.62:0/1009891172 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f3c080f1ba0 con 0x7f3c14107f20 2024-04-02T14:43:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.635+0000 7f3c13e00640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3bfc0a3bb0 0x7f3bfc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.636+0000 7f3c13e00640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3bfc0a3bb0 0x7f3bfc0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3c1419f7f0 tx=0x7f3c04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:43.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.640+0000 7f3c11600640 1 -- 172.21.15.62:0/1009891172 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3c0804b020 con 0x7f3c14107f20 2024-04-02T14:43:43.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:43 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3945242151' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:43.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:43 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3945242151' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:43.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.847+0000 7f3c1a600640 1 -- 172.21.15.62:0/1009891172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c14062730 con 0x7f3c14107f20 2024-04-02T14:43:43.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.847+0000 7f3c11600640 1 -- 172.21.15.62:0/1009891172 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f3c080be550 con 0x7f3c14107f20 2024-04-02T14:43:43.844 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:43:43.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.850+0000 7f3c1a600640 1 -- 172.21.15.62:0/1009891172 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3bfc0a3bb0 msgr2=0x7f3bfc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:43.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.850+0000 7f3c1a600640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3bfc0a3bb0 0x7f3bfc0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3c1419f7f0 tx=0x7f3c04009260 comp rx=0 tx=0).stop 2024-04-02T14:43:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.850+0000 7f3c1a600640 1 -- 172.21.15.62:0/1009891172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c14107f20 msgr2=0x7f3c1419e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.850+0000 7f3c1a600640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c14107f20 0x7f3c1419e2e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3c0802c0b0 tx=0x7f3c08005dc0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.850+0000 7f3c1a600640 1 -- 172.21.15.62:0/1009891172 shutdown_connections 2024-04-02T14:43:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.850+0000 7f3c1a600640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f3bfc0a3bb0 0x7f3bfc0a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.851+0000 7f3c1a600640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c14109a30 0x7f3c141a5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.851+0000 7f3c1a600640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c14109070 0x7f3c1419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.851+0000 7f3c1a600640 1 --2- 172.21.15.62:0/1009891172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c14107f20 0x7f3c1419e2e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.851+0000 7f3c1a600640 1 -- 172.21.15.62:0/1009891172 >> 172.21.15.62:0/1009891172 conn(0x7f3c14076ec0 msgr2=0x7f3c1410c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.851+0000 7f3c1a600640 1 -- 172.21.15.62:0/1009891172 shutdown_connections 2024-04-02T14:43:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:43.851+0000 7f3c1a600640 1 -- 172.21.15.62:0/1009891172 wait complete. 2024-04-02T14:43:43.867 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T14:43:43.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:43 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3945242151' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.098+0000 7f7a94a00640 1 -- 172.21.15.62:0/4077802351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a90102bd0 msgr2=0x7f7a90103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.098+0000 7f7a94a00640 1 --2- 172.21.15.62:0/4077802351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a90102bd0 0x7f7a90103050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009860 tx=0x7f7a7802bb70 comp rx=0 tx=0).stop 2024-04-02T14:43:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.098+0000 7f7a94a00640 1 -- 172.21.15.62:0/4077802351 shutdown_connections 2024-04-02T14:43:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.098+0000 7f7a94a00640 1 --2- 172.21.15.62:0/4077802351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a90103590 0x7f7a90109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.098+0000 7f7a94a00640 1 --2- 172.21.15.62:0/4077802351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a90102bd0 0x7f7a90103050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.098+0000 7f7a94a00640 1 --2- 172.21.15.62:0/4077802351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a901019e0 0x7f7a90101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.098+0000 7f7a94a00640 1 -- 172.21.15.62:0/4077802351 >> 172.21.15.62:0/4077802351 conn(0x7f7a900fcf70 msgr2=0x7f7a900ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:44.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.098+0000 7f7a94a00640 1 -- 172.21.15.62:0/4077802351 shutdown_connections 2024-04-02T14:43:44.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.098+0000 7f7a94a00640 1 -- 172.21.15.62:0/4077802351 wait complete. 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.099+0000 7f7a94a00640 1 Processor -- start 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.099+0000 7f7a94a00640 1 -- start start 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.099+0000 7f7a94a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a901019e0 0x7f7a90197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a94a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a90102bd0 0x7f7a901982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a94a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a90103590 0x7f7a9019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a94a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9010d790 con 0x7f7a901019e0 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a94a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9010d610 con 0x7f7a90102bd0 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a94a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9010d910 con 0x7f7a90103590 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a901019e0 0x7f7a90197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a901019e0 0x7f7a90197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46396/0 (socket says 172.21.15.62:46396) 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8f400640 1 -- 172.21.15.62:0/4279893908 learned_addr learned my addr 172.21.15.62:0/4279893908 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:44.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8fe00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a90103590 0x7f7a9019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8ea00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a90102bd0 0x7f7a901982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8f400640 1 -- 172.21.15.62:0/4279893908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a90103590 msgr2=0x7f7a9019f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8f400640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a90103590 0x7f7a9019f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8f400640 1 -- 172.21.15.62:0/4279893908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a90102bd0 msgr2=0x7f7a901982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8f400640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a90102bd0 0x7f7a901982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8f400640 1 -- 172.21.15.62:0/4279893908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9019f990 con 0x7f7a901019e0 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8fe00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a90103590 0x7f7a9019f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.100+0000 7f7a8ea00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a90102bd0 0x7f7a901982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.101+0000 7f7a8f400640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a901019e0 0x7f7a90197d70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000e9a0 tx=0x7f7a8000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.101+0000 7f7a87e00640 1 -- 172.21.15.62:0/4279893908 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000cd80 con 0x7f7a901019e0 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.101+0000 7f7a87e00640 1 -- 172.21.15.62:0/4279893908 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7a80010930 con 0x7f7a901019e0 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.101+0000 7f7a94a00640 1 -- 172.21.15.62:0/4279893908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9019fc80 con 0x7f7a901019e0 2024-04-02T14:43:44.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.101+0000 7f7a87e00640 1 -- 172.21.15.62:0/4279893908 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80018a70 con 0x7f7a901019e0 2024-04-02T14:43:44.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.101+0000 7f7a94a00640 1 -- 172.21.15.62:0/4279893908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a901a01a0 con 0x7f7a901019e0 2024-04-02T14:43:44.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.102+0000 7f7a94a00640 1 -- 172.21.15.62:0/4279893908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a90108ca0 con 0x7f7a901019e0 2024-04-02T14:43:44.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.104+0000 7f7a87e00640 1 -- 172.21.15.62:0/4279893908 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7a80018c10 con 0x7f7a901019e0 2024-04-02T14:43:44.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.107+0000 7f7a87e00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7a740a3bb0 0x7f7a740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:44.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.107+0000 7f7a87e00640 1 -- 172.21.15.62:0/4279893908 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f7a800d08d0 con 0x7f7a901019e0 2024-04-02T14:43:44.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.107+0000 7f7a8ea00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7a740a3bb0 0x7f7a740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:44.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.108+0000 7f7a8ea00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7a740a3bb0 0x7f7a740a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009830 tx=0x7f7a78005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:44.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.111+0000 7f7a87e00640 1 -- 172.21.15.62:0/4279893908 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7a80053f20 con 0x7f7a901019e0 2024-04-02T14:43:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.317+0000 7f7a94a00640 1 -- 172.21.15.62:0/4279893908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7a90062700 con 0x7f7a901019e0 2024-04-02T14:43:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.317+0000 7f7a87e00640 1 -- 172.21.15.62:0/4279893908 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f7a8009d280 con 0x7f7a901019e0 2024-04-02T14:43:44.314 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-02T14:43:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 -- 172.21.15.62:0/4279893908 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7a740a3bb0 msgr2=0x7f7a740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7a740a3bb0 0x7f7a740a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009830 tx=0x7f7a78005d20 comp rx=0 tx=0).stop 2024-04-02T14:43:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 -- 172.21.15.62:0/4279893908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a901019e0 msgr2=0x7f7a90197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a901019e0 0x7f7a90197d70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000e9a0 tx=0x7f7a8000ee60 comp rx=0 tx=0).stop 2024-04-02T14:43:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 -- 172.21.15.62:0/4279893908 shutdown_connections 2024-04-02T14:43:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f7a740a3bb0 0x7f7a740a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a90103590 0x7f7a9019f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a90102bd0 0x7f7a901982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 --2- 172.21.15.62:0/4279893908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a901019e0 0x7f7a90197d70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 -- 172.21.15.62:0/4279893908 >> 172.21.15.62:0/4279893908 conn(0x7f7a900fcf70 msgr2=0x7f7a90105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:44.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 -- 172.21.15.62:0/4279893908 shutdown_connections 2024-04-02T14:43:44.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.320+0000 7f7a94a00640 1 -- 172.21.15.62:0/4279893908 wait complete. 2024-04-02T14:43:44.338 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T14:43:44.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.573+0000 7fc3a1c00640 1 -- 172.21.15.62:0/487814459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc39c104e10 msgr2=0x7fc39c105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:44.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.573+0000 7fc3a1c00640 1 --2- 172.21.15.62:0/487814459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc39c104e10 0x7fc39c105290 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc384009900 tx=0x7fc38402bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:44.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.573+0000 7fc3a1c00640 1 -- 172.21.15.62:0/487814459 shutdown_connections 2024-04-02T14:43:44.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.573+0000 7fc3a1c00640 1 --2- 172.21.15.62:0/487814459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c1057d0 0x7fc39c10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.573+0000 7fc3a1c00640 1 --2- 172.21.15.62:0/487814459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc39c104e10 0x7fc39c105290 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.573+0000 7fc3a1c00640 1 --2- 172.21.15.62:0/487814459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc39c103c20 0x7fc39c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.573+0000 7fc3a1c00640 1 -- 172.21.15.62:0/487814459 >> 172.21.15.62:0/487814459 conn(0x7fc39c0ff1b0 msgr2=0x7fc39c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:44.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.573+0000 7fc3a1c00640 1 -- 172.21.15.62:0/487814459 shutdown_connections 2024-04-02T14:43:44.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.573+0000 7fc3a1c00640 1 -- 172.21.15.62:0/487814459 wait complete. 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.574+0000 7fc3a1c00640 1 Processor -- start 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.574+0000 7fc3a1c00640 1 -- start start 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.574+0000 7fc3a1c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c103c20 0x7fc39c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.574+0000 7fc3a1c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc39c104e10 0x7fc39c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.574+0000 7fc3a1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc39c1057d0 0x7fc39c1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.574+0000 7fc3a1c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c10f980 con 0x7fc39c1057d0 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.574+0000 7fc3a1c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c10f800 con 0x7fc39c104e10 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.574+0000 7fc3a1c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c10fb00 con 0x7fc39c103c20 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.574+0000 7fc3a1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc39c1057d0 0x7fc39c1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.575+0000 7fc3a1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc39c1057d0 0x7fc39c1a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46426/0 (socket says 172.21.15.62:46426) 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.575+0000 7fc3a1200640 1 -- 172.21.15.62:0/199372762 learned_addr learned my addr 172.21.15.62:0/199372762 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:44.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.575+0000 7fc393e00640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc39c104e10 0x7fc39c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.575+0000 7fc39be00640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c103c20 0x7fc39c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.575+0000 7fc3a1200640 1 -- 172.21.15.62:0/199372762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c103c20 msgr2=0x7fc39c199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.575+0000 7fc3a1200640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c103c20 0x7fc39c199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.575+0000 7fc3a1200640 1 -- 172.21.15.62:0/199372762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc39c104e10 msgr2=0x7fc39c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.575+0000 7fc3a1200640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc39c104e10 0x7fc39c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.575+0000 7fc3a1200640 1 -- 172.21.15.62:0/199372762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39c1a1bc0 con 0x7fc39c1057d0 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.575+0000 7fc3a1200640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc39c1057d0 0x7fc39c1a1550 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc39400eb10 tx=0x7fc39400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.576+0000 7fc399600640 1 -- 172.21.15.62:0/199372762 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394004b20 con 0x7fc39c1057d0 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.576+0000 7fc3a1c00640 1 -- 172.21.15.62:0/199372762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39c1a1eb0 con 0x7fc39c1057d0 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.576+0000 7fc399600640 1 -- 172.21.15.62:0/199372762 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc394004cc0 con 0x7fc39c1057d0 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.576+0000 7fc399600640 1 -- 172.21.15.62:0/199372762 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3940107f0 con 0x7fc39c1057d0 2024-04-02T14:43:44.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.576+0000 7fc3a1c00640 1 -- 172.21.15.62:0/199372762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39c1a2370 con 0x7fc39c1057d0 2024-04-02T14:43:44.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.577+0000 7fc3a1c00640 1 -- 172.21.15.62:0/199372762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc39c10af40 con 0x7fc39c1057d0 2024-04-02T14:43:44.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.579+0000 7fc399600640 1 -- 172.21.15.62:0/199372762 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc394005020 con 0x7fc39c1057d0 2024-04-02T14:43:44.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.580+0000 7fc399600640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc3800a3bb0 0x7fc3800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:44.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.580+0000 7fc399600640 1 -- 172.21.15.62:0/199372762 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fc3940d08c0 con 0x7fc39c1057d0 2024-04-02T14:43:44.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.580+0000 7fc39be00640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc3800a3bb0 0x7fc3800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:44.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.581+0000 7fc39be00640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc3800a3bb0 0x7fc3800a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc38c005fd0 tx=0x7fc38c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:44.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.584+0000 7fc399600640 1 -- 172.21.15.62:0/199372762 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc39409d270 con 0x7fc39c1057d0 2024-04-02T14:43:44.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:44 smithi062 ceph-mon[32116]: pgmap v392: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:44.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:44 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1009891172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:44.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:44 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/4279893908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:44.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:44 smithi204 ceph-mon[35948]: pgmap v392: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:44.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:44 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1009891172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:44.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:44 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/4279893908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:44.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.792+0000 7fc3a1c00640 1 -- 172.21.15.62:0/199372762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc39c062700 con 0x7fc39c1057d0 2024-04-02T14:43:44.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.793+0000 7fc399600640 1 -- 172.21.15.62:0/199372762 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fc39409cb70 con 0x7fc39c1057d0 2024-04-02T14:43:44.790 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-02T14:43:44.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 -- 172.21.15.62:0/199372762 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc3800a3bb0 msgr2=0x7fc3800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:44.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc3800a3bb0 0x7fc3800a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc38c005fd0 tx=0x7fc38c00b410 comp rx=0 tx=0).stop 2024-04-02T14:43:44.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 -- 172.21.15.62:0/199372762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc39c1057d0 msgr2=0x7fc39c1a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:44.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc39c1057d0 0x7fc39c1a1550 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc39400eb10 tx=0x7fc39400c2d0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 -- 172.21.15.62:0/199372762 shutdown_connections 2024-04-02T14:43:44.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc3800a3bb0 0x7fc3800a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc39c1057d0 0x7fc39c1a1550 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc39c104e10 0x7fc39c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 --2- 172.21.15.62:0/199372762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c103c20 0x7fc39c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:44.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 -- 172.21.15.62:0/199372762 >> 172.21.15.62:0/199372762 conn(0x7fc39c0ff1b0 msgr2=0x7fc39c108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:44.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 -- 172.21.15.62:0/199372762 shutdown_connections 2024-04-02T14:43:44.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:44.796+0000 7fc3a1c00640 1 -- 172.21.15.62:0/199372762 wait complete. 2024-04-02T14:43:44.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:43:44.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:44 smithi113 ceph-mon[36135]: pgmap v392: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:44.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:44 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1009891172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:44.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:44 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/4279893908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:45.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.046+0000 7f0633200640 1 -- 172.21.15.62:0/2397371400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06341023d0 msgr2=0x7f06341047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.046+0000 7f0633200640 1 --2- 172.21.15.62:0/2397371400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06341023d0 0x7f06341047c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0624009900 tx=0x7f062402bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.047+0000 7f0633200640 1 -- 172.21.15.62:0/2397371400 shutdown_connections 2024-04-02T14:43:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.047+0000 7f0633200640 1 --2- 172.21.15.62:0/2397371400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0634107630 0x7f0634109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.047+0000 7f0633200640 1 --2- 172.21.15.62:0/2397371400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0634104d00 0x7f06341070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.047+0000 7f0633200640 1 --2- 172.21.15.62:0/2397371400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06341023d0 0x7f06341047c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.047+0000 7f0633200640 1 -- 172.21.15.62:0/2397371400 >> 172.21.15.62:0/2397371400 conn(0x7f06340fbde0 msgr2=0x7f06340fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.047+0000 7f0633200640 1 -- 172.21.15.62:0/2397371400 shutdown_connections 2024-04-02T14:43:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.047+0000 7f0633200640 1 -- 172.21.15.62:0/2397371400 wait complete. 2024-04-02T14:43:45.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0633200640 1 Processor -- start 2024-04-02T14:43:45.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0633200640 1 -- start start 2024-04-02T14:43:45.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0633200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06341023d0 0x7f0634199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:45.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0633200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0634104d00 0x7f063419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0633200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0634107630 0x7f06341a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0633200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f063410d6b0 con 0x7f06341023d0 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0633200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f063410d530 con 0x7f0634104d00 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0633200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f063410d830 con 0x7f0634107630 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0631e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06341023d0 0x7f0634199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0631e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06341023d0 0x7f0634199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46450/0 (socket says 172.21.15.62:46450) 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0631e00640 1 -- 172.21.15.62:0/2642785271 learned_addr learned my addr 172.21.15.62:0/2642785271 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.048+0000 7f0631400640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0634104d00 0x7f063419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.049+0000 7f0632800640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0634107630 0x7f06341a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.049+0000 7f0631e00640 1 -- 172.21.15.62:0/2642785271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0634107630 msgr2=0x7f06341a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.049+0000 7f0631e00640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0634107630 0x7f06341a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.049+0000 7f0631e00640 1 -- 172.21.15.62:0/2642785271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0634104d00 msgr2=0x7f063419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.049+0000 7f0631e00640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0634104d00 0x7f063419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.049+0000 7f0631e00640 1 -- 172.21.15.62:0/2642785271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06341a1ba0 con 0x7f06341023d0 2024-04-02T14:43:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.049+0000 7f0631e00640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06341023d0 0x7f0634199f80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0624009230 tx=0x7f0624004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.049+0000 7f062aa00640 1 -- 172.21.15.62:0/2642785271 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624043020 con 0x7f06341023d0 2024-04-02T14:43:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.050+0000 7f062aa00640 1 -- 172.21.15.62:0/2642785271 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0624004250 con 0x7f06341023d0 2024-04-02T14:43:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.050+0000 7f0633200640 1 -- 172.21.15.62:0/2642785271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06341a1e30 con 0x7f06341023d0 2024-04-02T14:43:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.050+0000 7f062aa00640 1 -- 172.21.15.62:0/2642785271 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624003b50 con 0x7f06341023d0 2024-04-02T14:43:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.050+0000 7f0633200640 1 -- 172.21.15.62:0/2642785271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06341a22f0 con 0x7f06341023d0 2024-04-02T14:43:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.051+0000 7f0633200640 1 -- 172.21.15.62:0/2642785271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0634109180 con 0x7f06341023d0 2024-04-02T14:43:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.053+0000 7f062aa00640 1 -- 172.21.15.62:0/2642785271 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0624046020 con 0x7f06341023d0 2024-04-02T14:43:45.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.054+0000 7f062aa00640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06180a3c30 0x7f06180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:45.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.054+0000 7f062aa00640 1 -- 172.21.15.62:0/2642785271 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f06240f2420 con 0x7f06341023d0 2024-04-02T14:43:45.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.054+0000 7f0631400640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06180a3c30 0x7f06180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:45.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.055+0000 7f0631400640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06180a3c30 0x7f06180a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f063419b490 tx=0x7f0620005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:45.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.058+0000 7f062aa00640 1 -- 172.21.15.62:0/2642785271 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f062404d020 con 0x7f06341023d0 2024-04-02T14:43:45.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.308+0000 7f0633200640 1 -- 172.21.15.62:0/2642785271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06340627a0 con 0x7f06341023d0 2024-04-02T14:43:45.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.310+0000 7f062aa00640 1 -- 172.21.15.62:0/2642785271 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f06240bee30 con 0x7f06341023d0 2024-04-02T14:43:45.307 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:45.307 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:41:30.090009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24553,24565,24533,24535,24584]},"id":1}]} 2024-04-02T14:43:45.307 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-02T14:43:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.312+0000 7f0633200640 1 -- 172.21.15.62:0/2642785271 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06180a3c30 msgr2=0x7f06180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.312+0000 7f0633200640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06180a3c30 0x7f06180a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f063419b490 tx=0x7f0620005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.312+0000 7f0633200640 1 -- 172.21.15.62:0/2642785271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06341023d0 msgr2=0x7f0634199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.312+0000 7f0633200640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06341023d0 0x7f0634199f80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0624009230 tx=0x7f0624004ab0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.313+0000 7f0633200640 1 -- 172.21.15.62:0/2642785271 shutdown_connections 2024-04-02T14:43:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.313+0000 7f0633200640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f06180a3c30 0x7f06180a60f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.313+0000 7f0633200640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0634107630 0x7f06341a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.313+0000 7f0633200640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0634104d00 0x7f063419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.313+0000 7f0633200640 1 --2- 172.21.15.62:0/2642785271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06341023d0 0x7f0634199f80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.313+0000 7f0633200640 1 -- 172.21.15.62:0/2642785271 >> 172.21.15.62:0/2642785271 conn(0x7f06340fbde0 msgr2=0x7f0634101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.313+0000 7f0633200640 1 -- 172.21.15.62:0/2642785271 shutdown_connections 2024-04-02T14:43:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.313+0000 7f0633200640 1 -- 172.21.15.62:0/2642785271 wait complete. 2024-04-02T14:43:45.331 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:43:45.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.567+0000 7fa22b600640 1 -- 172.21.15.62:0/2940641372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224101b20 msgr2=0x7fa224101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:45.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.567+0000 7fa22b600640 1 --2- 172.21.15.62:0/2940641372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224101b20 0x7fa224101f40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa214009ca0 tx=0x7fa21402c100 comp rx=0 tx=0).stop 2024-04-02T14:43:45.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.567+0000 7fa22b600640 1 -- 172.21.15.62:0/2940641372 shutdown_connections 2024-04-02T14:43:45.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.567+0000 7fa22b600640 1 --2- 172.21.15.62:0/2940641372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2241036d0 0x7fa224109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.567+0000 7fa22b600640 1 --2- 172.21.15.62:0/2940641372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa224102d10 0x7fa224103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.567+0000 7fa22b600640 1 --2- 172.21.15.62:0/2940641372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224101b20 0x7fa224101f40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.567+0000 7fa22b600640 1 -- 172.21.15.62:0/2940641372 >> 172.21.15.62:0/2940641372 conn(0x7fa2240fd0b0 msgr2=0x7fa2240ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:45.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.567+0000 7fa22b600640 1 -- 172.21.15.62:0/2940641372 shutdown_connections 2024-04-02T14:43:45.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.567+0000 7fa22b600640 1 -- 172.21.15.62:0/2940641372 wait complete. 2024-04-02T14:43:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.568+0000 7fa22b600640 1 Processor -- start 2024-04-02T14:43:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa22b600640 1 -- start start 2024-04-02T14:43:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa22b600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224101b20 0x7fa22419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa22b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa224102d10 0x7fa22419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa22b600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2241036d0 0x7fa2241a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa22b600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa22410d8d0 con 0x7fa224101b20 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa22b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa22410d750 con 0x7fa224102d10 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa22b600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa22410da50 con 0x7fa2241036d0 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa228c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224101b20 0x7fa22419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa228c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224101b20 0x7fa22419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46470/0 (socket says 172.21.15.62:46470) 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa228c00640 1 -- 172.21.15.62:0/703694482 learned_addr learned my addr 172.21.15.62:0/703694482 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa229600640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2241036d0 0x7fa2241a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.569+0000 7fa223e00640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa224102d10 0x7fa22419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa228c00640 1 -- 172.21.15.62:0/703694482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2241036d0 msgr2=0x7fa2241a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa228c00640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2241036d0 0x7fa2241a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa228c00640 1 -- 172.21.15.62:0/703694482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa224102d10 msgr2=0x7fa22419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa228c00640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa224102d10 0x7fa22419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa228c00640 1 -- 172.21.15.62:0/703694482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2241a1d70 con 0x7fa224101b20 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa229600640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2241036d0 0x7fa2241a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa228c00640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224101b20 0x7fa22419a150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa214000f80 tx=0x7fa214034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa221600640 1 -- 172.21.15.62:0/703694482 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214039040 con 0x7fa224101b20 2024-04-02T14:43:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa221600640 1 -- 172.21.15.62:0/703694482 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa214044070 con 0x7fa224101b20 2024-04-02T14:43:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa22b600640 1 -- 172.21.15.62:0/703694482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2241a2000 con 0x7fa224101b20 2024-04-02T14:43:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa221600640 1 -- 172.21.15.62:0/703694482 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214004500 con 0x7fa224101b20 2024-04-02T14:43:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.570+0000 7fa22b600640 1 -- 172.21.15.62:0/703694482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2241a24c0 con 0x7fa224101b20 2024-04-02T14:43:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.571+0000 7fa22b600640 1 -- 172.21.15.62:0/703694482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2241a27e0 con 0x7fa224101b20 2024-04-02T14:43:45.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.573+0000 7fa221600640 1 -- 172.21.15.62:0/703694482 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa2140046a0 con 0x7fa224101b20 2024-04-02T14:43:45.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.575+0000 7fa221600640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2080a3bb0 0x7fa2080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:45.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.575+0000 7fa221600640 1 -- 172.21.15.62:0/703694482 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fa2140f2a00 con 0x7fa224101b20 2024-04-02T14:43:45.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.575+0000 7fa223e00640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2080a3bb0 0x7fa2080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:45.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.576+0000 7fa223e00640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2080a3bb0 0x7fa2080a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa22419b660 tx=0x7fa20c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:45.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.579+0000 7fa221600640 1 -- 172.21.15.62:0/703694482 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa2140bf3b0 con 0x7fa224101b20 2024-04-02T14:43:45.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:45 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/199372762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:45.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:45 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2642785271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:45.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:45 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/199372762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:45.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:45 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2642785271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:45.769 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:45 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/199372762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:45.769 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:45 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2642785271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.789+0000 7fa22b600640 1 -- 172.21.15.62:0/703694482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2241a29b0 con 0x7fa224101b20 2024-04-02T14:43:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.793+0000 7fa221600640 1 -- 172.21.15.62:0/703694482 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22583 (secure 0 0 0) 0x7fa2140becb0 con 0x7fa224101b20 2024-04-02T14:43:45.790 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:45.790 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:42:21.228892+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 -- 172.21.15.62:0/703694482 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2080a3bb0 msgr2=0x7fa2080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2080a3bb0 0x7fa2080a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa22419b660 tx=0x7fa20c005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 -- 172.21.15.62:0/703694482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224101b20 msgr2=0x7fa22419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224101b20 0x7fa22419a150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa214000f80 tx=0x7fa214034670 comp rx=0 tx=0).stop 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 -- 172.21.15.62:0/703694482 shutdown_connections 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fa2080a3bb0 0x7fa2080a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2241036d0 0x7fa2241a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa224102d10 0x7fa22419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 --2- 172.21.15.62:0/703694482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224101b20 0x7fa22419a150 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 -- 172.21.15.62:0/703694482 >> 172.21.15.62:0/703694482 conn(0x7fa2240fd0b0 msgr2=0x7fa224073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 -- 172.21.15.62:0/703694482 shutdown_connections 2024-04-02T14:43:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:45.796+0000 7fa22b600640 1 -- 172.21.15.62:0/703694482 wait complete. 2024-04-02T14:43:45.814 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T14:41:20.030393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T14:41:20.917405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:43:45.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:43:46.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.049+0000 7f6168e00640 1 -- 172.21.15.62:0/321486920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640780a0 msgr2=0x7f6164078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:46.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.049+0000 7f6168e00640 1 --2- 172.21.15.62:0/321486920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640780a0 0x7f6164078940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f615c009ca0 tx=0x7f615c02c0f0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.049+0000 7f6168e00640 1 -- 172.21.15.62:0/321486920 shutdown_connections 2024-04-02T14:43:46.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.049+0000 7f6168e00640 1 --2- 172.21.15.62:0/321486920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640780a0 0x7f6164078940 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.049+0000 7f6168e00640 1 --2- 172.21.15.62:0/321486920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6164077700 0x7f6164077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.050+0000 7f6168e00640 1 --2- 172.21.15.62:0/321486920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61640764c0 0x7f61640768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.050+0000 7f6168e00640 1 -- 172.21.15.62:0/321486920 >> 172.21.15.62:0/321486920 conn(0x7f61640ff2b0 msgr2=0x7f61641016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:46.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.050+0000 7f6168e00640 1 -- 172.21.15.62:0/321486920 shutdown_connections 2024-04-02T14:43:46.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.050+0000 7f6168e00640 1 -- 172.21.15.62:0/321486920 wait complete. 2024-04-02T14:43:46.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6168e00640 1 Processor -- start 2024-04-02T14:43:46.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6168e00640 1 -- start start 2024-04-02T14:43:46.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6168e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61640764c0 0x7f616419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:46.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6168e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6164077700 0x7f616419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:46.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6168e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640780a0 0x7f61641a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6168e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6164113ed0 con 0x7f61640780a0 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6168e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6164113d50 con 0x7f6164077700 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6168e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6164114050 con 0x7f61640764c0 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6161e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61640764c0 0x7f616419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6159400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6164077700 0x7f616419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6161e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61640764c0 0x7f616419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35422/0 (socket says 172.21.15.62:35422) 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6162800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640780a0 0x7f61641a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.051+0000 7f6161e00640 1 -- 172.21.15.62:0/455343104 learned_addr learned my addr 172.21.15.62:0/455343104 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.052+0000 7f6159400640 1 -- 172.21.15.62:0/455343104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61640764c0 msgr2=0x7f616419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.052+0000 7f6159400640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61640764c0 0x7f616419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.052+0000 7f6159400640 1 -- 172.21.15.62:0/455343104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640780a0 msgr2=0x7f61641a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.052+0000 7f6159400640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640780a0 0x7f61641a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.052+0000 7f6159400640 1 -- 172.21.15.62:0/455343104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61641a6070 con 0x7f6164077700 2024-04-02T14:43:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.052+0000 7f6159400640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6164077700 0x7f616419e990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f614c00e9f0 tx=0x7f614c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.052+0000 7f615b400640 1 -- 172.21.15.62:0/455343104 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c00ccc0 con 0x7f6164077700 2024-04-02T14:43:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.052+0000 7f615b400640 1 -- 172.21.15.62:0/455343104 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f614c005020 con 0x7f6164077700 2024-04-02T14:43:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.052+0000 7f6168e00640 1 -- 172.21.15.62:0/455343104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61641a6360 con 0x7f6164077700 2024-04-02T14:43:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.052+0000 7f615b400640 1 -- 172.21.15.62:0/455343104 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c004b40 con 0x7f6164077700 2024-04-02T14:43:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.052+0000 7f6168e00640 1 -- 172.21.15.62:0/455343104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61641a6860 con 0x7f6164077700 2024-04-02T14:43:46.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.054+0000 7f6168e00640 1 -- 172.21.15.62:0/455343104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6164110690 con 0x7f6164077700 2024-04-02T14:43:46.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.056+0000 7f615b400640 1 -- 172.21.15.62:0/455343104 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f614c004ce0 con 0x7f6164077700 2024-04-02T14:43:46.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.057+0000 7f615b400640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f61480a3bb0 0x7f61480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:46.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.057+0000 7f615b400640 1 -- 172.21.15.62:0/455343104 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f614c0d03c0 con 0x7f6164077700 2024-04-02T14:43:46.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.057+0000 7f6161e00640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f61480a3bb0 0x7f61480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:46.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.058+0000 7f6161e00640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f61480a3bb0 0x7f61480a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f61540095f0 tx=0x7f6154005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:46.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.061+0000 7f615b400640 1 -- 172.21.15.62:0/455343104 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f614c09cd70 con 0x7f6164077700 2024-04-02T14:43:46.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.268+0000 7f6168e00640 1 -- 172.21.15.62:0/455343104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6164062700 con 0x7f6164077700 2024-04-02T14:43:46.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.268+0000 7f615b400640 1 -- 172.21.15.62:0/455343104 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f614c09c670 con 0x7f6164077700 2024-04-02T14:43:46.265 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:43:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 -- 172.21.15.62:0/455343104 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f61480a3bb0 msgr2=0x7f61480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f61480a3bb0 0x7f61480a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f61540095f0 tx=0x7f6154005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 -- 172.21.15.62:0/455343104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6164077700 msgr2=0x7f616419e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6164077700 0x7f616419e990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f614c00e9f0 tx=0x7f614c00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 -- 172.21.15.62:0/455343104 shutdown_connections 2024-04-02T14:43:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f61480a3bb0 0x7f61480a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61640780a0 0x7f61641a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6164077700 0x7f616419e990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 --2- 172.21.15.62:0/455343104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61640764c0 0x7f616419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 -- 172.21.15.62:0/455343104 >> 172.21.15.62:0/455343104 conn(0x7f61640ff2b0 msgr2=0x7f6164100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 -- 172.21.15.62:0/455343104 shutdown_connections 2024-04-02T14:43:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.271+0000 7f6168e00640 1 -- 172.21.15.62:0/455343104 wait complete. 2024-04-02T14:43:46.287 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T14:43:46.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.520+0000 7fb638200640 1 -- 172.21.15.62:0/1136720836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb630104ed0 msgr2=0x7fb6301072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:46.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.520+0000 7fb638200640 1 --2- 172.21.15.62:0/1136720836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb630104ed0 0x7fb6301072c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009900 tx=0x7fb61c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:46.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.520+0000 7fb638200640 1 -- 172.21.15.62:0/1136720836 shutdown_connections 2024-04-02T14:43:46.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.520+0000 7fb638200640 1 --2- 172.21.15.62:0/1136720836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb630107800 0x7fb630109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.520+0000 7fb638200640 1 --2- 172.21.15.62:0/1136720836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb630104ed0 0x7fb6301072c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.520+0000 7fb638200640 1 --2- 172.21.15.62:0/1136720836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6301025a0 0x7fb630104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.520+0000 7fb638200640 1 -- 172.21.15.62:0/1136720836 >> 172.21.15.62:0/1136720836 conn(0x7fb6300fbf90 msgr2=0x7fb6300fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:46.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.520+0000 7fb638200640 1 -- 172.21.15.62:0/1136720836 shutdown_connections 2024-04-02T14:43:46.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.520+0000 7fb638200640 1 -- 172.21.15.62:0/1136720836 wait complete. 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb638200640 1 Processor -- start 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb638200640 1 -- start start 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb638200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6301025a0 0x7fb630197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb638200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb630104ed0 0x7fb6301984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb638200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb630107800 0x7fb63019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb638200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb63010d880 con 0x7fb6301025a0 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb638200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb63010d700 con 0x7fb630107800 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb638200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb63010da00 con 0x7fb630104ed0 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb635800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6301025a0 0x7fb630197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb635800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6301025a0 0x7fb630197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46522/0 (socket says 172.21.15.62:46522) 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb635800640 1 -- 172.21.15.62:0/2667926899 learned_addr learned my addr 172.21.15.62:0/2667926899 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb634e00640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb630104ed0 0x7fb6301984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.522+0000 7fb636200640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb630107800 0x7fb63019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:46.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.523+0000 7fb634e00640 1 -- 172.21.15.62:0/2667926899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb630107800 msgr2=0x7fb63019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:46.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.523+0000 7fb634e00640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb630107800 0x7fb63019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.523+0000 7fb634e00640 1 -- 172.21.15.62:0/2667926899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6301025a0 msgr2=0x7fb630197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:46.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.523+0000 7fb634e00640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6301025a0 0x7fb630197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.523+0000 7fb634e00640 1 -- 172.21.15.62:0/2667926899 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63019fbd0 con 0x7fb630104ed0 2024-04-02T14:43:46.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.523+0000 7fb635800640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6301025a0 0x7fb630197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:46.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.523+0000 7fb634e00640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb630104ed0 0x7fb6301984f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb61c02c170 tx=0x7fb61c004960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:46.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.524+0000 7fb62e000640 1 -- 172.21.15.62:0/2667926899 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c043020 con 0x7fb630104ed0 2024-04-02T14:43:46.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.524+0000 7fb62e000640 1 -- 172.21.15.62:0/2667926899 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb61c004250 con 0x7fb630104ed0 2024-04-02T14:43:46.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.524+0000 7fb62e000640 1 -- 172.21.15.62:0/2667926899 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c003af0 con 0x7fb630104ed0 2024-04-02T14:43:46.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.524+0000 7fb638200640 1 -- 172.21.15.62:0/2667926899 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63019fe60 con 0x7fb630104ed0 2024-04-02T14:43:46.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.524+0000 7fb638200640 1 -- 172.21.15.62:0/2667926899 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6301a0270 con 0x7fb630104ed0 2024-04-02T14:43:46.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.525+0000 7fb638200640 1 -- 172.21.15.62:0/2667926899 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6300fda80 con 0x7fb630104ed0 2024-04-02T14:43:46.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.527+0000 7fb62e000640 1 -- 172.21.15.62:0/2667926899 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb61c046020 con 0x7fb630104ed0 2024-04-02T14:43:46.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.528+0000 7fb62e000640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb6180a3bb0 0x7fb6180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:46.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.529+0000 7fb62e000640 1 -- 172.21.15.62:0/2667926899 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fb61c0f22e0 con 0x7fb630104ed0 2024-04-02T14:43:46.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.529+0000 7fb635800640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb6180a3bb0 0x7fb6180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:46.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.530+0000 7fb635800640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb6180a3bb0 0x7fb6180a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb6240095f0 tx=0x7fb624005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:46.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.533+0000 7fb62e000640 1 -- 172.21.15.62:0/2667926899 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb61c04d020 con 0x7fb630104ed0 2024-04-02T14:43:46.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:46 smithi062 ceph-mon[32116]: pgmap v393: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:46.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:46 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/703694482' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:46.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:46 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/455343104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:46.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.750+0000 7fb638200640 1 -- 172.21.15.62:0/2667926899 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb630062730 con 0x7fb630104ed0 2024-04-02T14:43:46.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.752+0000 7fb62e000640 1 -- 172.21.15.62:0/2667926899 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fb61c0bec90 con 0x7fb630104ed0 2024-04-02T14:43:46.749 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-02T14:43:46.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.754+0000 7fb638200640 1 -- 172.21.15.62:0/2667926899 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb6180a3bb0 msgr2=0x7fb6180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:46.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.754+0000 7fb638200640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb6180a3bb0 0x7fb6180a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb6240095f0 tx=0x7fb624005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:46.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.754+0000 7fb638200640 1 -- 172.21.15.62:0/2667926899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb630104ed0 msgr2=0x7fb6301984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:46.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.754+0000 7fb638200640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb630104ed0 0x7fb6301984f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb61c02c170 tx=0x7fb61c004960 comp rx=0 tx=0).stop 2024-04-02T14:43:46.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.754+0000 7fb638200640 1 -- 172.21.15.62:0/2667926899 shutdown_connections 2024-04-02T14:43:46.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.754+0000 7fb638200640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb6180a3bb0 0x7fb6180a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.754+0000 7fb638200640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb630107800 0x7fb63019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.754+0000 7fb638200640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb630104ed0 0x7fb6301984f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.754+0000 7fb638200640 1 --2- 172.21.15.62:0/2667926899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6301025a0 0x7fb630197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.754+0000 7fb638200640 1 -- 172.21.15.62:0/2667926899 >> 172.21.15.62:0/2667926899 conn(0x7fb6300fbf90 msgr2=0x7fb630104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:46.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.755+0000 7fb638200640 1 -- 172.21.15.62:0/2667926899 shutdown_connections 2024-04-02T14:43:46.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:46.755+0000 7fb638200640 1 -- 172.21.15.62:0/2667926899 wait complete. 2024-04-02T14:43:46.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:46 smithi204 ceph-mon[35948]: pgmap v393: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:46.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:46 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/703694482' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:46.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:46 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/455343104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:46.770 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T14:43:46.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:46 smithi113 ceph-mon[36135]: pgmap v393: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:43:46.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:46 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/703694482' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:46.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:46 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/455343104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:43:46.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.001+0000 7f95ab000640 1 -- 172.21.15.62:0/1108706400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95a4109bd0 msgr2=0x7f95a4110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:46.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.001+0000 7f95ab000640 1 --2- 172.21.15.62:0/1108706400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95a4109bd0 0x7f95a4110460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f959c009be0 tx=0x7f959c02bef0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.002+0000 7f95ab000640 1 -- 172.21.15.62:0/1108706400 shutdown_connections 2024-04-02T14:43:46.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.002+0000 7f95ab000640 1 --2- 172.21.15.62:0/1108706400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95a4109bd0 0x7f95a4110460 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.002+0000 7f95ab000640 1 --2- 172.21.15.62:0/1108706400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4109210 0x7f95a4109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:46.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.002+0000 7f95ab000640 1 --2- 172.21.15.62:0/1108706400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95a4108020 0x7f95a4108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.002+0000 7f95ab000640 1 -- 172.21.15.62:0/1108706400 >> 172.21.15.62:0/1108706400 conn(0x7f95a4076fd0 msgr2=0x7f95a40793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:47.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.002+0000 7f95ab000640 1 -- 172.21.15.62:0/1108706400 shutdown_connections 2024-04-02T14:43:47.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.002+0000 7f95ab000640 1 -- 172.21.15.62:0/1108706400 wait complete. 2024-04-02T14:43:47.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.003+0000 7f95ab000640 1 Processor -- start 2024-04-02T14:43:47.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.003+0000 7f95ab000640 1 -- start start 2024-04-02T14:43:47.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.003+0000 7f95ab000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95a4108020 0x7f95a419e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.003+0000 7f95ab000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4109210 0x7f95a419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.003+0000 7f95ab000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95a4109bd0 0x7f95a41a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.003+0000 7f95ab000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4113dd0 con 0x7f95a4108020 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.003+0000 7f95ab000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4113c50 con 0x7f95a4109bd0 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.003+0000 7f95ab000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4113f50 con 0x7f95a4109210 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.004+0000 7f95a9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95a4108020 0x7f95a419e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.004+0000 7f95a9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95a4108020 0x7f95a419e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46548/0 (socket says 172.21.15.62:46548) 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.004+0000 7f95a9c00640 1 -- 172.21.15.62:0/2320323323 learned_addr learned my addr 172.21.15.62:0/2320323323 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.004+0000 7f95aa600640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95a4109bd0 0x7f95a41a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.004+0000 7f95a9200640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4109210 0x7f95a419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.004+0000 7f95a9c00640 1 -- 172.21.15.62:0/2320323323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4109210 msgr2=0x7f95a419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.004+0000 7f95a9c00640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4109210 0x7f95a419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.004+0000 7f95a9c00640 1 -- 172.21.15.62:0/2320323323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95a4109bd0 msgr2=0x7f95a41a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.004+0000 7f95a9c00640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95a4109bd0 0x7f95a41a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.004+0000 7f95a9c00640 1 -- 172.21.15.62:0/2320323323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a41a5f80 con 0x7f95a4108020 2024-04-02T14:43:47.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.005+0000 7f95a9c00640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95a4108020 0x7f95a419e360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f959800e9c0 tx=0x7f959800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:47.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.005+0000 7f95a2000640 1 -- 172.21.15.62:0/2320323323 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959800ccc0 con 0x7f95a4108020 2024-04-02T14:43:47.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.005+0000 7f95a2000640 1 -- 172.21.15.62:0/2320323323 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f95980035f0 con 0x7f95a4108020 2024-04-02T14:43:47.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.005+0000 7f95ab000640 1 -- 172.21.15.62:0/2320323323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41a6270 con 0x7f95a4108020 2024-04-02T14:43:47.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.005+0000 7f95a2000640 1 -- 172.21.15.62:0/2320323323 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959800ce60 con 0x7f95a4108020 2024-04-02T14:43:47.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.005+0000 7f95ab000640 1 -- 172.21.15.62:0/2320323323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a41a67c0 con 0x7f95a4108020 2024-04-02T14:43:47.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.006+0000 7f95ab000640 1 -- 172.21.15.62:0/2320323323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a410f340 con 0x7f95a4108020 2024-04-02T14:43:47.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.008+0000 7f95a2000640 1 -- 172.21.15.62:0/2320323323 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f95980037a0 con 0x7f95a4108020 2024-04-02T14:43:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.010+0000 7f95a2000640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f958c0a3bb0 0x7f958c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.010+0000 7f95a2000640 1 -- 172.21.15.62:0/2320323323 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f95980d07f0 con 0x7f95a4108020 2024-04-02T14:43:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.010+0000 7f95a9200640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f958c0a3bb0 0x7f958c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.011+0000 7f95a9200640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f958c0a3bb0 0x7f958c0a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95a419f870 tx=0x7f9590009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:47.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.014+0000 7f95a2000640 1 -- 172.21.15.62:0/2320323323 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f959809d1a0 con 0x7f95a4108020 2024-04-02T14:43:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.220+0000 7f95ab000640 1 -- 172.21.15.62:0/2320323323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f95a4062700 con 0x7f95a4108020 2024-04-02T14:43:47.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.221+0000 7f95a2000640 1 -- 172.21.15.62:0/2320323323 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f959809caa0 con 0x7f95a4108020 2024-04-02T14:43:47.218 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-02T14:43:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 -- 172.21.15.62:0/2320323323 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f958c0a3bb0 msgr2=0x7f958c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f958c0a3bb0 0x7f958c0a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95a419f870 tx=0x7f9590009260 comp rx=0 tx=0).stop 2024-04-02T14:43:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 -- 172.21.15.62:0/2320323323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95a4108020 msgr2=0x7f95a419e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95a4108020 0x7f95a419e360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f959800e9c0 tx=0x7f959800edc0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 -- 172.21.15.62:0/2320323323 shutdown_connections 2024-04-02T14:43:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f958c0a3bb0 0x7f958c0a6070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95a4109bd0 0x7f95a41a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4109210 0x7f95a419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 --2- 172.21.15.62:0/2320323323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95a4108020 0x7f95a419e360 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 -- 172.21.15.62:0/2320323323 >> 172.21.15.62:0/2320323323 conn(0x7f95a4076fd0 msgr2=0x7f95a410c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 -- 172.21.15.62:0/2320323323 shutdown_connections 2024-04-02T14:43:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.224+0000 7f95ab000640 1 -- 172.21.15.62:0/2320323323 wait complete. 2024-04-02T14:43:47.242 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:43:47.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.470+0000 7f922f200640 1 -- 172.21.15.62:0/2518190255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230103b50 msgr2=0x7f9230103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.470+0000 7f922f200640 1 --2- 172.21.15.62:0/2518190255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230103b50 0x7f9230103f70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9220009900 tx=0x7f922002bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:47.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.470+0000 7f922f200640 1 -- 172.21.15.62:0/2518190255 shutdown_connections 2024-04-02T14:43:47.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.470+0000 7f922f200640 1 --2- 172.21.15.62:0/2518190255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230105700 0x7f923010bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.470+0000 7f922f200640 1 --2- 172.21.15.62:0/2518190255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9230104d40 0x7f92301051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.470+0000 7f922f200640 1 --2- 172.21.15.62:0/2518190255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230103b50 0x7f9230103f70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.470+0000 7f922f200640 1 -- 172.21.15.62:0/2518190255 >> 172.21.15.62:0/2518190255 conn(0x7f92300ff0e0 msgr2=0x7f9230101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:47.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.470+0000 7f922f200640 1 -- 172.21.15.62:0/2518190255 shutdown_connections 2024-04-02T14:43:47.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.470+0000 7f922f200640 1 -- 172.21.15.62:0/2518190255 wait complete. 2024-04-02T14:43:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.472+0000 7f922f200640 1 Processor -- start 2024-04-02T14:43:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.472+0000 7f922f200640 1 -- start start 2024-04-02T14:43:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.472+0000 7f922f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230103b50 0x7f9230199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.472+0000 7f922f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230104d40 0x7f923019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.472+0000 7f922f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9230105700 0x7f92301a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.472+0000 7f922f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f923010f8b0 con 0x7f9230105700 2024-04-02T14:43:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.472+0000 7f922f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f923010f730 con 0x7f9230103b50 2024-04-02T14:43:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.472+0000 7f922f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f923010fa30 con 0x7f9230104d40 2024-04-02T14:43:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230104d40 0x7f923019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230103b50 0x7f9230199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230104d40 0x7f923019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35468/0 (socket says 172.21.15.62:35468) 2024-04-02T14:43:47.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922d400640 1 -- 172.21.15.62:0/2051376449 learned_addr learned my addr 172.21.15.62:0/2051376449 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:47.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922e800640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9230105700 0x7f92301a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922d400640 1 -- 172.21.15.62:0/2051376449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230103b50 msgr2=0x7f9230199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922d400640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230103b50 0x7f9230199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922d400640 1 -- 172.21.15.62:0/2051376449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9230105700 msgr2=0x7f92301a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922d400640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9230105700 0x7f92301a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922d400640 1 -- 172.21.15.62:0/2051376449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92301a1b70 con 0x7f9230104d40 2024-04-02T14:43:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922de00640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230103b50 0x7f9230199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.473+0000 7f922d400640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230104d40 0x7f923019a490 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f921c00ea60 tx=0x7f921c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.474+0000 7f9226a00640 1 -- 172.21.15.62:0/2051376449 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c00cde0 con 0x7f9230104d40 2024-04-02T14:43:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.474+0000 7f9226a00640 1 -- 172.21.15.62:0/2051376449 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f921c0035f0 con 0x7f9230104d40 2024-04-02T14:43:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.474+0000 7f922f200640 1 -- 172.21.15.62:0/2051376449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92301a1e60 con 0x7f9230104d40 2024-04-02T14:43:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.474+0000 7f9226a00640 1 -- 172.21.15.62:0/2051376449 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c010720 con 0x7f9230104d40 2024-04-02T14:43:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.474+0000 7f922f200640 1 -- 172.21.15.62:0/2051376449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92301a2370 con 0x7f9230104d40 2024-04-02T14:43:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.475+0000 7f922f200640 1 -- 172.21.15.62:0/2051376449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923010ae70 con 0x7f9230104d40 2024-04-02T14:43:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.478+0000 7f9226a00640 1 -- 172.21.15.62:0/2051376449 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f921c0037a0 con 0x7f9230104d40 2024-04-02T14:43:47.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.479+0000 7f9226a00640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 0x7f92140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.480+0000 7f9226a00640 1 -- 172.21.15.62:0/2051376449 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f921c0d07e0 con 0x7f9230104d40 2024-04-02T14:43:47.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.480+0000 7f922de00640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 0x7f92140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.480+0000 7f922de00640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 0x7f92140a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9220000c00 tx=0x7f9220005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.481+0000 7f9226a00640 1 -- 172.21.15.62:0/2051376449 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f921c09d220 con 0x7f9230104d40 2024-04-02T14:43:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.730+0000 7f922f200640 1 -- 172.21.15.62:0/2051376449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f923002d5c0 con 0x7f9230104d40 2024-04-02T14:43:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.732+0000 7f9226a00640 1 -- 172.21.15.62:0/2051376449 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f921c09cb20 con 0x7f9230104d40 2024-04-02T14:43:47.729 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:47.729 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:41:30.090009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24553,24565,24533,24535,24584]},"id":1}]} 2024-04-02T14:43:47.729 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-02T14:43:47.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 -- 172.21.15.62:0/2051376449 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 msgr2=0x7f92140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 0x7f92140a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9220000c00 tx=0x7f9220005fd0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 -- 172.21.15.62:0/2051376449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230104d40 msgr2=0x7f923019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230104d40 0x7f923019a490 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f921c00ea60 tx=0x7f921c00ef20 comp rx=0 tx=0).stop 2024-04-02T14:43:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 -- 172.21.15.62:0/2051376449 shutdown_connections 2024-04-02T14:43:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f92140a3c30 0x7f92140a60f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9230105700 0x7f92301a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9230104d40 0x7f923019a490 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 --2- 172.21.15.62:0/2051376449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230103b50 0x7f9230199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 -- 172.21.15.62:0/2051376449 >> 172.21.15.62:0/2051376449 conn(0x7f92300ff0e0 msgr2=0x7f9230108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 -- 172.21.15.62:0/2051376449 shutdown_connections 2024-04-02T14:43:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.735+0000 7f922f200640 1 -- 172.21.15.62:0/2051376449 wait complete. 2024-04-02T14:43:47.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:47 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2667926899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:47.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:47 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2320323323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:47.754 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:43:47.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:47 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2667926899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:47.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:47 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2320323323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:47.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:47 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2667926899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:43:47.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:47 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2320323323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:43:47.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.980+0000 7fb46b200640 1 -- 172.21.15.62:0/1876088561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb46c103b50 msgr2=0x7fb46c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.980+0000 7fb46b200640 1 --2- 172.21.15.62:0/1876088561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb46c103b50 0x7fb46c103f70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009900 tx=0x7fb45c02bc40 comp rx=0 tx=0).stop 2024-04-02T14:43:47.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.980+0000 7fb46b200640 1 -- 172.21.15.62:0/1876088561 shutdown_connections 2024-04-02T14:43:47.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.980+0000 7fb46b200640 1 --2- 172.21.15.62:0/1876088561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c105700 0x7fb46c10bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.980+0000 7fb46b200640 1 --2- 172.21.15.62:0/1876088561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c104d40 0x7fb46c1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.980+0000 7fb46b200640 1 --2- 172.21.15.62:0/1876088561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb46c103b50 0x7fb46c103f70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.980+0000 7fb46b200640 1 -- 172.21.15.62:0/1876088561 >> 172.21.15.62:0/1876088561 conn(0x7fb46c0ff0e0 msgr2=0x7fb46c101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:47.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.980+0000 7fb46b200640 1 -- 172.21.15.62:0/1876088561 shutdown_connections 2024-04-02T14:43:47.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.980+0000 7fb46b200640 1 -- 172.21.15.62:0/1876088561 wait complete. 2024-04-02T14:43:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.981+0000 7fb46b200640 1 Processor -- start 2024-04-02T14:43:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.981+0000 7fb46b200640 1 -- start start 2024-04-02T14:43:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb46b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb46c103b50 0x7fb46c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb46b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c104d40 0x7fb46c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb46b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c105700 0x7fb46c1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb46b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c10f900 con 0x7fb46c103b50 2024-04-02T14:43:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb46b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c10f780 con 0x7fb46c104d40 2024-04-02T14:43:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb46b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c10fa80 con 0x7fb46c105700 2024-04-02T14:43:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb469e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb46c103b50 0x7fb46c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb469e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb46c103b50 0x7fb46c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46590/0 (socket says 172.21.15.62:46590) 2024-04-02T14:43:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb469e00640 1 -- 172.21.15.62:0/2864754531 learned_addr learned my addr 172.21.15.62:0/2864754531 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb461400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c104d40 0x7fb46c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb46a800640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c105700 0x7fb46c1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb469e00640 1 -- 172.21.15.62:0/2864754531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c105700 msgr2=0x7fb46c1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb469e00640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c105700 0x7fb46c1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb469e00640 1 -- 172.21.15.62:0/2864754531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c104d40 msgr2=0x7fb46c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb469e00640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c104d40 0x7fb46c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.982+0000 7fb469e00640 1 -- 172.21.15.62:0/2864754531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c1a1b70 con 0x7fb46c103b50 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.983+0000 7fb46a800640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c105700 0x7fb46c1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.983+0000 7fb469e00640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb46c103b50 0x7fb46c199f50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb45c000c00 tx=0x7fb45c02c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.983+0000 7fb463400640 1 -- 172.21.15.62:0/2864754531 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c043020 con 0x7fb46c103b50 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.983+0000 7fb46b200640 1 -- 172.21.15.62:0/2864754531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c1a1e00 con 0x7fb46c103b50 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.983+0000 7fb463400640 1 -- 172.21.15.62:0/2864754531 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb45c0042d0 con 0x7fb46c103b50 2024-04-02T14:43:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.983+0000 7fb463400640 1 -- 172.21.15.62:0/2864754531 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c003c20 con 0x7fb46c103b50 2024-04-02T14:43:47.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.983+0000 7fb46b200640 1 -- 172.21.15.62:0/2864754531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c1a2260 con 0x7fb46c103b50 2024-04-02T14:43:47.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.985+0000 7fb46b200640 1 -- 172.21.15.62:0/2864754531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46c10ae70 con 0x7fb46c103b50 2024-04-02T14:43:47.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.986+0000 7fb463400640 1 -- 172.21.15.62:0/2864754531 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb45c046020 con 0x7fb46c103b50 2024-04-02T14:43:47.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.989+0000 7fb463400640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb4500a3bb0 0x7fb4500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.989+0000 7fb463400640 1 -- 172.21.15.62:0/2864754531 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fb45c0f2590 con 0x7fb46c103b50 2024-04-02T14:43:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.989+0000 7fb461400640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb4500a3bb0 0x7fb4500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.990+0000 7fb461400640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb4500a3bb0 0x7fb4500a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb4540095f0 tx=0x7fb454005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:47.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:47.993+0000 7fb463400640 1 -- 172.21.15.62:0/2864754531 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb45c04d020 con 0x7fb46c103b50 2024-04-02T14:43:48.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.200+0000 7fb46b200640 1 -- 172.21.15.62:0/2864754531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb46c0627a0 con 0x7fb46c103b50 2024-04-02T14:43:48.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.201+0000 7fb463400640 1 -- 172.21.15.62:0/2864754531 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22583 (secure 0 0 0) 0x7fb45c0bef40 con 0x7fb46c103b50 2024-04-02T14:43:48.199 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:43:48.199 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:42:21.228892+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:43:48.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.204+0000 7fb46b200640 1 -- 172.21.15.62:0/2864754531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb4500a3bb0 msgr2=0x7fb4500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.204+0000 7fb46b200640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb4500a3bb0 0x7fb4500a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb4540095f0 tx=0x7fb454005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:48.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.204+0000 7fb46b200640 1 -- 172.21.15.62:0/2864754531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb46c103b50 msgr2=0x7fb46c199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.204+0000 7fb46b200640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb46c103b50 0x7fb46c199f50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb45c000c00 tx=0x7fb45c02c730 comp rx=0 tx=0).stop 2024-04-02T14:43:48.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.204+0000 7fb46b200640 1 -- 172.21.15.62:0/2864754531 shutdown_connections 2024-04-02T14:43:48.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.204+0000 7fb46b200640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fb4500a3bb0 0x7fb4500a6070 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.204+0000 7fb46b200640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46c105700 0x7fb46c1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.204+0000 7fb46b200640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c104d40 0x7fb46c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.204+0000 7fb46b200640 1 --2- 172.21.15.62:0/2864754531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb46c103b50 0x7fb46c199f50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.204+0000 7fb46b200640 1 -- 172.21.15.62:0/2864754531 >> 172.21.15.62:0/2864754531 conn(0x7fb46c0ff0e0 msgr2=0x7fb46c108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.205+0000 7fb46b200640 1 -- 172.21.15.62:0/2864754531 shutdown_connections 2024-04-02T14:43:48.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.205+0000 7fb46b200640 1 -- 172.21.15.62:0/2864754531 wait complete. 2024-04-02T14:43:48.223 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-02T14:41:22.009309+0000', 'modified': '2024-04-02T14:41:30.090009+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24584, 'mds_1': 24565, 'mds_2': 24553, 'mds_3': 24535, 'mds_4': 24533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24461': {'gid': 24461, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.113:6833/1862883972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6832', 'nonce': 1862883972}, {'type': 'v1', 'addr': '172.21.15.113:6833', 'nonce': 1862883972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'd', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/3805165269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 3805165269}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 3805165269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/3756932584', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 3756932584}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 3756932584}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.113:6835/360641878', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 360641878}, {'type': 'v1', 'addr': '172.21.15.113:6835', 'nonce': 360641878}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/3537379638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 3537379638}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 3537379638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.113:6837/1122486361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 1122486361}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 1122486361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.204:6839/631749651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 631749651}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 631749651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24553': {'gid': 24553, 'name': 'g', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.62:6839/3188264140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 3188264140}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 3188264140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24565': {'gid': 24565, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.62:6841/450176372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6840', 'nonce': 450176372}, {'type': 'v1', 'addr': '172.21.15.62:6841', 'nonce': 450176372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24584': {'gid': 24584, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6839/1791174899', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6838', 'nonce': 1791174899}, {'type': 'v1', 'addr': '172.21.15.113:6839', 'nonce': 1791174899}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24584, 'qdb_cluster': [24553, 24565, 24533, 24535, 24584]} 2024-04-02T14:43:48.223 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-02T14:43:48.223 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-02T14:43:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.453+0000 7fdf24a00640 1 -- 172.21.15.62:0/3565396802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf20109310 msgr2=0x7fdf20109790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.453+0000 7fdf24a00640 1 --2- 172.21.15.62:0/3565396802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf20109310 0x7fdf20109790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009900 tx=0x7fdf0802bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.453+0000 7fdf24a00640 1 -- 172.21.15.62:0/3565396802 shutdown_connections 2024-04-02T14:43:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.453+0000 7fdf24a00640 1 --2- 172.21.15.62:0/3565396802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf20109cd0 0x7fdf20110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.453+0000 7fdf24a00640 1 --2- 172.21.15.62:0/3565396802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf20109310 0x7fdf20109790 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.453+0000 7fdf24a00640 1 --2- 172.21.15.62:0/3565396802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf20108120 0x7fdf20108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.453+0000 7fdf24a00640 1 -- 172.21.15.62:0/3565396802 >> 172.21.15.62:0/3565396802 conn(0x7fdf200770c0 msgr2=0x7fdf20079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.454+0000 7fdf24a00640 1 -- 172.21.15.62:0/3565396802 shutdown_connections 2024-04-02T14:43:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.454+0000 7fdf24a00640 1 -- 172.21.15.62:0/3565396802 wait complete. 2024-04-02T14:43:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.454+0000 7fdf24a00640 1 Processor -- start 2024-04-02T14:43:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf24a00640 1 -- start start 2024-04-02T14:43:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf24a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf20108120 0x7fdf2019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf24a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf20109310 0x7fdf2019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf24a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf20109cd0 0x7fdf201a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf24a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20113e80 con 0x7fdf20108120 2024-04-02T14:43:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf24a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20113d00 con 0x7fdf20109cd0 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf24a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20114000 con 0x7fdf20109310 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf1de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf20108120 0x7fdf2019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf1de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf20108120 0x7fdf2019e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46614/0 (socket says 172.21.15.62:46614) 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf17e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf20109310 0x7fdf2019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf1de00640 1 -- 172.21.15.62:0/35846971 learned_addr learned my addr 172.21.15.62:0/35846971 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.455+0000 7fdf1e800640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf20109cd0 0x7fdf201a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf1de00640 1 -- 172.21.15.62:0/35846971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf20109310 msgr2=0x7fdf2019ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf1de00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf20109310 0x7fdf2019ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf1de00640 1 -- 172.21.15.62:0/35846971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf20109cd0 msgr2=0x7fdf201a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf1de00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf20109cd0 0x7fdf201a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf1de00640 1 -- 172.21.15.62:0/35846971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf201a6110 con 0x7fdf20108120 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf1e800640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf20109cd0 0x7fdf201a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf17e00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf20109310 0x7fdf2019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf1de00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf20108120 0x7fdf2019e4f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e9c0 tx=0x7fdf1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf16a00640 1 -- 172.21.15.62:0/35846971 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000cde0 con 0x7fdf20108120 2024-04-02T14:43:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf24a00640 1 -- 172.21.15.62:0/35846971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf201a6400 con 0x7fdf20108120 2024-04-02T14:43:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf16a00640 1 -- 172.21.15.62:0/35846971 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf10010930 con 0x7fdf20108120 2024-04-02T14:43:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.456+0000 7fdf16a00640 1 -- 172.21.15.62:0/35846971 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10018a80 con 0x7fdf20108120 2024-04-02T14:43:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.457+0000 7fdf24a00640 1 -- 172.21.15.62:0/35846971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf201a6970 con 0x7fdf20108120 2024-04-02T14:43:48.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.460+0000 7fdf277bd740 1 Processor -- start 2024-04-02T14:43:48.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.460+0000 7fdf16a00640 1 -- 172.21.15.62:0/35846971 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdf1001f020 con 0x7fdf20108120 2024-04-02T14:43:48.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.460+0000 7fdf277bd740 1 -- start start 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.460+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504a8f0 0x55ea2504ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.460+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504b2e0 0x55ea2504fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.460+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea250528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.460+0000 7fdf277bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ea24fb0060 con 0x55ea25050450 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.460+0000 7fdf277bd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ea24f58560 con 0x55ea2504a8f0 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.460+0000 7fdf277bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea24fac010 con 0x55ea2504b2e0 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf17e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea250528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf17e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea250528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46618/0 (socket says 172.21.15.62:46618) 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf17e00640 1 -- 172.21.15.62:0/1741423589 learned_addr learned my addr 172.21.15.62:0/1741423589 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf1e800640 1 --2- 172.21.15.62:0/1741423589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504a8f0 0x55ea2504ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf1de00640 1 --2- 172.21.15.62:0/1741423589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504b2e0 0x55ea2504fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf1e800640 1 -- 172.21.15.62:0/1741423589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504b2e0 msgr2=0x55ea2504fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf1e800640 1 --2- 172.21.15.62:0/1741423589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504b2e0 0x55ea2504fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf1e800640 1 -- 172.21.15.62:0/1741423589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 msgr2=0x55ea250528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf1e800640 1 --2- 172.21.15.62:0/1741423589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea250528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf1e800640 1 -- 172.21.15.62:0/1741423589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea24f7afd0 con 0x55ea2504a8f0 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf17e00640 1 --2- 172.21.15.62:0/1741423589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea250528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.461+0000 7fdf1e800640 1 --2- 172.21.15.62:0/1741423589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504a8f0 0x55ea2504ad10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdf18005fd0 tx=0x7fdf18011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a939ba66f515518c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf0f000640 1 -- 172.21.15.62:0/1741423589 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18004d20 con 0x55ea2504a8f0 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf0f000640 1 -- 172.21.15.62:0/1741423589 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf1800f9a0 con 0x55ea2504a8f0 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf0f000640 1 -- 172.21.15.62:0/1741423589 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18004610 con 0x55ea2504a8f0 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf16a00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdf040a3bb0 0x7fdf040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf16a00640 1 -- 172.21.15.62:0/35846971 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fdf100d06e0 con 0x7fdf20108120 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf1de00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdf040a3bb0 0x7fdf040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf277bd740 1 -- 172.21.15.62:0/1741423589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504a8f0 msgr2=0x55ea2504ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf277bd740 1 --2- 172.21.15.62:0/1741423589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504a8f0 0x55ea2504ad10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdf18005fd0 tx=0x7fdf18011bf0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf277bd740 1 -- 172.21.15.62:0/1741423589 shutdown_connections 2024-04-02T14:43:48.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf277bd740 1 --2- 172.21.15.62:0/1741423589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea250528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf277bd740 1 --2- 172.21.15.62:0/1741423589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504b2e0 0x55ea2504fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf277bd740 1 --2- 172.21.15.62:0/1741423589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504a8f0 0x55ea2504ad10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf277bd740 1 -- 172.21.15.62:0/1741423589 >> 172.21.15.62:0/1741423589 conn(0x55ea25045e60 msgr2=0x55ea25046260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.462+0000 7fdf277bd740 1 -- 172.21.15.62:0/1741423589 shutdown_connections 2024-04-02T14:43:48.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.463+0000 7fdf277bd740 1 -- 172.21.15.62:0/1741423589 wait complete. 2024-04-02T14:43:48.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.463+0000 7fdf1de00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdf040a3bb0 0x7fdf040a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdf10098df0 tx=0x7fdf100bd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.463+0000 7fdf277bd740 1 -- start start 2024-04-02T14:43:48.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.463+0000 7fdf277bd740 1 Processor -- start 2024-04-02T14:43:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25054e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea250553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea2505c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf277bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ea25053400 con 0x55ea25050450 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf277bd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ea24f6b920 con 0x55ea2504b2e0 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf277bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea25053580 con 0x55ea2504a8f0 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf277bd740 10 client.0 _ll_register_callbacks cb 0x55ea24f6b310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf1e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25054e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf17e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea250553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf1e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25054e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35522/0 (socket says 172.21.15.62:35522) 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf1e800640 1 -- 172.21.15.62:0/3196149100 learned_addr learned my addr 172.21.15.62:0/3196149100 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf17e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea250553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:32980/0 (socket says 172.21.15.62:32980) 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf1e800640 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea2505c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf17e00640 1 -- 172.21.15.62:0/3196149100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 msgr2=0x55ea25054e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf17e00640 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25054e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf17e00640 1 -- 172.21.15.62:0/3196149100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 msgr2=0x55ea2505c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf17e00640 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea2505c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf17e00640 1 -- 172.21.15.62:0/3196149100 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea24cc42d0 con 0x55ea2504b2e0 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.464+0000 7fdf1e800640 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea2505c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.465+0000 7fdf17e00640 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea250553a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802ca30 tx=0x7fdf080092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aedb2884311802b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.465+0000 7fdf03400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.465+0000 7fdf03400640 1 -- 172.21.15.62:0/3196149100 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08045070 con 0x55ea2504b2e0 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.465+0000 7fdf03400640 1 -- 172.21.15.62:0/3196149100 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fdf08005310 con 0x55ea2504b2e0 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.465+0000 7fdf277bd740 1 -- 172.21.15.62:0/3196149100 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf1800f9a0 con 0x55ea2504b2e0 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.466+0000 7fdf03400640 1 -- 172.21.15.62:0/3196149100 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf08051050 con 0x55ea2504b2e0 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.466+0000 7fdf277bd740 10 client.25190 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.466+0000 7fdf277bd740 1 -- 172.21.15.62:0/3196149100 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea24d8e6b0 con 0x55ea2504b2e0 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.466+0000 7fdf03400640 1 -- 172.21.15.62:0/3196149100 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf1800f9a0 con 0x55ea2504b2e0 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.466+0000 7fdf03400640 1 -- 172.21.15.62:0/3196149100 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdf080038f0 con 0x55ea2504b2e0 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.466+0000 7fdf277bd740 10 client.25190 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.466+0000 7fdf277bd740 10 client.25190 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.467+0000 7fdf277bd740 20 client.25190 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.467+0000 7fdf277bd740 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55ea25061c10 0x55ea25063080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.467+0000 7fdf277bd740 4 client.25190 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.467+0000 7fdf277bd740 1 -- 172.21.15.62:0/3196149100 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea24fbd5a0 con 0x55ea25061c10 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.467+0000 7fdf1e800640 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55ea25061c10 0x55ea25063080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.467+0000 7fdf1e800640 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55ea25061c10 0x55ea25063080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3195fd5595feb160 server_cookie=21c4c423f33f82b2 in_seq=0 out_seq=0 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.467+0000 7fdf03400640 10 client.25190 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf03400640 1 -- 172.21.15.62:0/3196149100 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea24fbd5a0 con 0x55ea25061c10 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf03400640 10 client.25190 handle_command_reply: tid=0 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf277bd740 1 client.25190 shutdown 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf277bd740 1 -- 172.21.15.62:0/3196149100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 msgr2=0x55ea250553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf277bd740 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea250553a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802ca30 tx=0x7fdf080092f0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf277bd740 1 -- 172.21.15.62:0/3196149100 shutdown_connections 2024-04-02T14:43:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf277bd740 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55ea25061c10 0x55ea25063080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf277bd740 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea2505c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf277bd740 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea250553a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf277bd740 1 --2- 172.21.15.62:0/3196149100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25054e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf03400640 0 client.25190 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:43:48.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf277bd740 1 -- 172.21.15.62:0/3196149100 >> 172.21.15.62:0/3196149100 conn(0x55ea25045e60 msgr2=0x55ea25045080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.468+0000 7fdf277bd740 1 -- 172.21.15.62:0/3196149100 shutdown_connections 2024-04-02T14:43:48.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.469+0000 7fdf277bd740 1 -- 172.21.15.62:0/3196149100 wait complete. 2024-04-02T14:43:48.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.469+0000 7fdf277bd740 20 client.25190 trim_cache size 0 max 16384 2024-04-02T14:43:48.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.488+0000 7fdf277bd740 1 Processor -- start 2024-04-02T14:43:48.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf277bd740 1 -- start start 2024-04-02T14:43:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25052840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea2505c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea25054e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf277bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ea250613c0 con 0x55ea25050450 2024-04-02T14:43:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf277bd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ea25061a90 con 0x55ea2504b2e0 2024-04-02T14:43:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf277bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea25062160 con 0x55ea2504a8f0 2024-04-02T14:43:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf17e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea2505c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf1e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25052840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf17e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea2505c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:32994/0 (socket says 172.21.15.62:32994) 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf17e00640 1 -- 172.21.15.62:0/1955049715 learned_addr learned my addr 172.21.15.62:0/1955049715 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.489+0000 7fdf1e800640 1 --2- 172.21.15.62:0/1955049715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea25054e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.490+0000 7fdf17e00640 1 -- 172.21.15.62:0/1955049715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 msgr2=0x55ea25052840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.490+0000 7fdf17e00640 1 --2- 172.21.15.62:0/1955049715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25052840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.490+0000 7fdf17e00640 1 -- 172.21.15.62:0/1955049715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 msgr2=0x55ea25054e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.490+0000 7fdf17e00640 1 --2- 172.21.15.62:0/1955049715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea25054e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.490+0000 7fdf17e00640 1 -- 172.21.15.62:0/1955049715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea25046da0 con 0x55ea2504b2e0 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.490+0000 7fdf1e800640 1 --2- 172.21.15.62:0/1955049715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea25054e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.490+0000 7fdf17e00640 1 --2- 172.21.15.62:0/1955049715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea2505c3a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdf08003eb0 tx=0x7fdf08005a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9891ae9926776cea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.490+0000 7fdf0e600640 1 -- 172.21.15.62:0/1955049715 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0804e030 con 0x55ea2504b2e0 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.490+0000 7fdf0e600640 1 -- 172.21.15.62:0/1955049715 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf080035f0 con 0x55ea2504b2e0 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.490+0000 7fdf0e600640 1 -- 172.21.15.62:0/1955049715 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08045070 con 0x55ea2504b2e0 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.491+0000 7fdf277bd740 1 -- 172.21.15.62:0/1955049715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 msgr2=0x55ea2505c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.491+0000 7fdf277bd740 1 --2- 172.21.15.62:0/1955049715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea2505c3a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdf08003eb0 tx=0x7fdf08005a60 comp rx=0 tx=0).stop 2024-04-02T14:43:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.491+0000 7fdf277bd740 1 -- 172.21.15.62:0/1955049715 shutdown_connections 2024-04-02T14:43:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.491+0000 7fdf277bd740 1 --2- 172.21.15.62:0/1955049715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea25054e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.491+0000 7fdf277bd740 1 --2- 172.21.15.62:0/1955049715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea2505c3a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.491+0000 7fdf277bd740 1 --2- 172.21.15.62:0/1955049715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25052840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.491+0000 7fdf277bd740 1 -- 172.21.15.62:0/1955049715 >> 172.21.15.62:0/1955049715 conn(0x55ea25045e60 msgr2=0x55ea25040de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.491+0000 7fdf277bd740 1 -- 172.21.15.62:0/1955049715 shutdown_connections 2024-04-02T14:43:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.491+0000 7fdf277bd740 1 -- 172.21.15.62:0/1955049715 wait complete. 2024-04-02T14:43:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.491+0000 7fdf277bd740 1 -- start start 2024-04-02T14:43:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.491+0000 7fdf277bd740 1 Processor -- start 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25058420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea25058960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf277bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea2505b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf277bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ea25062f00 con 0x55ea25050450 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf277bd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ea25062830 con 0x55ea2504b2e0 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf277bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea250635d0 con 0x55ea2504a8f0 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf277bd740 10 client.0 _ll_register_callbacks cb 0x7fdf08009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf17e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea25058960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf17e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea25058960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:33006/0 (socket says 172.21.15.62:33006) 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf17e00640 1 -- 172.21.15.62:0/2692578705 learned_addr learned my addr 172.21.15.62:0/2692578705 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf1e800640 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea2505b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.492+0000 7fdf1e800640 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25058420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.493+0000 7fdf17e00640 1 -- 172.21.15.62:0/2692578705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 msgr2=0x55ea25058420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.493+0000 7fdf17e00640 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25058420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.493+0000 7fdf17e00640 1 -- 172.21.15.62:0/2692578705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 msgr2=0x55ea2505b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.493+0000 7fdf17e00640 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea2505b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.493+0000 7fdf17e00640 1 -- 172.21.15.62:0/2692578705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea2505bf20 con 0x55ea2504b2e0 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.493+0000 7fdf1e800640 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25058420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.493+0000 7fdf17e00640 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea25058960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802c900 tx=0x7fdf0802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7b2a69cc4e9ee20 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.493+0000 7fdf03400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.493+0000 7fdf03400640 1 -- 172.21.15.62:0/2692578705 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08059030 con 0x55ea2504b2e0 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.493+0000 7fdf277bd740 1 -- 172.21.15.62:0/2692578705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf080035f0 con 0x55ea2504b2e0 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.493+0000 7fdf03400640 1 -- 172.21.15.62:0/2692578705 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fdf08005520 con 0x55ea2504b2e0 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.494+0000 7fdf03400640 1 -- 172.21.15.62:0/2692578705 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf08045070 con 0x55ea2504b2e0 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.494+0000 7fdf277bd740 10 client.25196 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.494+0000 7fdf277bd740 1 -- 172.21.15.62:0/2692578705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea250c84c0 con 0x55ea2504b2e0 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.494+0000 7fdf03400640 1 -- 172.21.15.62:0/2692578705 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf080035f0 con 0x55ea2504b2e0 2024-04-02T14:43:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.494+0000 7fdf03400640 1 -- 172.21.15.62:0/2692578705 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdf080049a0 con 0x55ea2504b2e0 2024-04-02T14:43:48.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.494+0000 7fdf277bd740 10 client.25196 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:48.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.494+0000 7fdf277bd740 10 client.25196 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T14:43:48.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.494+0000 7fdf277bd740 20 client.25196 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:48.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.494+0000 7fdf277bd740 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55ea250ccd70 0x55ea250cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.494+0000 7fdf277bd740 4 client.25196 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T14:43:48.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.494+0000 7fdf277bd740 1 -- 172.21.15.62:0/2692578705 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "status"}) -- 0x55ea24fb78f0 con 0x55ea250ccd70 2024-04-02T14:43:48.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.495+0000 7fdf1e800640 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55ea250ccd70 0x55ea250cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.495+0000 7fdf1e800640 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55ea250ccd70 0x55ea250cba80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c366bf3c0fb9d3 server_cookie=932e50f13ed15cd3 in_seq=0 out_seq=0 2024-04-02T14:43:48.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.495+0000 7fdf03400640 10 client.25196 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:43:48.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf03400640 1 -- 172.21.15.62:0/2692578705 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ea24fb78f0 con 0x55ea250ccd70 2024-04-02T14:43:48.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf03400640 10 client.25196 handle_command_reply: tid=0 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 1 client.25196 shutdown 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 1 -- 172.21.15.62:0/2692578705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 msgr2=0x55ea25058960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea25058960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802c900 tx=0x7fdf0802bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 1 -- 172.21.15.62:0/2692578705 shutdown_connections 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55ea250ccd70 0x55ea250cba80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea25050450 0x55ea2505b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf03400640 0 client.25196 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea2504b2e0 0x55ea25058960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 1 --2- 172.21.15.62:0/2692578705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2504a8f0 0x55ea25058420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 1 -- 172.21.15.62:0/2692578705 >> 172.21.15.62:0/2692578705 conn(0x55ea25045e60 msgr2=0x55ea25054e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 1 -- 172.21.15.62:0/2692578705 shutdown_connections 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 1 -- 172.21.15.62:0/2692578705 wait complete. 2024-04-02T14:43:48.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.496+0000 7fdf277bd740 20 client.25196 trim_cache size 0 max 16384 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 4, 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24533, 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 139.41379580700001, 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 166.76009700500001 2024-04-02T14:43:48.497 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:43:48.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 -- 172.21.15.62:0/35846971 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdf040a3bb0 msgr2=0x7fdf040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdf040a3bb0 0x7fdf040a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdf10098df0 tx=0x7fdf100bd040 comp rx=0 tx=0).stop 2024-04-02T14:43:48.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 -- 172.21.15.62:0/35846971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf20108120 msgr2=0x7fdf2019e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf20108120 0x7fdf2019e4f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e9c0 tx=0x7fdf1000ee80 comp rx=0 tx=0).stop 2024-04-02T14:43:48.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 -- 172.21.15.62:0/35846971 shutdown_connections 2024-04-02T14:43:48.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fdf040a3bb0 0x7fdf040a6070 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf20109cd0 0x7fdf201a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf20109310 0x7fdf2019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 --2- 172.21.15.62:0/35846971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf20108120 0x7fdf2019e4f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 -- 172.21.15.62:0/35846971 >> 172.21.15.62:0/35846971 conn(0x7fdf200770c0 msgr2=0x7fdf2010c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 -- 172.21.15.62:0/35846971 shutdown_connections 2024-04-02T14:43:48.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.497+0000 7fdf24a00640 1 -- 172.21.15.62:0/35846971 wait complete. 2024-04-02T14:43:48.514 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-02T14:43:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.739+0000 7fc8e6000640 1 -- 172.21.15.62:0/3097773934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e0109210 msgr2=0x7fc8e0109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.739+0000 7fc8e6000640 1 --2- 172.21.15.62:0/3097773934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e0109210 0x7fc8e0109690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009900 tx=0x7fc8cc02bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.739+0000 7fc8e6000640 1 -- 172.21.15.62:0/3097773934 shutdown_connections 2024-04-02T14:43:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.739+0000 7fc8e6000640 1 --2- 172.21.15.62:0/3097773934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0109bd0 0x7fc8e0110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.739+0000 7fc8e6000640 1 --2- 172.21.15.62:0/3097773934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e0109210 0x7fc8e0109690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.739+0000 7fc8e6000640 1 --2- 172.21.15.62:0/3097773934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e0108020 0x7fc8e0108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.739+0000 7fc8e6000640 1 -- 172.21.15.62:0/3097773934 >> 172.21.15.62:0/3097773934 conn(0x7fc8e0076fd0 msgr2=0x7fc8e00793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.739+0000 7fc8e6000640 1 -- 172.21.15.62:0/3097773934 shutdown_connections 2024-04-02T14:43:48.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.739+0000 7fc8e6000640 1 -- 172.21.15.62:0/3097773934 wait complete. 2024-04-02T14:43:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.740+0000 7fc8e6000640 1 Processor -- start 2024-04-02T14:43:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.741+0000 7fc8e6000640 1 -- start start 2024-04-02T14:43:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.741+0000 7fc8e6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e0108020 0x7fc8e019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.741+0000 7fc8e6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e0109210 0x7fc8e019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.741+0000 7fc8e6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0109bd0 0x7fc8e01a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.741+0000 7fc8e6000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0113d80 con 0x7fc8e0109210 2024-04-02T14:43:48.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.741+0000 7fc8e6000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0113c00 con 0x7fc8e0109bd0 2024-04-02T14:43:48.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.741+0000 7fc8e6000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0113f00 con 0x7fc8e0108020 2024-04-02T14:43:48.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.741+0000 7fc8e4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e0108020 0x7fc8e019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.741+0000 7fc8e4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e0108020 0x7fc8e019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35566/0 (socket says 172.21.15.62:35566) 2024-04-02T14:43:48.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.741+0000 7fc8e4c00640 1 -- 172.21.15.62:0/473798522 learned_addr learned my addr 172.21.15.62:0/473798522 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:48.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.742+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e0109210 0x7fc8e019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.742+0000 7fc8e5600640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0109bd0 0x7fc8e01a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.742+0000 7fc8e4c00640 1 -- 172.21.15.62:0/473798522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0109bd0 msgr2=0x7fc8e01a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.742+0000 7fc8e4c00640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0109bd0 0x7fc8e01a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.742+0000 7fc8e4c00640 1 -- 172.21.15.62:0/473798522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e0109210 msgr2=0x7fc8e019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.742+0000 7fc8e4c00640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e0109210 0x7fc8e019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.742+0000 7fc8e4c00640 1 -- 172.21.15.62:0/473798522 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e01a6020 con 0x7fc8e0108020 2024-04-02T14:43:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.742+0000 7fc8e4c00640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e0108020 0x7fc8e019e450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400e9c0 tx=0x7fc8d400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.742+0000 7fc8dd600640 1 -- 172.21.15.62:0/473798522 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d400ccc0 con 0x7fc8e0108020 2024-04-02T14:43:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.743+0000 7fc8dd600640 1 -- 172.21.15.62:0/473798522 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8d40035f0 con 0x7fc8e0108020 2024-04-02T14:43:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.743+0000 7fc8dd600640 1 -- 172.21.15.62:0/473798522 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d400ce60 con 0x7fc8e0108020 2024-04-02T14:43:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.743+0000 7fc8e6000640 1 -- 172.21.15.62:0/473798522 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e01a6310 con 0x7fc8e0108020 2024-04-02T14:43:48.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.743+0000 7fc8e6000640 1 -- 172.21.15.62:0/473798522 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e01a67e0 con 0x7fc8e0108020 2024-04-02T14:43:48.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.746+0000 7fc8dd600640 1 -- 172.21.15.62:0/473798522 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc8d40037a0 con 0x7fc8e0108020 2024-04-02T14:43:48.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.746+0000 7fc8ea390740 1 Processor -- start 2024-04-02T14:43:48.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.746+0000 7fc8ea390740 1 -- start start 2024-04-02T14:43:48.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.746+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751eb10 0x55948751ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.746+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751f500 0x559487524000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.746+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559487524670 0x559487526b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.746+0000 7fc8ea390740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559487484060 con 0x55948751eb10 2024-04-02T14:43:48.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.746+0000 7fc8ea390740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55948742c560 con 0x55948751f500 2024-04-02T14:43:48.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.746+0000 7fc8ea390740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559487480010 con 0x559487524670 2024-04-02T14:43:48.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.747+0000 7fc8e5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751eb10 0x55948751ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.747+0000 7fc8e5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751eb10 0x55948751ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46668/0 (socket says 172.21.15.62:46668) 2024-04-02T14:43:48.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.747+0000 7fc8e5600640 1 -- 172.21.15.62:0/2222150068 learned_addr learned my addr 172.21.15.62:0/2222150068 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:48.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.747+0000 7fc8e4c00640 1 --2- 172.21.15.62:0/2222150068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751f500 0x559487524000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.747+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/2222150068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559487524670 0x559487526b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.747+0000 7fc8e5600640 1 -- 172.21.15.62:0/2222150068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559487524670 msgr2=0x559487526b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.747+0000 7fc8e5600640 1 --2- 172.21.15.62:0/2222150068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559487524670 0x559487526b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.747+0000 7fc8e5600640 1 -- 172.21.15.62:0/2222150068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751f500 msgr2=0x559487524000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.747+0000 7fc8e5600640 1 --2- 172.21.15.62:0/2222150068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751f500 0x559487524000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.747+0000 7fc8e5600640 1 -- 172.21.15.62:0/2222150068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55948744efd0 con 0x55948751eb10 2024-04-02T14:43:48.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.747+0000 7fc8e5600640 1 --2- 172.21.15.62:0/2222150068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751eb10 0x55948751ef30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8011440 tx=0x7fc8d8011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a03d20f799b1081 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.748+0000 7fc8dd600640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc8c80a3c30 0x7fc8c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.748+0000 7fc8d1c00640 1 -- 172.21.15.62:0/2222150068 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8003c30 con 0x55948751eb10 2024-04-02T14:43:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.748+0000 7fc8dd600640 1 -- 172.21.15.62:0/473798522 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fc8d40d08c0 con 0x7fc8e0108020 2024-04-02T14:43:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.748+0000 7fc8d1c00640 1 -- 172.21.15.62:0/2222150068 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8d8003dd0 con 0x55948751eb10 2024-04-02T14:43:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.748+0000 7fc8e4c00640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc8c80a3c30 0x7fc8c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.748+0000 7fc8d1c00640 1 -- 172.21.15.62:0/2222150068 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800fb70 con 0x55948751eb10 2024-04-02T14:43:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.749+0000 7fc8ea390740 1 -- 172.21.15.62:0/2222150068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751eb10 msgr2=0x55948751ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.749+0000 7fc8e4c00640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc8c80a3c30 0x7fc8c80a60f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40058c0 tx=0x7fc8d40d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.749+0000 7fc8ea390740 1 --2- 172.21.15.62:0/2222150068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751eb10 0x55948751ef30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8011440 tx=0x7fc8d8011900 comp rx=0 tx=0).stop 2024-04-02T14:43:48.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.749+0000 7fc8ea390740 1 -- 172.21.15.62:0/2222150068 shutdown_connections 2024-04-02T14:43:48.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.749+0000 7fc8ea390740 1 --2- 172.21.15.62:0/2222150068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559487524670 0x559487526b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.749+0000 7fc8ea390740 1 --2- 172.21.15.62:0/2222150068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751f500 0x559487524000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.749+0000 7fc8ea390740 1 --2- 172.21.15.62:0/2222150068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751eb10 0x55948751ef30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.749+0000 7fc8ea390740 1 -- 172.21.15.62:0/2222150068 >> 172.21.15.62:0/2222150068 conn(0x55948751a080 msgr2=0x55948751a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.749+0000 7fc8ea390740 1 -- 172.21.15.62:0/2222150068 shutdown_connections 2024-04-02T14:43:48.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.749+0000 7fc8ea390740 1 -- 172.21.15.62:0/2222150068 wait complete. 2024-04-02T14:43:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.749+0000 7fc8ea390740 1 -- start start 2024-04-02T14:43:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8ea390740 1 Processor -- start 2024-04-02T14:43:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751eb10 0x559487529010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 0x559487529550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559487524670 0x559487530590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8ea390740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559487527620 con 0x55948751f500 2024-04-02T14:43:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8ea390740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55948743f920 con 0x55948751eb10 2024-04-02T14:43:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8ea390740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5594875277a0 con 0x559487524670 2024-04-02T14:43:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8dfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 0x559487529550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8ea390740 10 client.0 _ll_register_callbacks cb 0x55948743f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8dfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 0x559487529550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46676/0 (socket says 172.21.15.62:46676) 2024-04-02T14:43:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8dfe00640 1 -- 172.21.15.62:0/1638047189 learned_addr learned my addr 172.21.15.62:0/1638047189 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.750+0000 7fc8e5600640 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751eb10 0x559487529010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8e5600640 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559487524670 0x559487530590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8dfe00640 1 -- 172.21.15.62:0/1638047189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559487524670 msgr2=0x559487530590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559487524670 0x559487530590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8dfe00640 1 -- 172.21.15.62:0/1638047189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751eb10 msgr2=0x559487529010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751eb10 0x559487529010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8dfe00640 1 -- 172.21.15.62:0/1638047189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594871982d0 con 0x55948751f500 2024-04-02T14:43:48.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8e5600640 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751eb10 0x559487529010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:48.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 0x559487529550 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009900 tx=0x7fc8cc02c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a59910e134b65a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8c6000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:43:48.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8c6000640 1 -- 172.21.15.62:0/1638047189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc004250 con 0x55948751f500 2024-04-02T14:43:48.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8c6000640 1 -- 172.21.15.62:0/1638047189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fc8cc0043f0 con 0x55948751f500 2024-04-02T14:43:48.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.751+0000 7fc8ea390740 1 -- 172.21.15.62:0/1638047189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d8003dd0 con 0x55948751f500 2024-04-02T14:43:48.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.752+0000 7fc8c6000640 1 -- 172.21.15.62:0/1638047189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8cc005030 con 0x55948751f500 2024-04-02T14:43:48.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.752+0000 7fc8ea390740 10 client.15093 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:48.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.752+0000 7fc8ea390740 1 -- 172.21.15.62:0/1638047189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594872626b0 con 0x55948751f500 2024-04-02T14:43:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.752+0000 7fc8c6000640 1 -- 172.21.15.62:0/1638047189 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8d8003dd0 con 0x55948751f500 2024-04-02T14:43:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.752+0000 7fc8c6000640 1 -- 172.21.15.62:0/1638047189 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc8cc0035f0 con 0x55948751f500 2024-04-02T14:43:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.752+0000 7fc8ea390740 10 client.15093 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.752+0000 7fc8ea390740 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T14:43:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.752+0000 7fc8ea390740 20 client.15093 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.752+0000 7fc8ea390740 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x559487535e70 0x5594875372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.752+0000 7fc8ea390740 4 client.15093 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:43:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.752+0000 7fc8ea390740 1 -- 172.21.15.62:0/1638047189 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594874915a0 con 0x559487535e70 2024-04-02T14:43:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.753+0000 7fc8e5600640 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x559487535e70 0x5594875372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.753+0000 7fc8e5600640 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x559487535e70 0x5594875372e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44cd1c82d8e25264 server_cookie=f06f988e1b568bcf in_seq=0 out_seq=0 2024-04-02T14:43:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.753+0000 7fc8c6000640 10 client.15093 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:43:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8c6000640 1 -- 172.21.15.62:0/1638047189 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594874915a0 con 0x559487535e70 2024-04-02T14:43:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8c6000640 10 client.15093 handle_command_reply: tid=0 2024-04-02T14:43:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8ea390740 1 client.15093 shutdown 2024-04-02T14:43:48.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8ea390740 1 -- 172.21.15.62:0/1638047189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 msgr2=0x559487529550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8ea390740 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 0x559487529550 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009900 tx=0x7fc8cc02c800 comp rx=0 tx=0).stop 2024-04-02T14:43:48.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8ea390740 1 -- 172.21.15.62:0/1638047189 shutdown_connections 2024-04-02T14:43:48.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8ea390740 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x559487535e70 0x5594875372e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8ea390740 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559487524670 0x559487530590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8ea390740 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 0x559487529550 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8ea390740 1 --2- 172.21.15.62:0/1638047189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751eb10 0x559487529010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8c6000640 0 client.15093 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:43:48.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.754+0000 7fc8ea390740 1 -- 172.21.15.62:0/1638047189 >> 172.21.15.62:0/1638047189 conn(0x55948751a080 msgr2=0x5594875191c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.755+0000 7fc8ea390740 1 -- 172.21.15.62:0/1638047189 shutdown_connections 2024-04-02T14:43:48.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.755+0000 7fc8ea390740 1 -- 172.21.15.62:0/1638047189 wait complete. 2024-04-02T14:43:48.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.755+0000 7fc8ea390740 20 client.15093 trim_cache size 0 max 16384 2024-04-02T14:43:48.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.775+0000 7fc8ea390740 1 Processor -- start 2024-04-02T14:43:48.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.775+0000 7fc8ea390740 1 -- start start 2024-04-02T14:43:48.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 0x55948751ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 0x559487530550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559487524670 0x559487514de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8ea390740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559487535620 con 0x55948751f500 2024-04-02T14:43:48.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8ea390740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559487535cf0 con 0x559487524670 2024-04-02T14:43:48.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8ea390740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5594875363c0 con 0x55948751eb10 2024-04-02T14:43:48.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8dfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 0x559487530550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8e5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 0x55948751ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8e5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 0x55948751ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35596/0 (socket says 172.21.15.62:35596) 2024-04-02T14:43:48.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8e5600640 1 -- 172.21.15.62:0/1081908378 learned_addr learned my addr 172.21.15.62:0/1081908378 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:48.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8e5600640 1 --2- 172.21.15.62:0/1081908378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559487524670 0x559487514de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8e5600640 1 -- 172.21.15.62:0/1081908378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559487524670 msgr2=0x559487514de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8e5600640 1 --2- 172.21.15.62:0/1081908378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559487524670 0x559487514de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8e5600640 1 -- 172.21.15.62:0/1081908378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 msgr2=0x559487530550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8e5600640 1 --2- 172.21.15.62:0/1081908378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 0x559487530550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8e5600640 1 -- 172.21.15.62:0/1081908378 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55948751fd90 con 0x55948751eb10 2024-04-02T14:43:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.776+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/1081908378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 0x559487530550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.777+0000 7fc8e5600640 1 --2- 172.21.15.62:0/1081908378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 0x55948751ffa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80099a0 tx=0x7fc8d8009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed6cd01acff5be7b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.777+0000 7fc8d1200640 1 -- 172.21.15.62:0/1081908378 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d802ed50 con 0x55948751eb10 2024-04-02T14:43:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.777+0000 7fc8d1200640 1 -- 172.21.15.62:0/1081908378 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8d802e6d0 con 0x55948751eb10 2024-04-02T14:43:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.777+0000 7fc8d1200640 1 -- 172.21.15.62:0/1081908378 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8004650 con 0x55948751eb10 2024-04-02T14:43:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.778+0000 7fc8ea390740 1 -- 172.21.15.62:0/1081908378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 msgr2=0x55948751ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.778+0000 7fc8ea390740 1 --2- 172.21.15.62:0/1081908378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 0x55948751ffa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80099a0 tx=0x7fc8d8009bf0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.778+0000 7fc8ea390740 1 -- 172.21.15.62:0/1081908378 shutdown_connections 2024-04-02T14:43:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.778+0000 7fc8ea390740 1 --2- 172.21.15.62:0/1081908378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559487524670 0x559487514de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.778+0000 7fc8ea390740 1 --2- 172.21.15.62:0/1081908378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55948751f500 0x559487530550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.778+0000 7fc8ea390740 1 --2- 172.21.15.62:0/1081908378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 0x55948751ffa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.778+0000 7fc8ea390740 1 -- 172.21.15.62:0/1081908378 >> 172.21.15.62:0/1081908378 conn(0x55948751a080 msgr2=0x55948751a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.778+0000 7fc8ea390740 1 -- 172.21.15.62:0/1081908378 shutdown_connections 2024-04-02T14:43:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.778+0000 7fc8ea390740 1 -- 172.21.15.62:0/1081908378 wait complete. 2024-04-02T14:43:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.778+0000 7fc8ea390740 1 -- start start 2024-04-02T14:43:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.778+0000 7fc8ea390740 1 Processor -- start 2024-04-02T14:43:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 0x55948752e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751f500 0x559487597e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8ea390740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559487524670 0x55948759a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8ea390740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559487537160 con 0x559487524670 2024-04-02T14:43:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8ea390740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559487536a90 con 0x55948751f500 2024-04-02T14:43:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8ea390740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559487537830 con 0x55948751eb10 2024-04-02T14:43:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8ea390740 10 client.0 _ll_register_callbacks cb 0x7fc8cc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8e5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559487524670 0x55948759a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8e5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559487524670 0x55948759a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46702/0 (socket says 172.21.15.62:46702) 2024-04-02T14:43:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8e5600640 1 -- 172.21.15.62:0/499369953 learned_addr learned my addr 172.21.15.62:0/499369953 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8e5600640 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 0x55948752e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.779+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751f500 0x559487597e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.780+0000 7fc8e5600640 1 -- 172.21.15.62:0/499369953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 msgr2=0x55948752e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.780+0000 7fc8e5600640 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 0x55948752e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.780+0000 7fc8e5600640 1 -- 172.21.15.62:0/499369953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751f500 msgr2=0x559487597e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.780+0000 7fc8e5600640 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751f500 0x559487597e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.780+0000 7fc8e5600640 1 -- 172.21.15.62:0/499369953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55948759ac50 con 0x559487524670 2024-04-02T14:43:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.780+0000 7fc8e5600640 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559487524670 0x55948759a420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8006010 tx=0x7fc8d8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d220107328810a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.780+0000 7fc8c6000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:43:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.780+0000 7fc8c6000640 1 -- 172.21.15.62:0/499369953 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8004660 con 0x559487524670 2024-04-02T14:43:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.780+0000 7fc8c6000640 1 -- 172.21.15.62:0/499369953 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7fc8d8003d30 con 0x559487524670 2024-04-02T14:43:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.780+0000 7fc8ea390740 1 -- 172.21.15.62:0/499369953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d802e6d0 con 0x559487524670 2024-04-02T14:43:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.781+0000 7fc8c6000640 1 -- 172.21.15.62:0/499369953 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8d8003f30 con 0x559487524670 2024-04-02T14:43:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.781+0000 7fc8ea390740 10 client.15099 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.781+0000 7fc8ea390740 1 -- 172.21.15.62:0/499369953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55948759bf50 con 0x559487524670 2024-04-02T14:43:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.781+0000 7fc8c6000640 1 -- 172.21.15.62:0/499369953 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8d802e6d0 con 0x559487524670 2024-04-02T14:43:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.781+0000 7fc8c6000640 1 -- 172.21.15.62:0/499369953 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc8d800f540 con 0x559487524670 2024-04-02T14:43:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.781+0000 7fc8ea390740 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.781+0000 7fc8ea390740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T14:43:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.781+0000 7fc8ea390740 20 client.15099 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:48.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.781+0000 7fc8ea390740 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x5594875a04d0 0x55948759cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:48.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.781+0000 7fc8ea390740 4 client.15099 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T14:43:48.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.781+0000 7fc8ea390740 1 -- 172.21.15.62:0/499369953 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "status"}) -- 0x55948748b8f0 con 0x5594875a04d0 2024-04-02T14:43:48.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.782+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x5594875a04d0 0x55948759cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:48.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.782+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x5594875a04d0 0x55948759cfe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3ebb96af205620e1 server_cookie=2038a5b04d299939 in_seq=0 out_seq=0 2024-04-02T14:43:48.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.782+0000 7fc8c6000640 10 client.15099 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:43:48.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.782+0000 7fc8c6000640 1 -- 172.21.15.62:0/499369953 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55948748b8f0 con 0x5594875a04d0 2024-04-02T14:43:48.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.782+0000 7fc8c6000640 10 client.15099 handle_command_reply: tid=0 2024-04-02T14:43:48.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.782+0000 7fc8ea390740 1 client.15099 shutdown 2024-04-02T14:43:48.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8ea390740 1 -- 172.21.15.62:0/499369953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559487524670 msgr2=0x55948759a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8ea390740 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559487524670 0x55948759a420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8006010 tx=0x7fc8d8000f40 comp rx=0 tx=0).stop 2024-04-02T14:43:48.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8ea390740 1 -- 172.21.15.62:0/499369953 shutdown_connections 2024-04-02T14:43:48.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8ea390740 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x5594875a04d0 0x55948759cfe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8ea390740 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559487524670 0x55948759a420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8ea390740 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55948751f500 0x559487597e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8ea390740 1 --2- 172.21.15.62:0/499369953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55948751eb10 0x55948752e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8ea390740 1 -- 172.21.15.62:0/499369953 >> 172.21.15.62:0/499369953 conn(0x55948751a080 msgr2=0x55948751ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8c6000640 0 client.15099 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:43:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8ea390740 1 -- 172.21.15.62:0/499369953 shutdown_connections 2024-04-02T14:43:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8ea390740 1 -- 172.21.15.62:0/499369953 wait complete. 2024-04-02T14:43:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.783+0000 7fc8ea390740 20 client.15099 trim_cache size 0 max 16384 2024-04-02T14:43:48.797 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:43:48.798 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:43:48.798 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 3, 2024-04-02T14:43:48.798 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24535, 2024-04-02T14:43:48.798 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-02T14:43:48.798 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-02T14:43:48.798 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-02T14:43:48.798 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 140.717692734, 2024-04-02T14:43:48.798 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-04-02T14:43:48.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-04-02T14:43:48.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-04-02T14:43:48.799 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 164.20353238000001 2024-04-02T14:43:48.799 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:43:48.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 -- 172.21.15.62:0/473798522 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc8c80a3c30 msgr2=0x7fc8c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc8c80a3c30 0x7fc8c80a60f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40058c0 tx=0x7fc8d40d6530 comp rx=0 tx=0).stop 2024-04-02T14:43:48.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 -- 172.21.15.62:0/473798522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e0108020 msgr2=0x7fc8e019e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:48.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e0108020 0x7fc8e019e450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400e9c0 tx=0x7fc8d400edc0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 -- 172.21.15.62:0/473798522 shutdown_connections 2024-04-02T14:43:48.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7fc8c80a3c30 0x7fc8c80a60f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0109bd0 0x7fc8e01a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e0109210 0x7fc8e019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 --2- 172.21.15.62:0/473798522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e0108020 0x7fc8e019e450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:48.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 -- 172.21.15.62:0/473798522 >> 172.21.15.62:0/473798522 conn(0x7fc8e0076fd0 msgr2=0x7fc8e010c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:48.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 -- 172.21.15.62:0/473798522 shutdown_connections 2024-04-02T14:43:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:48.784+0000 7fc8e6000640 1 -- 172.21.15.62:0/473798522 wait complete. 2024-04-02T14:43:48.802 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:48 smithi062 ceph-mon[32116]: pgmap v394: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:48.802 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:48 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2051376449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:48.802 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:48 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2864754531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:48.804 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-02T14:43:48.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:48 smithi113 ceph-mon[36135]: pgmap v394: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:48.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:48 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2051376449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:48.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:48 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2864754531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:49.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:48 smithi204 ceph-mon[35948]: pgmap v394: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:43:49.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:48 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2051376449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:43:49.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:48 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2864754531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:43:49.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.028+0000 7f25d0a00640 1 -- 172.21.15.62:0/3827301143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25cc103590 msgr2=0x7f25cc109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.028+0000 7f25d0a00640 1 --2- 172.21.15.62:0/3827301143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25cc103590 0x7f25cc109e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009ca0 tx=0x7f25c402c100 comp rx=0 tx=0).stop 2024-04-02T14:43:49.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.028+0000 7f25d0a00640 1 -- 172.21.15.62:0/3827301143 shutdown_connections 2024-04-02T14:43:49.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.028+0000 7f25d0a00640 1 --2- 172.21.15.62:0/3827301143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25cc103590 0x7f25cc109e20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.028+0000 7f25d0a00640 1 --2- 172.21.15.62:0/3827301143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc102bd0 0x7f25cc103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.028+0000 7f25d0a00640 1 --2- 172.21.15.62:0/3827301143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25cc1019e0 0x7f25cc101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.028+0000 7f25d0a00640 1 -- 172.21.15.62:0/3827301143 >> 172.21.15.62:0/3827301143 conn(0x7f25cc0fcf70 msgr2=0x7f25cc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.028+0000 7f25d0a00640 1 -- 172.21.15.62:0/3827301143 shutdown_connections 2024-04-02T14:43:49.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.029+0000 7f25d0a00640 1 -- 172.21.15.62:0/3827301143 wait complete. 2024-04-02T14:43:49.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.030+0000 7f25d0a00640 1 Processor -- start 2024-04-02T14:43:49.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.030+0000 7f25d0a00640 1 -- start start 2024-04-02T14:43:49.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.030+0000 7f25d0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25cc1019e0 0x7f25cc197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.030+0000 7f25d0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc102bd0 0x7f25cc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25d0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25cc103590 0x7f25cc19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25d0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc10d790 con 0x7f25cc103590 2024-04-02T14:43:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25d0a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc10d610 con 0x7f25cc1019e0 2024-04-02T14:43:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25d0a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc10d910 con 0x7f25cc102bd0 2024-04-02T14:43:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25cb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25cc1019e0 0x7f25cc197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25cb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25cc1019e0 0x7f25cc197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:33092/0 (socket says 172.21.15.62:33092) 2024-04-02T14:43:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25cc103590 0x7f25cc19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25cb400640 1 -- 172.21.15.62:0/378540855 learned_addr learned my addr 172.21.15.62:0/378540855 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25caa00640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc102bd0 0x7f25cc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25cb400640 1 -- 172.21.15.62:0/378540855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc102bd0 msgr2=0x7f25cc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25cb400640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc102bd0 0x7f25cc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25cb400640 1 -- 172.21.15.62:0/378540855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25cc103590 msgr2=0x7f25cc19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25cb400640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25cc103590 0x7f25cc19f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25cb400640 1 -- 172.21.15.62:0/378540855 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25cc19fa30 con 0x7f25cc1019e0 2024-04-02T14:43:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25caa00640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc102bd0 0x7f25cc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.031+0000 7f25cbe00640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25cc103590 0x7f25cc19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.032+0000 7f25cb400640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25cc1019e0 0x7f25cc197e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00e9c0 tx=0x7f25bc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.032+0000 7f25c3e00640 1 -- 172.21.15.62:0/378540855 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc00cde0 con 0x7f25cc1019e0 2024-04-02T14:43:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.032+0000 7f25c3e00640 1 -- 172.21.15.62:0/378540855 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f25bc010930 con 0x7f25cc1019e0 2024-04-02T14:43:49.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.032+0000 7f25d0a00640 1 -- 172.21.15.62:0/378540855 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25cc19fd20 con 0x7f25cc1019e0 2024-04-02T14:43:49.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.032+0000 7f25c3e00640 1 -- 172.21.15.62:0/378540855 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc018a60 con 0x7f25cc1019e0 2024-04-02T14:43:49.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.032+0000 7f25d0a00640 1 -- 172.21.15.62:0/378540855 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25cc1a0200 con 0x7f25cc1019e0 2024-04-02T14:43:49.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.035+0000 7f25c3e00640 1 -- 172.21.15.62:0/378540855 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f25bc01f020 con 0x7f25cc1019e0 2024-04-02T14:43:49.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.035+0000 7f25d4e20740 1 Processor -- start 2024-04-02T14:43:49.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.035+0000 7f25d4e20740 1 -- start start 2024-04-02T14:43:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.036+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3009340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.036+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 0x5650e300e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.036+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e300ea80 0x5650e3010f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.036+0000 7f25d4e20740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650e2f6a060 con 0x5650e3009910 2024-04-02T14:43:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.036+0000 7f25d4e20740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5650e2f12560 con 0x5650e300ea80 2024-04-02T14:43:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.036+0000 7f25d4e20740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5650e2f66010 con 0x5650e3008f20 2024-04-02T14:43:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.036+0000 7f25cb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 0x5650e300e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.036+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3009340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.036+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3009340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35640/0 (socket says 172.21.15.62:35640) 2024-04-02T14:43:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.036+0000 7f25cbe00640 1 -- 172.21.15.62:0/3456965080 learned_addr learned my addr 172.21.15.62:0/3456965080 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.036+0000 7f25caa00640 1 --2- 172.21.15.62:0/3456965080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e300ea80 0x5650e3010f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25cbe00640 1 -- 172.21.15.62:0/3456965080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e300ea80 msgr2=0x5650e3010f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25cbe00640 1 --2- 172.21.15.62:0/3456965080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e300ea80 0x5650e3010f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25cbe00640 1 -- 172.21.15.62:0/3456965080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 msgr2=0x5650e300e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25cbe00640 1 --2- 172.21.15.62:0/3456965080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 0x5650e300e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25cbe00640 1 -- 172.21.15.62:0/3456965080 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e2f34fd0 con 0x5650e3008f20 2024-04-02T14:43:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25cbe00640 1 --2- 172.21.15.62:0/3456965080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3009340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009ca0 tx=0x7f25c4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5c8b5a6e12f9c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25c3e00640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25b00a3c30 0x7f25b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25b7e00640 1 -- 172.21.15.62:0/3456965080 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4043070 con 0x5650e3008f20 2024-04-02T14:43:49.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25c3e00640 1 -- 172.21.15.62:0/378540855 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f25bc0d0610 con 0x7f25cc1019e0 2024-04-02T14:43:49.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25cb400640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25b00a3c30 0x7f25b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25b7e00640 1 -- 172.21.15.62:0/3456965080 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f25c4032d40 con 0x5650e3008f20 2024-04-02T14:43:49.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.037+0000 7f25b7e00640 1 -- 172.21.15.62:0/3456965080 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4039040 con 0x5650e3008f20 2024-04-02T14:43:49.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.038+0000 7f25d4e20740 1 -- 172.21.15.62:0/3456965080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 msgr2=0x5650e3009340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.038+0000 7f25cb400640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25b00a3c30 0x7f25b00a60f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5650e3009d10 tx=0x7f25bc0d60e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.038+0000 7f25d4e20740 1 --2- 172.21.15.62:0/3456965080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3009340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009ca0 tx=0x7f25c4005ef0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.038+0000 7f25d4e20740 1 -- 172.21.15.62:0/3456965080 shutdown_connections 2024-04-02T14:43:49.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.038+0000 7f25d4e20740 1 --2- 172.21.15.62:0/3456965080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e300ea80 0x5650e3010f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.038+0000 7f25d4e20740 1 --2- 172.21.15.62:0/3456965080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 0x5650e300e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.038+0000 7f25d4e20740 1 --2- 172.21.15.62:0/3456965080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3009340 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.038+0000 7f25d4e20740 1 -- 172.21.15.62:0/3456965080 >> 172.21.15.62:0/3456965080 conn(0x5650e3004490 msgr2=0x5650e3004890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.038+0000 7f25d4e20740 1 -- 172.21.15.62:0/3456965080 shutdown_connections 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.038+0000 7f25d4e20740 1 -- 172.21.15.62:0/3456965080 wait complete. 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.039+0000 7f25d4e20740 1 -- start start 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.039+0000 7f25d4e20740 1 Processor -- start 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.039+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e3008f20 0x5650e3013430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.039+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 0x5650e3013970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.039+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e300ea80 0x5650e301a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.039+0000 7f25d4e20740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650e30119d0 con 0x5650e3009910 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.039+0000 7f25d4e20740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5650e2f25920 con 0x5650e3008f20 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.039+0000 7f25d4e20740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5650e3011b50 con 0x5650e300ea80 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25d4e20740 10 client.0 _ll_register_callbacks cb 0x5650e2f25310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25caa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 0x5650e3013970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e3008f20 0x5650e3013430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25caa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 0x5650e3013970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46734/0 (socket says 172.21.15.62:46734) 2024-04-02T14:43:49.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25caa00640 1 -- 172.21.15.62:0/2721510034 learned_addr learned my addr 172.21.15.62:0/2721510034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25cbe00640 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e300ea80 0x5650e301a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25caa00640 1 -- 172.21.15.62:0/2721510034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e300ea80 msgr2=0x5650e301a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25caa00640 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e300ea80 0x5650e301a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25caa00640 1 -- 172.21.15.62:0/2721510034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e3008f20 msgr2=0x5650e3013430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25caa00640 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e3008f20 0x5650e3013430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25caa00640 1 -- 172.21.15.62:0/2721510034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e2c7e2d0 con 0x5650e3009910 2024-04-02T14:43:49.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25caa00640 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 0x5650e3013970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f25b80126b0 tx=0x7f25b8012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2834b7909962466 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.040+0000 7f25b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:43:49.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.041+0000 7f25b4c00640 1 -- 172.21.15.62:0/2721510034 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8011b50 con 0x5650e3009910 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.041+0000 7f25b4c00640 1 -- 172.21.15.62:0/2721510034 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f25b8011cf0 con 0x5650e3009910 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.041+0000 7f25d4e20740 1 -- 172.21.15.62:0/2721510034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c4032d40 con 0x5650e3009910 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.041+0000 7f25b4c00640 1 -- 172.21.15.62:0/2721510034 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25b8005020 con 0x5650e3009910 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.041+0000 7f25d4e20740 10 client.15111 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.041+0000 7f25d4e20740 1 -- 172.21.15.62:0/2721510034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650e2d486b0 con 0x5650e3009910 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.041+0000 7f25b4c00640 1 -- 172.21.15.62:0/2721510034 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f25c4032d40 con 0x5650e3009910 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.042+0000 7f25b4c00640 1 -- 172.21.15.62:0/2721510034 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f25b801c470 con 0x5650e3009910 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.042+0000 7f25d4e20740 10 client.15111 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.042+0000 7f25d4e20740 10 client.15111 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.042+0000 7f25d4e20740 20 client.15111 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.042+0000 7f25d4e20740 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5650e30202a0 0x5650e3021710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.042+0000 7f25d4e20740 4 client.15111 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.042+0000 7f25d4e20740 1 -- 172.21.15.62:0/2721510034 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650e2f775a0 con 0x5650e30202a0 2024-04-02T14:43:49.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.042+0000 7f25cbe00640 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5650e30202a0 0x5650e3021710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.043+0000 7f25cbe00640 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5650e30202a0 0x5650e3021710 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6547f346184ad45e server_cookie=af634cc402ff3dc0 in_seq=0 out_seq=0 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.043+0000 7f25b4c00640 10 client.15111 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.043+0000 7f25b4c00640 1 -- 172.21.15.62:0/2721510034 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650e2f775a0 con 0x5650e30202a0 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.043+0000 7f25b4c00640 10 client.15111 handle_command_reply: tid=0 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.043+0000 7f25d4e20740 1 client.15111 shutdown 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25d4e20740 1 -- 172.21.15.62:0/2721510034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 msgr2=0x5650e3013970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25d4e20740 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 0x5650e3013970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f25b80126b0 tx=0x7f25b8012b70 comp rx=0 tx=0).stop 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25d4e20740 1 -- 172.21.15.62:0/2721510034 shutdown_connections 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25d4e20740 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5650e30202a0 0x5650e3021710 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25d4e20740 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e300ea80 0x5650e301a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25d4e20740 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3009910 0x5650e3013970 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25d4e20740 1 --2- 172.21.15.62:0/2721510034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e3008f20 0x5650e3013430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25b4c00640 0 client.15111 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25d4e20740 1 -- 172.21.15.62:0/2721510034 >> 172.21.15.62:0/2721510034 conn(0x5650e3004490 msgr2=0x5650e2fff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25d4e20740 1 -- 172.21.15.62:0/2721510034 shutdown_connections 2024-04-02T14:43:49.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25d4e20740 1 -- 172.21.15.62:0/2721510034 wait complete. 2024-04-02T14:43:49.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.044+0000 7f25d4e20740 20 client.15111 trim_cache size 0 max 16384 2024-04-02T14:43:49.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.064+0000 7f25d4e20740 1 Processor -- start 2024-04-02T14:43:49.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.064+0000 7f25d4e20740 1 -- start start 2024-04-02T14:43:49.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.064+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3008f20 0x5650e3012a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.064+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3009910 0x5650e301a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.064+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e300ea80 0x5650e300ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.064+0000 7f25d4e20740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650e301fa50 con 0x5650e3008f20 2024-04-02T14:43:49.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.064+0000 7f25d4e20740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5650e3020120 con 0x5650e300ea80 2024-04-02T14:43:49.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.064+0000 7f25d4e20740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5650e30207f0 con 0x5650e3009910 2024-04-02T14:43:49.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.065+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3008f20 0x5650e3012a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.065+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3008f20 0x5650e3012a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46750/0 (socket says 172.21.15.62:46750) 2024-04-02T14:43:49.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.065+0000 7f25cbe00640 1 -- 172.21.15.62:0/113265279 learned_addr learned my addr 172.21.15.62:0/113265279 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.065+0000 7f25caa00640 1 --2- 172.21.15.62:0/113265279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3009910 0x5650e301a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.065+0000 7f25cbe00640 1 --2- 172.21.15.62:0/113265279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e300ea80 0x5650e300ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.065+0000 7f25cbe00640 1 -- 172.21.15.62:0/113265279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3009910 msgr2=0x5650e301a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.065+0000 7f25cbe00640 1 --2- 172.21.15.62:0/113265279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3009910 0x5650e301a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.065+0000 7f25cbe00640 1 -- 172.21.15.62:0/113265279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e300ea80 msgr2=0x5650e300ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.065+0000 7f25cbe00640 1 --2- 172.21.15.62:0/113265279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e300ea80 0x5650e300ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.065+0000 7f25cbe00640 1 -- 172.21.15.62:0/113265279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e300a1a0 con 0x5650e3008f20 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.065+0000 7f25cbe00640 1 --2- 172.21.15.62:0/113265279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3008f20 0x5650e3012a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f25c4000f80 tx=0x7f25c404b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d41f75336a3a57d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25b7400640 1 -- 172.21.15.62:0/113265279 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4039040 con 0x5650e3008f20 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25b7400640 1 -- 172.21.15.62:0/113265279 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f25c4032d40 con 0x5650e3008f20 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25b7400640 1 -- 172.21.15.62:0/113265279 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4043040 con 0x5650e3008f20 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25d4e20740 1 -- 172.21.15.62:0/113265279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3008f20 msgr2=0x5650e3012a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25d4e20740 1 --2- 172.21.15.62:0/113265279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3008f20 0x5650e3012a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f25c4000f80 tx=0x7f25c404b040 comp rx=0 tx=0).stop 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25d4e20740 1 -- 172.21.15.62:0/113265279 shutdown_connections 2024-04-02T14:43:49.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25d4e20740 1 --2- 172.21.15.62:0/113265279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e300ea80 0x5650e300ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25d4e20740 1 --2- 172.21.15.62:0/113265279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3009910 0x5650e301a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25d4e20740 1 --2- 172.21.15.62:0/113265279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e3008f20 0x5650e3012a50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25d4e20740 1 -- 172.21.15.62:0/113265279 >> 172.21.15.62:0/113265279 conn(0x5650e3004490 msgr2=0x5650e2ffade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25d4e20740 1 -- 172.21.15.62:0/113265279 shutdown_connections 2024-04-02T14:43:49.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.066+0000 7f25d4e20740 1 -- 172.21.15.62:0/113265279 wait complete. 2024-04-02T14:43:49.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.067+0000 7f25d4e20740 1 -- start start 2024-04-02T14:43:49.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.067+0000 7f25d4e20740 1 Processor -- start 2024-04-02T14:43:49.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.067+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3015980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.067+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e3009910 0x5650e3015ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.067+0000 7f25d4e20740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e300ea80 0x5650e3018ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.067+0000 7f25d4e20740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650e3021590 con 0x5650e300ea80 2024-04-02T14:43:49.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.067+0000 7f25d4e20740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5650e3020ec0 con 0x5650e3009910 2024-04-02T14:43:49.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.067+0000 7f25d4e20740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5650e3021c60 con 0x5650e3008f20 2024-04-02T14:43:49.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.067+0000 7f25d4e20740 10 client.0 _ll_register_callbacks cb 0x7f25b8009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:49.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3015980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3015980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35664/0 (socket says 172.21.15.62:35664) 2024-04-02T14:43:49.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25cbe00640 1 -- 172.21.15.62:0/4030182594 learned_addr learned my addr 172.21.15.62:0/4030182594 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25cbe00640 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e300ea80 0x5650e3018ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25caa00640 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e3009910 0x5650e3015ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25cbe00640 1 -- 172.21.15.62:0/4030182594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e3009910 msgr2=0x5650e3015ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25cbe00640 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e3009910 0x5650e3015ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25cbe00640 1 -- 172.21.15.62:0/4030182594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e300ea80 msgr2=0x5650e3018ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25cbe00640 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e300ea80 0x5650e3018ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25cbe00640 1 -- 172.21.15.62:0/4030182594 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e30194d0 con 0x5650e3008f20 2024-04-02T14:43:49.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25cbe00640 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3015980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f25c4005e30 tx=0x7f25c4003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ecf87263a26364c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.068+0000 7f25b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:43:49.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25b4c00640 1 -- 172.21.15.62:0/4030182594 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4039040 con 0x5650e3008f20 2024-04-02T14:43:49.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25b4c00640 1 -- 172.21.15.62:0/4030182594 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f25c40577c0 con 0x5650e3008f20 2024-04-02T14:43:49.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25d4e20740 1 -- 172.21.15.62:0/4030182594 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c4032d40 con 0x5650e3008f20 2024-04-02T14:43:49.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25b4c00640 1 -- 172.21.15.62:0/4030182594 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25c4043040 con 0x5650e3008f20 2024-04-02T14:43:49.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25d4e20740 10 client.25216 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:49.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25d4e20740 1 -- 172.21.15.62:0/4030182594 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650e301a7d0 con 0x5650e3008f20 2024-04-02T14:43:49.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25b4c00640 1 -- 172.21.15.62:0/4030182594 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f25c4032d40 con 0x5650e3008f20 2024-04-02T14:43:49.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25b4c00640 1 -- 172.21.15.62:0/4030182594 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f25c4057ce0 con 0x5650e3008f20 2024-04-02T14:43:49.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25d4e20740 10 client.25216 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:49.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25d4e20740 10 client.25216 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-02T14:43:49.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25d4e20740 20 client.25216 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:49.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25d4e20740 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5650e308a240 0x5650e3088f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.069+0000 7f25d4e20740 4 client.25216 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T14:43:49.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.070+0000 7f25d4e20740 1 -- 172.21.15.62:0/4030182594 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "status"}) -- 0x5650e2f718f0 con 0x5650e308a240 2024-04-02T14:43:49.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.070+0000 7f25caa00640 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5650e308a240 0x5650e3088f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.070+0000 7f25caa00640 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5650e308a240 0x5650e3088f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fed30da7a902c216 server_cookie=8a409be90677a5b5 in_seq=0 out_seq=0 2024-04-02T14:43:49.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.070+0000 7f25b4c00640 10 client.25216 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:43:49.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25b4c00640 1 -- 172.21.15.62:0/4030182594 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5650e2f718f0 con 0x5650e308a240 2024-04-02T14:43:49.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25b4c00640 10 client.25216 handle_command_reply: tid=0 2024-04-02T14:43:49.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 1 client.25216 shutdown 2024-04-02T14:43:49.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 1 -- 172.21.15.62:0/4030182594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 msgr2=0x5650e3015980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3015980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f25c4005e30 tx=0x7f25c4003ee0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 1 -- 172.21.15.62:0/4030182594 shutdown_connections 2024-04-02T14:43:49.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5650e308a240 0x5650e3088f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e300ea80 0x5650e3018ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25b4c00640 0 client.25216 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:43:49.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5650e3009910 0x5650e3015ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 1 --2- 172.21.15.62:0/4030182594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e3008f20 0x5650e3015980 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 1 -- 172.21.15.62:0/4030182594 >> 172.21.15.62:0/4030182594 conn(0x5650e3004490 msgr2=0x5650e300ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 1 -- 172.21.15.62:0/4030182594 shutdown_connections 2024-04-02T14:43:49.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 1 -- 172.21.15.62:0/4030182594 wait complete. 2024-04-02T14:43:49.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.071+0000 7f25d4e20740 20 client.25216 trim_cache size 0 max 16384 2024-04-02T14:43:49.079 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:43:49.080 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:43:49.080 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-04-02T14:43:49.080 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24553, 2024-04-02T14:43:49.080 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-02T14:43:49.080 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-02T14:43:49.080 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-02T14:43:49.080 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 142.0230574, 2024-04-02T14:43:49.080 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-04-02T14:43:49.081 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-04-02T14:43:49.081 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-04-02T14:43:49.081 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 160.38754462899999 2024-04-02T14:43:49.081 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:43:49.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 -- 172.21.15.62:0/378540855 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25b00a3c30 msgr2=0x7f25b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25b00a3c30 0x7f25b00a60f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5650e3009d10 tx=0x7f25bc0d60e0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 -- 172.21.15.62:0/378540855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25cc1019e0 msgr2=0x7f25cc197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25cc1019e0 0x7f25cc197e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00e9c0 tx=0x7f25bc00ee80 comp rx=0 tx=0).stop 2024-04-02T14:43:49.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 -- 172.21.15.62:0/378540855 shutdown_connections 2024-04-02T14:43:49.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f25b00a3c30 0x7f25b00a60f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25cc103590 0x7f25cc19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc102bd0 0x7f25cc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 --2- 172.21.15.62:0/378540855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25cc1019e0 0x7f25cc197e10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 -- 172.21.15.62:0/378540855 >> 172.21.15.62:0/378540855 conn(0x7f25cc0fcf70 msgr2=0x7f25cc105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 -- 172.21.15.62:0/378540855 shutdown_connections 2024-04-02T14:43:49.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.072+0000 7f25d0a00640 1 -- 172.21.15.62:0/378540855 wait complete. 2024-04-02T14:43:49.087 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-02T14:43:49.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.319+0000 7f91b8e00640 1 -- 172.21.15.62:0/791081636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b4107730 msgr2=0x7f91b4109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.319+0000 7f91b8e00640 1 --2- 172.21.15.62:0/791081636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b4107730 0x7f91b4109b20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009ca0 tx=0x7f91ac02c0f0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.319+0000 7f91b8e00640 1 -- 172.21.15.62:0/791081636 shutdown_connections 2024-04-02T14:43:49.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.319+0000 7f91b8e00640 1 --2- 172.21.15.62:0/791081636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b4107730 0x7f91b4109b20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.319+0000 7f91b8e00640 1 --2- 172.21.15.62:0/791081636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b4104e00 0x7f91b41071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.319+0000 7f91b8e00640 1 --2- 172.21.15.62:0/791081636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b41024d0 0x7f91b41048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.319+0000 7f91b8e00640 1 -- 172.21.15.62:0/791081636 >> 172.21.15.62:0/791081636 conn(0x7f91b40fbe80 msgr2=0x7f91b40fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.319+0000 7f91b8e00640 1 -- 172.21.15.62:0/791081636 shutdown_connections 2024-04-02T14:43:49.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.319+0000 7f91b8e00640 1 -- 172.21.15.62:0/791081636 wait complete. 2024-04-02T14:43:49.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.320+0000 7f91b8e00640 1 Processor -- start 2024-04-02T14:43:49.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.320+0000 7f91b8e00640 1 -- start start 2024-04-02T14:43:49.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b41024d0 0x7f91b4197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b8e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b4104e00 0x7f91b41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b4107730 0x7f91b419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b8e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f91b410d7a0 con 0x7f91b41024d0 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b8e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f91b410d620 con 0x7f91b4107730 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b8e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f91b410d920 con 0x7f91b4104e00 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b41024d0 0x7f91b4197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b41024d0 0x7f91b4197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46766/0 (socket says 172.21.15.62:46766) 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b3400640 1 -- 172.21.15.62:0/1972940613 learned_addr learned my addr 172.21.15.62:0/1972940613 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91abe00640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b4104e00 0x7f91b41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b3e00640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b4107730 0x7f91b419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b3400640 1 -- 172.21.15.62:0/1972940613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b4104e00 msgr2=0x7f91b41982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b3400640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b4104e00 0x7f91b41982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b3400640 1 -- 172.21.15.62:0/1972940613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b4107730 msgr2=0x7f91b419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b3400640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b4107730 0x7f91b419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b3400640 1 -- 172.21.15.62:0/1972940613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b419f9b0 con 0x7f91b41024d0 2024-04-02T14:43:49.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91abe00640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b4104e00 0x7f91b41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:49.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.321+0000 7f91b3e00640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b4107730 0x7f91b419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:49.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.322+0000 7f91b3400640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b41024d0 0x7f91b4197da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f91a400e9f0 tx=0x7f91a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.322+0000 7f91b0c00640 1 -- 172.21.15.62:0/1972940613 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a400cd80 con 0x7f91b41024d0 2024-04-02T14:43:49.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.322+0000 7f91b0c00640 1 -- 172.21.15.62:0/1972940613 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f91a4010930 con 0x7f91b41024d0 2024-04-02T14:43:49.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.322+0000 7f91b8e00640 1 -- 172.21.15.62:0/1972940613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b419fca0 con 0x7f91b41024d0 2024-04-02T14:43:49.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.322+0000 7f91b0c00640 1 -- 172.21.15.62:0/1972940613 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4018b00 con 0x7f91b41024d0 2024-04-02T14:43:49.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.322+0000 7f91b8e00640 1 -- 172.21.15.62:0/1972940613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b41a0190 con 0x7f91b41024d0 2024-04-02T14:43:49.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.325+0000 7f91bd1ba740 1 Processor -- start 2024-04-02T14:43:49.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91bd1ba740 1 -- start start 2024-04-02T14:43:49.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91b0c00640 1 -- 172.21.15.62:0/1972940613 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f91a4018ca0 con 0x7f91b41024d0 2024-04-02T14:43:49.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c323ff20 0x5644c3240340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c3240910 0x5644c3245410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3247f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5644c31a1060 con 0x5644c323ff20 2024-04-02T14:43:49.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5644c3149560 con 0x5644c3245a80 2024-04-02T14:43:49.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5644c319d010 con 0x5644c3240910 2024-04-02T14:43:49.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c323ff20 0x5644c3240340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c323ff20 0x5644c3240340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46774/0 (socket says 172.21.15.62:46774) 2024-04-02T14:43:49.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91b3e00640 1 -- 172.21.15.62:0/290100903 learned_addr learned my addr 172.21.15.62:0/290100903 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91abe00640 1 --2- 172.21.15.62:0/290100903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3247f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.326+0000 7f91b3400640 1 --2- 172.21.15.62:0/290100903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c3240910 0x5644c3245410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.327+0000 7f91b3e00640 1 -- 172.21.15.62:0/290100903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c3240910 msgr2=0x5644c3245410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.327+0000 7f91b3e00640 1 --2- 172.21.15.62:0/290100903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c3240910 0x5644c3245410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.327+0000 7f91b3e00640 1 -- 172.21.15.62:0/290100903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 msgr2=0x5644c3247f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.327+0000 7f91b3e00640 1 --2- 172.21.15.62:0/290100903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3247f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.327+0000 7f91b3e00640 1 -- 172.21.15.62:0/290100903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c316bfd0 con 0x5644c323ff20 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.327+0000 7f91b3400640 1 --2- 172.21.15.62:0/290100903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c3240910 0x5644c3245410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.327+0000 7f91b3e00640 1 --2- 172.21.15.62:0/290100903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c323ff20 0x5644c3240340 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009ca0 tx=0x7f91ac005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdebc06f29ba6f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.327+0000 7f91b0c00640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f91980a3c30 0x7f91980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.327+0000 7f91a8c00640 1 -- 172.21.15.62:0/290100903 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac043070 con 0x5644c323ff20 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.328+0000 7f91a8c00640 1 -- 172.21.15.62:0/290100903 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f91ac032d30 con 0x5644c323ff20 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.328+0000 7f91a8c00640 1 -- 172.21.15.62:0/290100903 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac039040 con 0x5644c323ff20 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.328+0000 7f91b0c00640 1 -- 172.21.15.62:0/1972940613 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f91a40d07b0 con 0x7f91b41024d0 2024-04-02T14:43:49.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.328+0000 7f91b3400640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f91980a3c30 0x7f91980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.328+0000 7f91bd1ba740 1 -- 172.21.15.62:0/290100903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c323ff20 msgr2=0x5644c3240340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.328+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/290100903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c323ff20 0x5644c3240340 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009ca0 tx=0x7f91ac005ef0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.328+0000 7f91b3400640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f91980a3c30 0x7f91980a60f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f91a409c810 tx=0x7f91a4018520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.328+0000 7f91bd1ba740 1 -- 172.21.15.62:0/290100903 shutdown_connections 2024-04-02T14:43:49.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.328+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/290100903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3247f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.328+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/290100903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c3240910 0x5644c3245410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.328+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/290100903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c323ff20 0x5644c3240340 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.329+0000 7f91bd1ba740 1 -- 172.21.15.62:0/290100903 >> 172.21.15.62:0/290100903 conn(0x5644c3237080 msgr2=0x5644c3237480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.329+0000 7f91bd1ba740 1 -- 172.21.15.62:0/290100903 shutdown_connections 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.329+0000 7f91bd1ba740 1 -- 172.21.15.62:0/290100903 wait complete. 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.329+0000 7f91bd1ba740 1 -- start start 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.329+0000 7f91bd1ba740 1 Processor -- start 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 0x5644c324a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c324a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3251990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5644c32489d0 con 0x5644c3240910 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5644c315c920 con 0x5644c3245a80 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5644c3248b50 con 0x5644c323ff20 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91bd1ba740 10 client.0 _ll_register_callbacks cb 0x5644c315c310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c324a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 0x5644c324a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c324a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46778/0 (socket says 172.21.15.62:46778) 2024-04-02T14:43:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 0x5644c324a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35712/0 (socket says 172.21.15.62:35712) 2024-04-02T14:43:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91abe00640 1 -- 172.21.15.62:0/1507341487 learned_addr learned my addr 172.21.15.62:0/1507341487 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.330+0000 7f91b3e00640 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3251990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.331+0000 7f91abe00640 1 -- 172.21.15.62:0/1507341487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 msgr2=0x5644c324a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.331+0000 7f91abe00640 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 0x5644c324a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.331+0000 7f91abe00640 1 -- 172.21.15.62:0/1507341487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 msgr2=0x5644c3251990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.331+0000 7f91abe00640 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3251990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.331+0000 7f91abe00640 1 -- 172.21.15.62:0/1507341487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c2eb52d0 con 0x5644c3240910 2024-04-02T14:43:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.331+0000 7f91b3e00640 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 0x5644c324a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.331+0000 7f91abe00640 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c324a950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f919c0117c0 tx=0x7f919c011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a105d3ac0702e614 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.331+0000 7f91a1600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:43:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.331+0000 7f91a1600640 1 -- 172.21.15.62:0/1507341487 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c010b40 con 0x5644c3240910 2024-04-02T14:43:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.332+0000 7f91a1600640 1 -- 172.21.15.62:0/1507341487 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f919c010ce0 con 0x5644c3240910 2024-04-02T14:43:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.332+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1507341487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91ac032d30 con 0x5644c3240910 2024-04-02T14:43:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.332+0000 7f91a1600640 1 -- 172.21.15.62:0/1507341487 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f919c005020 con 0x5644c3240910 2024-04-02T14:43:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.332+0000 7f91bd1ba740 10 client.15129 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.332+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1507341487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644c2f7f6b0 con 0x5644c3240910 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.332+0000 7f91a1600640 1 -- 172.21.15.62:0/1507341487 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f91ac032d30 con 0x5644c3240910 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.332+0000 7f91a1600640 1 -- 172.21.15.62:0/1507341487 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f919c026460 con 0x5644c3240910 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.333+0000 7f91bd1ba740 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.333+0000 7f91bd1ba740 10 client.15129 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.333+0000 7f91bd1ba740 20 client.15129 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.333+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5644c32572a0 0x5644c3258710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.333+0000 7f91bd1ba740 4 client.15129 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.333+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1507341487 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644c31ae5a0 con 0x5644c32572a0 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.333+0000 7f91b3e00640 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5644c32572a0 0x5644c3258710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.334+0000 7f91b3e00640 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5644c32572a0 0x5644c3258710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f6c5b1947f43a1 server_cookie=7f2b863788453ee1 in_seq=0 out_seq=0 2024-04-02T14:43:49.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.334+0000 7f91a1600640 10 client.15129 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.334+0000 7f91a1600640 1 -- 172.21.15.62:0/1507341487 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644c31ae5a0 con 0x5644c32572a0 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.334+0000 7f91a1600640 10 client.15129 handle_command_reply: tid=0 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.334+0000 7f91bd1ba740 1 client.15129 shutdown 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1507341487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 msgr2=0x5644c324a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c324a950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f919c0117c0 tx=0x7f919c011c80 comp rx=0 tx=0).stop 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1507341487 shutdown_connections 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5644c32572a0 0x5644c3258710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3251990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c324a950 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1507341487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 0x5644c324a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1507341487 >> 172.21.15.62:0/1507341487 conn(0x5644c3237080 msgr2=0x5644c32361c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91a1600640 0 client.15129 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:43:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1507341487 shutdown_connections 2024-04-02T14:43:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1507341487 wait complete. 2024-04-02T14:43:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.335+0000 7f91bd1ba740 20 client.15129 trim_cache size 0 max 16384 2024-04-02T14:43:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.356+0000 7f91bd1ba740 1 Processor -- start 2024-04-02T14:43:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.356+0000 7f91bd1ba740 1 -- start start 2024-04-02T14:43:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.356+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c323ff20 0x5644c3245440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.356+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c3240910 0x5644c3251950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.356+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3245a80 0x5644c3235930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.356+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5644c3256a50 con 0x5644c3245a80 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.356+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5644c3257120 con 0x5644c323ff20 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.356+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5644c32577f0 con 0x5644c3240910 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3245a80 0x5644c3235930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3245a80 0x5644c3235930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46780/0 (socket says 172.21.15.62:46780) 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91b3e00640 1 -- 172.21.15.62:0/3454218124 learned_addr learned my addr 172.21.15.62:0/3454218124 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91abe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c3240910 0x5644c3251950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91b3e00640 1 --2- 172.21.15.62:0/3454218124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c323ff20 0x5644c3245440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91b3e00640 1 -- 172.21.15.62:0/3454218124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c3240910 msgr2=0x5644c3251950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91b3e00640 1 --2- 172.21.15.62:0/3454218124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c3240910 0x5644c3251950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91b3e00640 1 -- 172.21.15.62:0/3454218124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c323ff20 msgr2=0x5644c3245440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91b3e00640 1 --2- 172.21.15.62:0/3454218124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c323ff20 0x5644c3245440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91b3e00640 1 -- 172.21.15.62:0/3454218124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c32411a0 con 0x5644c3245a80 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91b3e00640 1 --2- 172.21.15.62:0/3454218124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3245a80 0x5644c3235930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0325a0 tx=0x7f91ac005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d240f140023ce4a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.357+0000 7f91a3e00640 1 -- 172.21.15.62:0/3454218124 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac05c070 con 0x5644c3245a80 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.358+0000 7f91a3e00640 1 -- 172.21.15.62:0/3454218124 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f91ac057350 con 0x5644c3245a80 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.358+0000 7f91a3e00640 1 -- 172.21.15.62:0/3454218124 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac039040 con 0x5644c3245a80 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.358+0000 7f91bd1ba740 1 -- 172.21.15.62:0/3454218124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3245a80 msgr2=0x5644c3235930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.358+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/3454218124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3245a80 0x5644c3235930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0325a0 tx=0x7f91ac005ef0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.358+0000 7f91bd1ba740 1 -- 172.21.15.62:0/3454218124 shutdown_connections 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.358+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/3454218124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3245a80 0x5644c3235930 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.358+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/3454218124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c3240910 0x5644c3251950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.358+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/3454218124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c323ff20 0x5644c3245440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.358+0000 7f91bd1ba740 1 -- 172.21.15.62:0/3454218124 >> 172.21.15.62:0/3454218124 conn(0x5644c3237080 msgr2=0x5644c3237460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.358+0000 7f91bd1ba740 1 -- 172.21.15.62:0/3454218124 shutdown_connections 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.359+0000 7f91bd1ba740 1 -- 172.21.15.62:0/3454218124 wait complete. 2024-04-02T14:43:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.359+0000 7f91bd1ba740 1 -- start start 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.359+0000 7f91bd1ba740 1 Processor -- start 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.359+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 0x5644c324fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c32500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91bd1ba740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3250750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5644c3258590 con 0x5644c3240910 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5644c3257ec0 con 0x5644c3245a80 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91bd1ba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5644c3258c60 con 0x5644c323ff20 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91bd1ba740 10 client.0 _ll_register_callbacks cb 0x7f919c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c32500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c32500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46792/0 (socket says 172.21.15.62:46792) 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91abe00640 1 -- 172.21.15.62:0/1316740599 learned_addr learned my addr 172.21.15.62:0/1316740599 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 0x5644c324fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91b3e00640 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3250750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91b3e00640 1 -- 172.21.15.62:0/1316740599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 msgr2=0x5644c3250750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91b3e00640 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3250750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91b3e00640 1 -- 172.21.15.62:0/1316740599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 msgr2=0x5644c32500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91b3e00640 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c32500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91b3e00640 1 -- 172.21.15.62:0/1316740599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c3250f80 con 0x5644c323ff20 2024-04-02T14:43:49.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.360+0000 7f91abe00640 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c32500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:49.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.361+0000 7f91b3e00640 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 0x5644c324fb90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91ac002230 tx=0x7f91ac032670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1d8b7144e572d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.361+0000 7f91a1600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:43:49.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.361+0000 7f91a1600640 1 -- 172.21.15.62:0/1316740599 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac05c040 con 0x5644c323ff20 2024-04-02T14:43:49.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.361+0000 7f91a1600640 1 -- 172.21.15.62:0/1316740599 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f91ac032870 con 0x5644c323ff20 2024-04-02T14:43:49.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.361+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1316740599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91ac057350 con 0x5644c323ff20 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.361+0000 7f91a1600640 1 -- 172.21.15.62:0/1316740599 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f91ac039040 con 0x5644c323ff20 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.361+0000 7f91bd1ba740 10 client.25234 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.361+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1316740599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644c32bd5f0 con 0x5644c323ff20 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.361+0000 7f91a1600640 1 -- 172.21.15.62:0/1316740599 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f91ac057350 con 0x5644c323ff20 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.361+0000 7f91a1600640 1 -- 172.21.15.62:0/1316740599 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f91ac057ce0 con 0x5644c323ff20 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.362+0000 7f91bd1ba740 10 client.25234 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.362+0000 7f91bd1ba740 10 client.25234 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.362+0000 7f91bd1ba740 20 client.25234 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.362+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5644c32c1aa0 0x5644c32c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.362+0000 7f91bd1ba740 4 client.25234 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.362+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1316740599 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "status"}) -- 0x5644c31a88f0 con 0x5644c32c1aa0 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.362+0000 7f91abe00640 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5644c32c1aa0 0x5644c32c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.362+0000 7f91abe00640 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5644c32c1aa0 0x5644c32c07b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3d58dc9c34c200c server_cookie=dfcb6365c4adf41 in_seq=0 out_seq=0 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.362+0000 7f91a1600640 10 client.25234 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91a1600640 1 -- 172.21.15.62:0/1316740599 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5644c31a88f0 con 0x5644c32c1aa0 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91a1600640 10 client.25234 handle_command_reply: tid=0 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 1 client.25234 shutdown 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1316740599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 msgr2=0x5644c324fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 0x5644c324fb90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91ac002230 tx=0x7f91ac032670 comp rx=0 tx=0).stop 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1316740599 shutdown_connections 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5644c32c1aa0 0x5644c32c07b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644c3245a80 0x5644c3250750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91a1600640 0 client.25234 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c3240910 0x5644c32500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 1 --2- 172.21.15.62:0/1316740599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644c323ff20 0x5644c324fb90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1316740599 >> 172.21.15.62:0/1316740599 conn(0x5644c3237080 msgr2=0x5644c3235930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1316740599 shutdown_connections 2024-04-02T14:43:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 1 -- 172.21.15.62:0/1316740599 wait complete. 2024-04-02T14:43:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.363+0000 7f91bd1ba740 20 client.25234 trim_cache size 0 max 16384 2024-04-02T14:43:49.369 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:43:49.369 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:43:49.369 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-04-02T14:43:49.369 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24565, 2024-04-02T14:43:49.369 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-02T14:43:49.369 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-02T14:43:49.369 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-02T14:43:49.370 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 143.32799380599999, 2024-04-02T14:43:49.370 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-04-02T14:43:49.370 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-04-02T14:43:49.370 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-04-02T14:43:49.370 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 156.59601467499999 2024-04-02T14:43:49.370 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 -- 172.21.15.62:0/1972940613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f91980a3c30 msgr2=0x7f91980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f91980a3c30 0x7f91980a60f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f91a409c810 tx=0x7f91a4018520 comp rx=0 tx=0).stop 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 -- 172.21.15.62:0/1972940613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b41024d0 msgr2=0x7f91b4197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b41024d0 0x7f91b4197da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f91a400e9f0 tx=0x7f91a400eeb0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 -- 172.21.15.62:0/1972940613 shutdown_connections 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f91980a3c30 0x7f91980a60f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b4107730 0x7f91b419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b4104e00 0x7f91b41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 --2- 172.21.15.62:0/1972940613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b41024d0 0x7f91b4197da0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 -- 172.21.15.62:0/1972940613 >> 172.21.15.62:0/1972940613 conn(0x7f91b40fbe80 msgr2=0x7f91b4102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 -- 172.21.15.62:0/1972940613 shutdown_connections 2024-04-02T14:43:49.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.364+0000 7f91b8e00640 1 -- 172.21.15.62:0/1972940613 wait complete. 2024-04-02T14:43:49.380 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T14:43:49.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.591+0000 7ff0f3200640 1 -- 172.21.15.62:0/3270390798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0ec1057d0 msgr2=0x7ff0ec10c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.591+0000 7ff0f3200640 1 --2- 172.21.15.62:0/3270390798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0ec1057d0 0x7ff0ec10c060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009ca0 tx=0x7ff0e802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.591+0000 7ff0f3200640 1 -- 172.21.15.62:0/3270390798 shutdown_connections 2024-04-02T14:43:49.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.591+0000 7ff0f3200640 1 --2- 172.21.15.62:0/3270390798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0ec1057d0 0x7ff0ec10c060 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.591+0000 7ff0f3200640 1 --2- 172.21.15.62:0/3270390798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0ec104e10 0x7ff0ec105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.591+0000 7ff0f3200640 1 --2- 172.21.15.62:0/3270390798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0ec103c20 0x7ff0ec104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.591+0000 7ff0f3200640 1 -- 172.21.15.62:0/3270390798 >> 172.21.15.62:0/3270390798 conn(0x7ff0ec0ff1b0 msgr2=0x7ff0ec1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.591+0000 7ff0f3200640 1 -- 172.21.15.62:0/3270390798 shutdown_connections 2024-04-02T14:43:49.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.591+0000 7ff0f3200640 1 -- 172.21.15.62:0/3270390798 wait complete. 2024-04-02T14:43:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.592+0000 7ff0f3200640 1 Processor -- start 2024-04-02T14:43:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f3200640 1 -- start start 2024-04-02T14:43:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0ec103c20 0x7ff0ec199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0ec104e10 0x7ff0ec19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f3200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0ec1057d0 0x7ff0ec1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec10f9c0 con 0x7ff0ec103c20 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f3200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec10f840 con 0x7ff0ec1057d0 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec10fb40 con 0x7ff0ec104e10 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0ec104e10 0x7ff0ec19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0ec1057d0 0x7ff0ec1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0ec103c20 0x7ff0ec199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0ec104e10 0x7ff0ec19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35736/0 (socket says 172.21.15.62:35736) 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.593+0000 7ff0f1400640 1 -- 172.21.15.62:0/3348352132 learned_addr learned my addr 172.21.15.62:0/3348352132 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.594+0000 7ff0f2800640 1 -- 172.21.15.62:0/3348352132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0ec104e10 msgr2=0x7ff0ec19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.594+0000 7ff0f2800640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0ec104e10 0x7ff0ec19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.594+0000 7ff0f2800640 1 -- 172.21.15.62:0/3348352132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0ec103c20 msgr2=0x7ff0ec199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.594+0000 7ff0f2800640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0ec103c20 0x7ff0ec199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.594+0000 7ff0f2800640 1 -- 172.21.15.62:0/3348352132 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec1a1c10 con 0x7ff0ec1057d0 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.594+0000 7ff0f2800640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0ec1057d0 0x7ff0ec1a15a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8037040 tx=0x7ff0e8034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.594+0000 7ff0e6a00640 1 -- 172.21.15.62:0/3348352132 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8039040 con 0x7ff0ec1057d0 2024-04-02T14:43:49.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.594+0000 7ff0e6a00640 1 -- 172.21.15.62:0/3348352132 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0e8044070 con 0x7ff0ec1057d0 2024-04-02T14:43:49.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.595+0000 7ff0f3200640 1 -- 172.21.15.62:0/3348352132 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec1a1ea0 con 0x7ff0ec1057d0 2024-04-02T14:43:49.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.595+0000 7ff0e6a00640 1 -- 172.21.15.62:0/3348352132 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e80043a0 con 0x7ff0ec1057d0 2024-04-02T14:43:49.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.595+0000 7ff0f3200640 1 -- 172.21.15.62:0/3348352132 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec1a2360 con 0x7ff0ec1057d0 2024-04-02T14:43:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.598+0000 7ff0e6a00640 1 -- 172.21.15.62:0/3348352132 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff0e8004540 con 0x7ff0ec1057d0 2024-04-02T14:43:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.598+0000 7ff0f757a740 1 Processor -- start 2024-04-02T14:43:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.598+0000 7ff0f757a740 1 -- start start 2024-04-02T14:43:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.598+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc6df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.598+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6e500 0x56499bc73000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.598+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc75b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.598+0000 7ff0f757a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56499bbd3060 con 0x56499bc6db10 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.598+0000 7ff0f757a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56499bb7b560 con 0x56499bc6e500 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.598+0000 7ff0f757a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56499bbcf010 con 0x56499bc73670 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0f1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc6df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0f1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc6df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46824/0 (socket says 172.21.15.62:46824) 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0f1400640 1 -- 172.21.15.62:0/3593982957 learned_addr learned my addr 172.21.15.62:0/3593982957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/3593982957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6e500 0x56499bc73000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0f1400640 1 --2- 172.21.15.62:0/3593982957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc75b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0f1400640 1 -- 172.21.15.62:0/3593982957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 msgr2=0x56499bc75b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0f1400640 1 --2- 172.21.15.62:0/3593982957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc75b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0f1400640 1 -- 172.21.15.62:0/3593982957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6e500 msgr2=0x56499bc73000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0f1400640 1 --2- 172.21.15.62:0/3593982957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6e500 0x56499bc73000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0f1400640 1 -- 172.21.15.62:0/3593982957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499bb9dfd0 con 0x56499bc6db10 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0f1400640 1 --2- 172.21.15.62:0/3593982957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc6df30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc013630 tx=0x7ff0dc009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57258f20691a60e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.599+0000 7ff0e6a00640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0d40a3c30 0x7ff0d40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.600+0000 7ff0db000640 1 -- 172.21.15.62:0/3593982957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00f600 con 0x56499bc6db10 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.600+0000 7ff0e6a00640 1 -- 172.21.15.62:0/3348352132 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7ff0e80f2920 con 0x7ff0ec1057d0 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.600+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0d40a3c30 0x7ff0d40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.600+0000 7ff0db000640 1 -- 172.21.15.62:0/3593982957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0dc00fca0 con 0x56499bc6db10 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.600+0000 7ff0db000640 1 -- 172.21.15.62:0/3593982957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc010c70 con 0x56499bc6db10 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.600+0000 7ff0f757a740 1 -- 172.21.15.62:0/3593982957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 msgr2=0x56499bc6df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.600+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3593982957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc6df30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc013630 tx=0x7ff0dc009560 comp rx=0 tx=0).stop 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.600+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0d40a3c30 0x7ff0d40a60f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x56499bc6e900 tx=0x7ff0e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.601+0000 7ff0f757a740 1 -- 172.21.15.62:0/3593982957 shutdown_connections 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.601+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3593982957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc75b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.601+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3593982957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6e500 0x56499bc73000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.601+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3593982957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc6df30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.601+0000 7ff0f757a740 1 -- 172.21.15.62:0/3593982957 >> 172.21.15.62:0/3593982957 conn(0x56499bc69080 msgr2=0x56499bc69480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.601+0000 7ff0f757a740 1 -- 172.21.15.62:0/3593982957 shutdown_connections 2024-04-02T14:43:49.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.601+0000 7ff0f757a740 1 -- 172.21.15.62:0/3593982957 wait complete. 2024-04-02T14:43:49.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.601+0000 7ff0f757a740 1 -- start start 2024-04-02T14:43:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.601+0000 7ff0f757a740 1 Processor -- start 2024-04-02T14:43:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6db10 0x56499bc780c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6e500 0x56499bc78600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc7f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f757a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56499bc76620 con 0x56499bc6e500 2024-04-02T14:43:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f757a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56499bb8e920 con 0x56499bc6db10 2024-04-02T14:43:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f757a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56499bc767a0 con 0x56499bc73670 2024-04-02T14:43:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f757a740 10 client.0 _ll_register_callbacks cb 0x56499bb8e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6e500 0x56499bc78600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f2800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc7f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6db10 0x56499bc780c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f2800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc7f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35750/0 (socket says 172.21.15.62:35750) 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f2800640 1 -- 172.21.15.62:0/1928882688 learned_addr learned my addr 172.21.15.62:0/1928882688 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6db10 0x56499bc780c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:33216/0 (socket says 172.21.15.62:33216) 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.602+0000 7ff0f2800640 1 -- 172.21.15.62:0/1928882688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6db10 msgr2=0x56499bc780c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0f2800640 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6db10 0x56499bc780c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0f2800640 1 -- 172.21.15.62:0/1928882688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6e500 msgr2=0x56499bc78600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0f2800640 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6e500 0x56499bc78600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0f2800640 1 -- 172.21.15.62:0/1928882688 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499b8e72d0 con 0x56499bc73670 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6db10 0x56499bc780c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0f2800640 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc7f640 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80bb030 tx=0x7ff0e80be7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ebbd35aff75596 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0cf400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0cf400640 1 -- 172.21.15.62:0/1928882688 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8032970 con 0x56499bc73670 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0cf400640 1 -- 172.21.15.62:0/1928882688 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7ff0e80bee30 con 0x56499bc73670 2024-04-02T14:43:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0f757a740 1 -- 172.21.15.62:0/1928882688 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0dc00fca0 con 0x56499bc73670 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0cf400640 1 -- 172.21.15.62:0/1928882688 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0e8004830 con 0x56499bc73670 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0f757a740 10 client.25246 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.603+0000 7ff0f757a740 1 -- 172.21.15.62:0/1928882688 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56499b9b16b0 con 0x56499bc73670 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.604+0000 7ff0cf400640 1 -- 172.21.15.62:0/1928882688 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0dc00fca0 con 0x56499bc73670 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.604+0000 7ff0cf400640 1 -- 172.21.15.62:0/1928882688 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff0e803c030 con 0x56499bc73670 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.604+0000 7ff0f757a740 10 client.25246 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.604+0000 7ff0f757a740 10 client.25246 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.604+0000 7ff0f757a740 20 client.25246 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.604+0000 7ff0f757a740 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x56499bc84ec0 0x56499bc86330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.604+0000 7ff0f757a740 4 client.25246 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.604+0000 7ff0f757a740 1 -- 172.21.15.62:0/1928882688 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56499bbe05a0 con 0x56499bc84ec0 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.604+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x56499bc84ec0 0x56499bc86330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.605+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x56499bc84ec0 0x56499bc86330 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb4321a19ba0be2b server_cookie=dcbdd9bc754fd8e7 in_seq=0 out_seq=0 2024-04-02T14:43:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.605+0000 7ff0cf400640 10 client.25246 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0cf400640 1 -- 172.21.15.62:0/1928882688 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56499bbe05a0 con 0x56499bc84ec0 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0cf400640 10 client.25246 handle_command_reply: tid=0 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 1 client.25246 shutdown 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 1 -- 172.21.15.62:0/1928882688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 msgr2=0x56499bc7f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc7f640 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80bb030 tx=0x7ff0e80be7d0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 1 -- 172.21.15.62:0/1928882688 shutdown_connections 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x56499bc84ec0 0x56499bc86330 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc7f640 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6e500 0x56499bc78600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 1 --2- 172.21.15.62:0/1928882688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6db10 0x56499bc780c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0cf400640 0 client.25246 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 1 -- 172.21.15.62:0/1928882688 >> 172.21.15.62:0/1928882688 conn(0x56499bc69080 msgr2=0x56499bc681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 1 -- 172.21.15.62:0/1928882688 shutdown_connections 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 1 -- 172.21.15.62:0/1928882688 wait complete. 2024-04-02T14:43:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.606+0000 7ff0f757a740 20 client.25246 trim_cache size 0 max 16384 2024-04-02T14:43:49.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.626+0000 7ff0f757a740 1 Processor -- start 2024-04-02T14:43:49.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.626+0000 7ff0f757a740 1 -- start start 2024-04-02T14:43:49.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc69460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6e500 0x56499bc7f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc83b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f757a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56499bc84670 con 0x56499bc6db10 2024-04-02T14:43:49.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f757a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56499bc84d40 con 0x56499bc6e500 2024-04-02T14:43:49.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f757a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56499bc85410 con 0x56499bc73670 2024-04-02T14:43:49.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc69460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc69460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46844/0 (socket says 172.21.15.62:46844) 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f1e00640 1 -- 172.21.15.62:0/3025868496 learned_addr learned my addr 172.21.15.62:0/3025868496 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f1400640 1 --2- 172.21.15.62:0/3025868496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6e500 0x56499bc7f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f2800640 1 --2- 172.21.15.62:0/3025868496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc83b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f1e00640 1 -- 172.21.15.62:0/3025868496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 msgr2=0x56499bc83b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/3025868496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc83b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f1e00640 1 -- 172.21.15.62:0/3025868496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6e500 msgr2=0x56499bc7f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/3025868496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6e500 0x56499bc7f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.627+0000 7ff0f1e00640 1 -- 172.21.15.62:0/3025868496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499bc78970 con 0x56499bc6db10 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.628+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/3025868496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc69460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x56499bc6ef70 tx=0x7ff0e001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb78e4570de1544 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.628+0000 7ff0da600640 1 -- 172.21.15.62:0/3025868496 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0003dc0 con 0x56499bc6db10 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.628+0000 7ff0da600640 1 -- 172.21.15.62:0/3025868496 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0e002b920 con 0x56499bc6db10 2024-04-02T14:43:49.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.628+0000 7ff0da600640 1 -- 172.21.15.62:0/3025868496 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00049c0 con 0x56499bc6db10 2024-04-02T14:43:49.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.629+0000 7ff0f757a740 1 -- 172.21.15.62:0/3025868496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 msgr2=0x56499bc69460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.629+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3025868496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc69460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x56499bc6ef70 tx=0x7ff0e001a000 comp rx=0 tx=0).stop 2024-04-02T14:43:49.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.629+0000 7ff0f757a740 1 -- 172.21.15.62:0/3025868496 shutdown_connections 2024-04-02T14:43:49.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.629+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3025868496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc73670 0x56499bc83b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.629+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3025868496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc6e500 0x56499bc7f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.629+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3025868496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc69460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.629+0000 7ff0f757a740 1 -- 172.21.15.62:0/3025868496 >> 172.21.15.62:0/3025868496 conn(0x56499bc69080 msgr2=0x56499bc76ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.629+0000 7ff0f757a740 1 -- 172.21.15.62:0/3025868496 shutdown_connections 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.629+0000 7ff0f757a740 1 -- 172.21.15.62:0/3025868496 wait complete. 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.629+0000 7ff0f757a740 1 -- start start 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f757a740 1 Processor -- start 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc7d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc6e500 0x56499bce6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f757a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc73670 0x56499bce93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f757a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56499bc861b0 con 0x56499bc6db10 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f757a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56499bc85ae0 con 0x56499bc73670 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f757a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56499bc86880 con 0x56499bc6e500 2024-04-02T14:43:49.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f757a740 10 client.0 _ll_register_callbacks cb 0x7ff0e80bae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:43:49.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc7d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc7d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46858/0 (socket says 172.21.15.62:46858) 2024-04-02T14:43:49.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f1e00640 1 -- 172.21.15.62:0/3378130603 learned_addr learned my addr 172.21.15.62:0/3378130603 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f1400640 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc6e500 0x56499bce6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.630+0000 7ff0f2800640 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc73670 0x56499bce93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.631+0000 7ff0f1e00640 1 -- 172.21.15.62:0/3378130603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc6e500 msgr2=0x56499bce6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.631+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc6e500 0x56499bce6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.631+0000 7ff0f1e00640 1 -- 172.21.15.62:0/3378130603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc73670 msgr2=0x56499bce93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:43:49.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.631+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc73670 0x56499bce93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.631+0000 7ff0f1e00640 1 -- 172.21.15.62:0/3378130603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499bce9910 con 0x56499bc6db10 2024-04-02T14:43:49.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.631+0000 7ff0f1e00640 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc7d5f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0000c00 tx=0x7ff0e0009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd589f8f154863fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.631+0000 7ff0cf400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:43:49.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.631+0000 7ff0cf400640 1 -- 172.21.15.62:0/3378130603 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0039030 con 0x56499bc6db10 2024-04-02T14:43:49.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.631+0000 7ff0cf400640 1 -- 172.21.15.62:0/3378130603 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7ff0e0030070 con 0x56499bc6db10 2024-04-02T14:43:49.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.631+0000 7ff0f757a740 1 -- 172.21.15.62:0/3378130603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56499bbda8f0 con 0x56499bc6db10 2024-04-02T14:43:49.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.632+0000 7ff0cf400640 1 -- 172.21.15.62:0/3378130603 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0e002b420 con 0x56499bc6db10 2024-04-02T14:43:49.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.632+0000 7ff0f757a740 10 client.15153 fetch_fsmap learned FSMap version 27 2024-04-02T14:43:49.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.632+0000 7ff0f757a740 1 -- 172.21.15.62:0/3378130603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56499bceabf0 con 0x56499bc6db10 2024-04-02T14:43:49.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.632+0000 7ff0cf400640 1 -- 172.21.15.62:0/3378130603 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x56499bbda8f0 con 0x56499bc6db10 2024-04-02T14:43:49.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.632+0000 7ff0cf400640 1 -- 172.21.15.62:0/3378130603 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff0e00404d0 con 0x56499bc6db10 2024-04-02T14:43:49.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.632+0000 7ff0f757a740 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T14:43:49.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.632+0000 7ff0f757a740 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.632+0000 7ff0f757a740 20 client.15153 populate_metadata read hostname 'smithi062' 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.632+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x56499bcef490 0x56499bcee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.632+0000 7ff0f757a740 4 client.15153 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.632+0000 7ff0f757a740 1 -- 172.21.15.62:0/3378130603 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "status"}) -- 0x56499bb587d0 con 0x56499bcef490 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.633+0000 7ff0f1400640 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x56499bcef490 0x56499bcee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.633+0000 7ff0f1400640 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x56499bcef490 0x56499bcee1a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63fe8fb5ab2119c7 server_cookie=4b593bd7f16c2853 in_seq=0 out_seq=0 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.633+0000 7ff0cf400640 10 client.15153 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.633+0000 7ff0cf400640 1 -- 172.21.15.62:0/3378130603 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56499bb587d0 con 0x56499bcef490 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.633+0000 7ff0cf400640 10 client.15153 handle_command_reply: tid=0 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.633+0000 7ff0f757a740 1 client.15153 shutdown 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.633+0000 7ff0f757a740 1 -- 172.21.15.62:0/3378130603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 msgr2=0x56499bc7d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.633+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc7d5f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0000c00 tx=0x7ff0e0009f60 comp rx=0 tx=0).stop 2024-04-02T14:43:49.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f757a740 1 -- 172.21.15.62:0/3378130603 shutdown_connections 2024-04-02T14:43:49.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x56499bcef490 0x56499bcee1a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56499bc73670 0x56499bce93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56499bc6e500 0x56499bce6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f757a740 1 --2- 172.21.15.62:0/3378130603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56499bc6db10 0x56499bc7d5f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0cf400640 0 client.15153 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:43:49.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f757a740 1 -- 172.21.15.62:0/3378130603 >> 172.21.15.62:0/3378130603 conn(0x56499bc69080 msgr2=0x56499bc69460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f757a740 1 -- 172.21.15.62:0/3378130603 shutdown_connections 2024-04-02T14:43:49.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f757a740 1 -- 172.21.15.62:0/3378130603 wait complete. 2024-04-02T14:43:49.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f757a740 20 client.15153 trim_cache size 0 max 16384 2024-04-02T14:43:49.636 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e86b65e-f0fd-11ee-b647-cb9ed24678a4", 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24584, 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 146.62764762099999, 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 154.27857038600001 2024-04-02T14:43:49.637 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:43:49.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f3200640 1 -- 172.21.15.62:0/3348352132 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0d40a3c30 msgr2=0x7ff0d40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f3200640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0d40a3c30 0x7ff0d40a60f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x56499bc6e900 tx=0x7ff0e0005f90 comp rx=0 tx=0).stop 2024-04-02T14:43:49.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f3200640 1 -- 172.21.15.62:0/3348352132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0ec1057d0 msgr2=0x7ff0ec1a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.634+0000 7ff0f3200640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0ec1057d0 0x7ff0ec1a15a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8037040 tx=0x7ff0e8034670 comp rx=0 tx=0).stop 2024-04-02T14:43:49.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.635+0000 7ff0f3200640 1 -- 172.21.15.62:0/3348352132 shutdown_connections 2024-04-02T14:43:49.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.635+0000 7ff0f3200640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7ff0d40a3c30 0x7ff0d40a60f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.635+0000 7ff0f3200640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0ec1057d0 0x7ff0ec1a15a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.635+0000 7ff0f3200640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0ec104e10 0x7ff0ec19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.635+0000 7ff0f3200640 1 --2- 172.21.15.62:0/3348352132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0ec103c20 0x7ff0ec199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.635+0000 7ff0f3200640 1 -- 172.21.15.62:0/3348352132 >> 172.21.15.62:0/3348352132 conn(0x7ff0ec0ff1b0 msgr2=0x7ff0ec108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.635+0000 7ff0f3200640 1 -- 172.21.15.62:0/3348352132 shutdown_connections 2024-04-02T14:43:49.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.635+0000 7ff0f3200640 1 -- 172.21.15.62:0/3348352132 wait complete. 2024-04-02T14:43:49.650 INFO:teuthology.run_tasks:Running task sequential... 2024-04-02T14:43:49.660 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-02T14:43:49.660 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-02T14:43:49.660 INFO:teuthology.task.print:Enabling mgr modules 2024-04-02T14:43:49.660 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-02T14:43:49.661 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T14:43:49.661 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-04-02T14:43:49.661 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-02T14:43:49.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.881+0000 7f8e1d400640 1 -- 172.21.15.62:0/2619647505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1810f050 msgr2=0x7f8e18111440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.881+0000 7f8e1d400640 1 --2- 172.21.15.62:0/2619647505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1810f050 0x7f8e18111440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009be0 tx=0x7f8e1002bef0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.881+0000 7f8e1d400640 1 -- 172.21.15.62:0/2619647505 shutdown_connections 2024-04-02T14:43:49.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.881+0000 7f8e1d400640 1 --2- 172.21.15.62:0/2619647505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1810f050 0x7f8e18111440 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.881+0000 7f8e1d400640 1 --2- 172.21.15.62:0/2619647505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e1810c720 0x7f8e1810eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.881+0000 7f8e1d400640 1 --2- 172.21.15.62:0/2619647505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e1806a900 0x7f8e18108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.881+0000 7f8e1d400640 1 -- 172.21.15.62:0/2619647505 >> 172.21.15.62:0/2619647505 conn(0x7f8e1806ed00 msgr2=0x7f8e18071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:49.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.881+0000 7f8e1d400640 1 -- 172.21.15.62:0/2619647505 shutdown_connections 2024-04-02T14:43:49.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.881+0000 7f8e1d400640 1 -- 172.21.15.62:0/2619647505 wait complete. 2024-04-02T14:43:49.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.882+0000 7f8e1d400640 1 Processor -- start 2024-04-02T14:43:49.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.882+0000 7f8e1d400640 1 -- start start 2024-04-02T14:43:49.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e1d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1806a900 0x7f8e181a2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e1d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e1810c720 0x7f8e181a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e1d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e1810f050 0x7f8e181a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e1d400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18115010 con 0x7f8e1810f050 2024-04-02T14:43:49.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e1d400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18114e90 con 0x7f8e1810c720 2024-04-02T14:43:49.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e1d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18115190 con 0x7f8e1806a900 2024-04-02T14:43:49.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e16800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1806a900 0x7f8e181a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e17200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e1810f050 0x7f8e181a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e16800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1806a900 0x7f8e181a2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35766/0 (socket says 172.21.15.62:35766) 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e16800640 1 -- 172.21.15.62:0/1661702305 learned_addr learned my addr 172.21.15.62:0/1661702305 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e15e00640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e1810c720 0x7f8e181a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e16800640 1 -- 172.21.15.62:0/1661702305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e1810c720 msgr2=0x7f8e181a2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e16800640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e1810c720 0x7f8e181a2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e16800640 1 -- 172.21.15.62:0/1661702305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e1810f050 msgr2=0x7f8e181a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e16800640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e1810f050 0x7f8e181a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e16800640 1 -- 172.21.15.62:0/1661702305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e181aa450 con 0x7f8e1806a900 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.883+0000 7f8e15e00640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e1810c720 0x7f8e181a2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.884+0000 7f8e16800640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1806a900 0x7f8e181a2830 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800e9c0 tx=0x7f8e0800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.884+0000 7f8e0f400640 1 -- 172.21.15.62:0/1661702305 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0800cd80 con 0x7f8e1806a900 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.884+0000 7f8e0f400640 1 -- 172.21.15.62:0/1661702305 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8e080035f0 con 0x7f8e1806a900 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.884+0000 7f8e0f400640 1 -- 172.21.15.62:0/1661702305 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08010670 con 0x7f8e1806a900 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.884+0000 7f8e1d400640 1 -- 172.21.15.62:0/1661702305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e181aa740 con 0x7f8e1806a900 2024-04-02T14:43:49.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.884+0000 7f8e1d400640 1 -- 172.21.15.62:0/1661702305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e181aac90 con 0x7f8e1806a900 2024-04-02T14:43:49.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.885+0000 7f8e1d400640 1 -- 172.21.15.62:0/1661702305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e181aaf50 con 0x7f8e1806a900 2024-04-02T14:43:49.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.887+0000 7f8e0f400640 1 -- 172.21.15.62:0/1661702305 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8e080037a0 con 0x7f8e1806a900 2024-04-02T14:43:49.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.889+0000 7f8e0f400640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8dfc0a3cd0 0x7f8dfc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:49.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.889+0000 7f8e0f400640 1 -- 172.21.15.62:0/1661702305 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f8e080d07d0 con 0x7f8e1806a900 2024-04-02T14:43:49.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.889+0000 7f8e15e00640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8dfc0a3cd0 0x7f8dfc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:49.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.889+0000 7f8e15e00640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8dfc0a3cd0 0x7f8dfc0a6190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e181a3d40 tx=0x7f8e04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:49.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:49.893+0000 7f8e0f400640 1 -- 172.21.15.62:0/1661702305 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8e0809d180 con 0x7f8e1806a900 2024-04-02T14:43:50.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.156+0000 7f8e1d400640 1 -- 172.21.15.62:0/1661702305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f8e181ab120 con 0x7f8e1806a900 2024-04-02T14:43:50.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.550+0000 7f8e0f400640 1 -- 172.21.15.62:0/1661702305 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f8e080990f0 con 0x7f8e1806a900 2024-04-02T14:43:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.552+0000 7f8e0f400640 1 -- 172.21.15.62:0/1661702305 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f8e080652e0 con 0x7f8e1806a900 2024-04-02T14:43:50.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.555+0000 7f8e1d400640 1 -- 172.21.15.62:0/1661702305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8dfc0a3cd0 msgr2=0x7f8dfc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:50.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.555+0000 7f8e1d400640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8dfc0a3cd0 0x7f8dfc0a6190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e181a3d40 tx=0x7f8e04009260 comp rx=0 tx=0).stop 2024-04-02T14:43:50.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.555+0000 7f8e1d400640 1 -- 172.21.15.62:0/1661702305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1806a900 msgr2=0x7f8e181a2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:50.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.555+0000 7f8e1d400640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1806a900 0x7f8e181a2830 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800e9c0 tx=0x7f8e0800ee80 comp rx=0 tx=0).stop 2024-04-02T14:43:50.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.555+0000 7f8e1d400640 1 -- 172.21.15.62:0/1661702305 shutdown_connections 2024-04-02T14:43:50.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.555+0000 7f8e1d400640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f8dfc0a3cd0 0x7f8dfc0a6190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:50.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.555+0000 7f8e1d400640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e1810f050 0x7f8e181a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:50.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.556+0000 7f8e1d400640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e1810c720 0x7f8e181a2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:50.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.556+0000 7f8e1d400640 1 --2- 172.21.15.62:0/1661702305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1806a900 0x7f8e181a2830 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:50.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.556+0000 7f8e1d400640 1 -- 172.21.15.62:0/1661702305 >> 172.21.15.62:0/1661702305 conn(0x7f8e1806ed00 msgr2=0x7f8e18109b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:50.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.556+0000 7f8e1d400640 1 -- 172.21.15.62:0/1661702305 shutdown_connections 2024-04-02T14:43:50.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.556+0000 7f8e1d400640 1 -- 172.21.15.62:0/1661702305 wait complete. 2024-04-02T14:43:50.572 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-02T14:43:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.783+0000 7f1f51600640 1 -- 172.21.15.62:0/1905062856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4c105900 msgr2=0x7f1f4c10c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.783+0000 7f1f51600640 1 --2- 172.21.15.62:0/1905062856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4c105900 0x7f1f4c10c190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009ca0 tx=0x7f1f4402c0f0 comp rx=0 tx=0).stop 2024-04-02T14:43:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.783+0000 7f1f51600640 1 -- 172.21.15.62:0/1905062856 shutdown_connections 2024-04-02T14:43:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.783+0000 7f1f51600640 1 --2- 172.21.15.62:0/1905062856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4c105900 0x7f1f4c10c190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.783+0000 7f1f51600640 1 --2- 172.21.15.62:0/1905062856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4c104f40 0x7f1f4c1053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.783+0000 7f1f51600640 1 --2- 172.21.15.62:0/1905062856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c103d50 0x7f1f4c104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.783+0000 7f1f51600640 1 -- 172.21.15.62:0/1905062856 >> 172.21.15.62:0/1905062856 conn(0x7f1f4c0ff2c0 msgr2=0x7f1f4c101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.783+0000 7f1f51600640 1 -- 172.21.15.62:0/1905062856 shutdown_connections 2024-04-02T14:43:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.783+0000 7f1f51600640 1 -- 172.21.15.62:0/1905062856 wait complete. 2024-04-02T14:43:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.784+0000 7f1f51600640 1 Processor -- start 2024-04-02T14:43:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.784+0000 7f1f51600640 1 -- start start 2024-04-02T14:43:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.784+0000 7f1f51600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c103d50 0x7f1f4c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.784+0000 7f1f51600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4c104f40 0x7f1f4c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.784+0000 7f1f51600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4c105900 0x7f1f4c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.784+0000 7f1f51600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c10fb00 con 0x7f1f4c103d50 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.784+0000 7f1f51600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c10f980 con 0x7f1f4c105900 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.784+0000 7f1f51600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c10fc80 con 0x7f1f4c104f40 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4c104f40 0x7f1f4c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c103d50 0x7f1f4c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4c104f40 0x7f1f4c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:35796/0 (socket says 172.21.15.62:35796) 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4a000640 1 -- 172.21.15.62:0/3909349966 learned_addr learned my addr 172.21.15.62:0/3909349966 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4b400640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4c105900 0x7f1f4c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4a000640 1 -- 172.21.15.62:0/3909349966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4c105900 msgr2=0x7f1f4c1a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4a000640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4c105900 0x7f1f4c1a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4a000640 1 -- 172.21.15.62:0/3909349966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c103d50 msgr2=0x7f1f4c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4a000640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c103d50 0x7f1f4c19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4a000640 1 -- 172.21.15.62:0/3909349966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c1a1c20 con 0x7f1f4c104f40 2024-04-02T14:43:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4b400640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4c105900 0x7f1f4c1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:43:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f4a000640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4c104f40 0x7f1f4c19a540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800e9f0 tx=0x7f1f3800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.785+0000 7f1f43400640 1 -- 172.21.15.62:0/3909349966 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3800cd80 con 0x7f1f4c104f40 2024-04-02T14:43:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.786+0000 7f1f43400640 1 -- 172.21.15.62:0/3909349966 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1f38005520 con 0x7f1f4c104f40 2024-04-02T14:43:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.786+0000 7f1f43400640 1 -- 172.21.15.62:0/3909349966 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38010690 con 0x7f1f4c104f40 2024-04-02T14:43:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.786+0000 7f1f51600640 1 -- 172.21.15.62:0/3909349966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4c1a1f10 con 0x7f1f4c104f40 2024-04-02T14:43:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.786+0000 7f1f51600640 1 -- 172.21.15.62:0/3909349966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4c1a23d0 con 0x7f1f4c104f40 2024-04-02T14:43:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.787+0000 7f1f51600640 1 -- 172.21.15.62:0/3909349966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4c10afa0 con 0x7f1f4c104f40 2024-04-02T14:43:50.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.789+0000 7f1f43400640 1 -- 172.21.15.62:0/3909349966 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f1f380046c0 con 0x7f1f4c104f40 2024-04-02T14:43:50.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.790+0000 7f1f43400640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f300a3d20 0x7f1f300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:50.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.791+0000 7f1f4aa00640 1 -- 172.21.15.62:0/3909349966 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f300a3d20 msgr2=0x7f1f300a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1995176952 2024-04-02T14:43:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.791+0000 7f1f43400640 1 -- 172.21.15.62:0/3909349966 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f1f380d0820 con 0x7f1f4c104f40 2024-04-02T14:43:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.791+0000 7f1f4aa00640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f300a3d20 0x7f1f300a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T14:43:50.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.795+0000 7f1f43400640 1 -- 172.21.15.62:0/3909349966 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1f3809d1d0 con 0x7f1f4c104f40 2024-04-02T14:43:50.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:50 smithi062 ceph-mon[32116]: pgmap v395: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T14:43:50.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:50 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1661702305' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T14:43:50.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:50 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T14:43:50.840 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:50 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: ignoring --setuser ceph since I am not root 2024-04-02T14:43:50.840 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:50 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: ignoring --setgroup ceph since I am not root 2024-04-02T14:43:50.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:50 smithi113 ceph-mon[36135]: pgmap v395: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T14:43:50.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:50 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1661702305' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T14:43:50.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:50 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T14:43:50.867 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:50 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: ignoring --setuser ceph since I am not root 2024-04-02T14:43:50.868 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:50 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: ignoring --setgroup ceph since I am not root 2024-04-02T14:43:50.868 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:50 smithi113 ceph-mgr[37527]: -- 172.21.15.113:0/2968578270 <== mon.2 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc76cbe5a0 con 0x55dc76c92c00 2024-04-02T14:43:50.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.992+0000 7f1f4aa00640 1 -- 172.21.15.62:0/3909349966 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f300a3d20 msgr2=0x7f1f300a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1995176952 2024-04-02T14:43:50.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:50.992+0000 7f1f4aa00640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f300a3d20 0x7f1f300a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T14:43:51.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:50 smithi204 ceph-mon[35948]: pgmap v395: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T14:43:51.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:50 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1661702305' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T14:43:51.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:50 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T14:43:51.006 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:50 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: ignoring --setuser ceph since I am not root 2024-04-02T14:43:51.006 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:50 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: ignoring --setgroup ceph since I am not root 2024-04-02T14:43:51.007 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:50 smithi204 ceph-mgr[37688]: -- 172.21.15.204:0/1027237134 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557205c805a0 con 0x557205c55000 2024-04-02T14:43:51.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.032+0000 7f1f51600640 1 -- 172.21.15.62:0/3909349966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f1f4c02d370 con 0x7f1f4c104f40 2024-04-02T14:43:51.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.039+0000 7f1f43400640 1 -- 172.21.15.62:0/3909349966 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f1f3809cad0 con 0x7f1f4c104f40 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 -- 172.21.15.62:0/3909349966 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f300a3d20 msgr2=0x7f1f300a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f300a3d20 0x7f1f300a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 -- 172.21.15.62:0/3909349966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4c104f40 msgr2=0x7f1f4c19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4c104f40 0x7f1f4c19a540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800e9f0 tx=0x7f1f3800edf0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 -- 172.21.15.62:0/3909349966 shutdown_connections 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1f300a3d20 0x7f1f300a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4c105900 0x7f1f4c1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f4c104f40 0x7f1f4c19a540 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 --2- 172.21.15.62:0/3909349966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c103d50 0x7f1f4c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 -- 172.21.15.62:0/3909349966 >> 172.21.15.62:0/3909349966 conn(0x7f1f4c0ff2c0 msgr2=0x7f1f4c108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 -- 172.21.15.62:0/3909349966 shutdown_connections 2024-04-02T14:43:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.042+0000 7f1f51600640 1 -- 172.21.15.62:0/3909349966 wait complete. 2024-04-02T14:43:51.057 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-02T14:43:51.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.265+0000 7f2620000640 1 -- 172.21.15.62:0/1074972720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2618103d50 msgr2=0x7f2618104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.265+0000 7f2620000640 1 --2- 172.21.15.62:0/1074972720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2618103d50 0x7f2618104170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f260c009900 tx=0x7f260c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:43:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.265+0000 7f2620000640 1 -- 172.21.15.62:0/1074972720 shutdown_connections 2024-04-02T14:43:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.265+0000 7f2620000640 1 --2- 172.21.15.62:0/1074972720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2618105900 0x7f261810c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.265+0000 7f2620000640 1 --2- 172.21.15.62:0/1074972720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2618104f40 0x7f26181053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.265+0000 7f2620000640 1 --2- 172.21.15.62:0/1074972720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2618103d50 0x7f2618104170 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.265+0000 7f2620000640 1 -- 172.21.15.62:0/1074972720 >> 172.21.15.62:0/1074972720 conn(0x7f26180ff2c0 msgr2=0x7f2618101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.265+0000 7f2620000640 1 -- 172.21.15.62:0/1074972720 shutdown_connections 2024-04-02T14:43:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.266+0000 7f2620000640 1 -- 172.21.15.62:0/1074972720 wait complete. 2024-04-02T14:43:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.267+0000 7f2620000640 1 Processor -- start 2024-04-02T14:43:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.267+0000 7f2620000640 1 -- start start 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f2620000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2618103d50 0x7f261819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f2620000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2618104f40 0x7f261819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f2620000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2618105900 0x7f26181a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f2620000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f261810fab0 con 0x7f2618105900 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f2620000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f261810f930 con 0x7f2618103d50 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f2620000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f261810fc30 con 0x7f2618104f40 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f261d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2618103d50 0x7f261819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f261d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2618103d50 0x7f261819a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:33304/0 (socket says 172.21.15.62:33304) 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f261d600640 1 -- 172.21.15.62:0/2008456835 learned_addr learned my addr 172.21.15.62:0/2008456835 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f261cc00640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2618104f40 0x7f261819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f261e000640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2618105900 0x7f26181a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f261d600640 1 -- 172.21.15.62:0/2008456835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2618104f40 msgr2=0x7f261819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f261d600640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2618104f40 0x7f261819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f261d600640 1 -- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2618105900 msgr2=0x7f26181a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f261d600640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2618105900 0x7f26181a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f261d600640 1 -- 172.21.15.62:0/2008456835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26181a1d40 con 0x7f2618103d50 2024-04-02T14:43:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.268+0000 7f261e000640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2618105900 0x7f26181a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:43:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.269+0000 7f261d600640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2618103d50 0x7f261819a120 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f260c000c00 tx=0x7f260c004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.269+0000 7f2616000640 1 -- 172.21.15.62:0/2008456835 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c043020 con 0x7f2618103d50 2024-04-02T14:43:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.269+0000 7f2616000640 1 -- 172.21.15.62:0/2008456835 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f260c004250 con 0x7f2618103d50 2024-04-02T14:43:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.269+0000 7f2616000640 1 -- 172.21.15.62:0/2008456835 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c003b90 con 0x7f2618103d50 2024-04-02T14:43:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.269+0000 7f2620000640 1 -- 172.21.15.62:0/2008456835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26181a1fd0 con 0x7f2618103d50 2024-04-02T14:43:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.269+0000 7f2620000640 1 -- 172.21.15.62:0/2008456835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26181a2430 con 0x7f2618103d50 2024-04-02T14:43:51.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.270+0000 7f2620000640 1 -- 172.21.15.62:0/2008456835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261810afa0 con 0x7f2618103d50 2024-04-02T14:43:51.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.272+0000 7f2616000640 1 -- 172.21.15.62:0/2008456835 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f260c046020 con 0x7f2618103d50 2024-04-02T14:43:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.273+0000 7f2616000640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26000a3ca0 0x7f26000a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.273+0000 7f261cc00640 1 -- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26000a3ca0 msgr2=0x7f26000a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1995176952 2024-04-02T14:43:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.273+0000 7f2616000640 1 -- 172.21.15.62:0/2008456835 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f260c0f2530 con 0x7f2618103d50 2024-04-02T14:43:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.273+0000 7f261cc00640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26000a3ca0 0x7f26000a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T14:43:51.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.277+0000 7f2616000640 1 -- 172.21.15.62:0/2008456835 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f260c04d020 con 0x7f2618103d50 2024-04-02T14:43:51.367 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:51 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:51.101+0000 7ff03da17200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T14:43:51.367 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:51 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:51.194+0000 7ff03da17200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T14:43:51.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.474+0000 7f261cc00640 1 -- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26000a3ca0 msgr2=0x7f26000a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1995176952 2024-04-02T14:43:51.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.474+0000 7f261cc00640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26000a3ca0 0x7f26000a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T14:43:51.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.479+0000 7f2620000640 1 -- 172.21.15.62:0/2008456835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f261802d370 con 0x7f2618103d50 2024-04-02T14:43:51.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.486+0000 7f2616000640 1 -- 172.21.15.62:0/2008456835 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7f260c0bef40 con 0x7f2618103d50 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 -- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26000a3ca0 msgr2=0x7f26000a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26000a3ca0 0x7f26000a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 -- 172.21.15.62:0/2008456835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2618103d50 msgr2=0x7f261819a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2618103d50 0x7f261819a120 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f260c000c00 tx=0x7f260c004a30 comp rx=0 tx=0).stop 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 -- 172.21.15.62:0/2008456835 shutdown_connections 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f26000a3ca0 0x7f26000a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2618105900 0x7f26181a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2618104f40 0x7f261819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 --2- 172.21.15.62:0/2008456835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2618103d50 0x7f261819a120 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 -- 172.21.15.62:0/2008456835 >> 172.21.15.62:0/2008456835 conn(0x7f26180ff2c0 msgr2=0x7f2618108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 -- 172.21.15.62:0/2008456835 shutdown_connections 2024-04-02T14:43:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:51.490+0000 7f2620000640 1 -- 172.21.15.62:0/2008456835 wait complete. 2024-04-02T14:43:51.505 INFO:teuthology.run_tasks:Running task kclient... 2024-04-02T14:43:51.531 INFO:tasks.kclient:Mounting kernel clients... 2024-04-02T14:43:51.532 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-02T14:43:51.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:43:51.532 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-02T14:43:51.533 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:51.104+0000 7f370b2fa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T14:43:51.533 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:51.198+0000 7f370b2fa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T14:43:51.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:43:51.585 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-04-02T14:43:51.677 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-04-02T14:43:51.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:43:51.682 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'rbytes': False, 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-02T14:43:51.682 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T14:43:51.682 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T14:43:51.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T14:43:51.755 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:51 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:51.443+0000 7f0d0d271200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T14:43:51.755 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:51 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:51.575+0000 7f0d0d271200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T14:43:51.756 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:51 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T14:43:51.756 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:51 smithi062 ceph-mon[32116]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-02T14:43:51.756 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:51 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' 2024-04-02T14:43:51.756 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:51 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' 2024-04-02T14:43:51.826 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:51 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T14:43:51.827 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:51 smithi113 ceph-mon[36135]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-02T14:43:51.827 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:51 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' 2024-04-02T14:43:51.827 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:51 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' 2024-04-02T14:43:51.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:51 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T14:43:51.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:51 smithi204 ceph-mon[35948]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-02T14:43:51.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:51 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' 2024-04-02T14:43:51.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:51 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' 2024-04-02T14:43:52.116 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:43:52.117 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:51 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:51.830+0000 7ff03da17200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T14:43:52.117 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:51 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:51.922+0000 7ff03da17200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T14:43:52.117 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:52.062+0000 7ff03da17200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T14:43:52.166 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:51.838+0000 7f370b2fa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T14:43:52.166 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:51.931+0000 7f370b2fa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T14:43:52.166 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:52.073+0000 7f370b2fa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T14:43:52.166 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:52.168+0000 7f370b2fa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T14:43:52.495 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:52.157+0000 7ff03da17200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T14:43:52.495 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:52.239+0000 7ff03da17200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T14:43:52.495 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:52.499+0000 7ff03da17200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T14:43:52.506 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:52.252+0000 7f370b2fa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T14:43:52.527 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:52 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:52.422+0000 7f0d0d271200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T14:43:52.782 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:52.607+0000 7ff03da17200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T14:43:52.782 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:52.692+0000 7ff03da17200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T14:43:52.782 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:52.785+0000 7ff03da17200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T14:43:52.801 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:52.515+0000 7f370b2fa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T14:43:52.801 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:52.624+0000 7f370b2fa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T14:43:52.801 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:52.710+0000 7f370b2fa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T14:43:52.801 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:52.803+0000 7f370b2fa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T14:43:52.840 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:52 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:52.556+0000 7f0d0d271200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T14:43:52.840 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:52 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:52.744+0000 7f0d0d271200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T14:43:53.117 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:52.959+0000 7ff03da17200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T14:43:53.238 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:52 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:52.873+0000 7f0d0d271200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T14:43:53.238 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:52 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:52.987+0000 7f0d0d271200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T14:43:53.256 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:52.981+0000 7f370b2fa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T14:43:53.589 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:53 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:53.332+0000 7f0d0d271200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T14:43:53.590 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:53 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:53.478+0000 7f0d0d271200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T14:43:53.896 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:53 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:53.599+0000 7f0d0d271200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T14:43:53.896 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:53 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:53.727+0000 7f0d0d271200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T14:43:54.177 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:53 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:53.954+0000 7f0d0d271200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T14:43:54.256 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:53 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:53.944+0000 7f370b2fa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T14:43:54.256 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:54 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:54.075+0000 7f370b2fa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T14:43:54.367 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:53 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:53.912+0000 7ff03da17200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T14:43:54.367 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:54 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:54.041+0000 7ff03da17200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T14:43:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.517+0000 7f1df1e00700 1 -- 172.21.15.62:0/2194226093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec10e2b0 msgr2=0x7f1dec115ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.517+0000 7f1df1e00700 1 --2- 172.21.15.62:0/2194226093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec10e2b0 0x7f1dec115ec0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0099c0 tx=0x7f1ddc009cd0 comp rx=0 tx=0).stop 2024-04-02T14:43:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.517+0000 7f1df1e00700 1 -- 172.21.15.62:0/2194226093 shutdown_connections 2024-04-02T14:43:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.517+0000 7f1df1e00700 1 --2- 172.21.15.62:0/2194226093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dec109850 0x7f1dec109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.517+0000 7f1df1e00700 1 --2- 172.21.15.62:0/2194226093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec10e2b0 0x7f1dec115ec0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.517+0000 7f1df1e00700 1 --2- 172.21.15.62:0/2194226093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dec10a200 0x7f1dec10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.517+0000 7f1df1e00700 1 -- 172.21.15.62:0/2194226093 >> 172.21.15.62:0/2194226093 conn(0x7f1dec06d5a0 msgr2=0x7f1dec06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.517+0000 7f1df1e00700 1 -- 172.21.15.62:0/2194226093 shutdown_connections 2024-04-02T14:43:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.517+0000 7f1df1e00700 1 -- 172.21.15.62:0/2194226093 wait complete. 2024-04-02T14:43:54.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.519+0000 7f1df1e00700 1 Processor -- start 2024-04-02T14:43:54.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.519+0000 7f1df1e00700 1 -- start start 2024-04-02T14:43:54.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.519+0000 7f1df1e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec109850 0x7f1dec1a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:54.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.519+0000 7f1df1e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dec10a200 0x7f1dec1a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:54.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.519+0000 7f1df1e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dec10e2b0 0x7f1dec1a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:54.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df1e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec11b280 con 0x7f1dec10e2b0 2024-04-02T14:43:54.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df1e00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec11b100 con 0x7f1dec10a200 2024-04-02T14:43:54.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df1e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec11b400 con 0x7f1dec109850 2024-04-02T14:43:54.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df0a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec109850 0x7f1dec1a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:54.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df0a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec109850 0x7f1dec1a3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:42366/0 (socket says 172.21.15.62:42366) 2024-04-02T14:43:54.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df0a00700 1 -- 172.21.15.62:0/985272582 learned_addr learned my addr 172.21.15.62:0/985272582 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:43:54.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1debe00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dec10a200 0x7f1dec1a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:43:54.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df0a00700 1 -- 172.21.15.62:0/985272582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dec10a200 msgr2=0x7f1dec1a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:54.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df0a00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dec10a200 0x7f1dec1a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:54.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df0a00700 1 -- 172.21.15.62:0/985272582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dec10e2b0 msgr2=0x7f1dec1a7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:54.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df0a00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dec10e2b0 0x7f1dec1a7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:54.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df0a00700 1 -- 172.21.15.62:0/985272582 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec1a8720 con 0x7f1dec109850 2024-04-02T14:43:54.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.520+0000 7f1df0a00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec109850 0x7f1dec1a3710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1de40060b0 tx=0x7f1de4007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:43:54.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.521+0000 7f1de9600700 1 -- 172.21.15.62:0/985272582 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de401a070 con 0x7f1dec109850 2024-04-02T14:43:54.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.521+0000 7f1de9600700 1 -- 172.21.15.62:0/985272582 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1de400de20 con 0x7f1dec109850 2024-04-02T14:43:54.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.521+0000 7f1de9600700 1 -- 172.21.15.62:0/985272582 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4014830 con 0x7f1dec109850 2024-04-02T14:43:54.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.521+0000 7f1df1e00700 1 -- 172.21.15.62:0/985272582 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dec1a8a10 con 0x7f1dec109850 2024-04-02T14:43:54.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.521+0000 7f1df1e00700 1 -- 172.21.15.62:0/985272582 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dec1b0290 con 0x7f1dec109850 2024-04-02T14:43:54.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.522+0000 7f1df1e00700 1 -- 172.21.15.62:0/985272582 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dec0692b0 con 0x7f1dec109850 2024-04-02T14:43:54.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.525+0000 7f1de9600700 1 -- 172.21.15.62:0/985272582 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f1de401d070 con 0x7f1dec109850 2024-04-02T14:43:54.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.526+0000 7f1de9600700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1dd00a3c90 0x7f1dd00a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:43:54.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.526+0000 7f1debe00700 1 -- 172.21.15.62:0/985272582 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1dd00a3c90 msgr2=0x7f1dd00a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1995176952 2024-04-02T14:43:54.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.526+0000 7f1de9600700 1 -- 172.21.15.62:0/985272582 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f1de40cfd70 con 0x7f1dec109850 2024-04-02T14:43:54.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.526+0000 7f1debe00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1dd00a3c90 0x7f1dd00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T14:43:54.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.528+0000 7f1de9600700 1 -- 172.21.15.62:0/985272582 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1de4010070 con 0x7f1dec109850 2024-04-02T14:43:54.656 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:54 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:54.372+0000 7ff03da17200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T14:43:54.656 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:54 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:54.461+0000 7ff03da17200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T14:43:54.656 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:54 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:54.659+0000 7ff03da17200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:43:54.699 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:54 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:54.412+0000 7f370b2fa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T14:43:54.699 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:54 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:54.501+0000 7f370b2fa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T14:43:54.699 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:54 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:54.702+0000 7f370b2fa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:43:54.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.727+0000 7f1debe00700 1 -- 172.21.15.62:0/985272582 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1dd00a3c90 msgr2=0x7f1dd00a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1995176952 2024-04-02T14:43:54.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.727+0000 7f1debe00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1dd00a3c90 0x7f1dd00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T14:43:54.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.817+0000 7f1df1e00700 1 -- 172.21.15.62:0/985272582 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1dec1b0460 con 0x7f1dec109850 2024-04-02T14:43:54.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.818+0000 7f1de9600700 1 -- 172.21.15.62:0/985272582 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f1de409c720 con 0x7f1dec109850 2024-04-02T14:43:54.815 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-02T14:43:54.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.821+0000 7f1df1e00700 1 -- 172.21.15.62:0/985272582 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1dd00a3c90 msgr2=0x7f1dd00a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T14:43:54.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.821+0000 7f1df1e00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1dd00a3c90 0x7f1dd00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:54.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.821+0000 7f1df1e00700 1 -- 172.21.15.62:0/985272582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec109850 msgr2=0x7f1dec1a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:43:54.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.821+0000 7f1df1e00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec109850 0x7f1dec1a3710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1de40060b0 tx=0x7f1de4007f90 comp rx=0 tx=0).stop 2024-04-02T14:43:54.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.821+0000 7f1df1e00700 1 -- 172.21.15.62:0/985272582 shutdown_connections 2024-04-02T14:43:54.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.821+0000 7f1df1e00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f1dd00a3c90 0x7f1dd00a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:54.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.821+0000 7f1df1e00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dec10e2b0 0x7f1dec1a7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:54.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.821+0000 7f1df1e00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec109850 0x7f1dec1a3710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:54.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.821+0000 7f1df1e00700 1 --2- 172.21.15.62:0/985272582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dec10a200 0x7f1dec1a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:43:54.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.821+0000 7f1df1e00700 1 -- 172.21.15.62:0/985272582 >> 172.21.15.62:0/985272582 conn(0x7f1dec06d5a0 msgr2=0x7f1dec114420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:43:54.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.821+0000 7f1df1e00700 1 -- 172.21.15.62:0/985272582 shutdown_connections 2024-04-02T14:43:54.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:54.822+0000 7f1df1e00700 1 -- 172.21.15.62:0/985272582 wait complete. 2024-04-02T14:43:55.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:54 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/985272582' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T14:43:55.006 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:54 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:54.886+0000 7f370b2fa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T14:43:55.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:54 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/985272582' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T14:43:55.117 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:54 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/985272582' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T14:43:55.117 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:54 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:54.842+0000 7ff03da17200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T14:43:55.380 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:55 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:55.126+0000 7ff03da17200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:43:55.429 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:55 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:55.170+0000 7f370b2fa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:43:55.429 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:55 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:55.432+0000 7f370b2fa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T14:43:55.564 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:55 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:55.382+0000 7f0d0d271200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T14:43:55.674 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T14:43:55.674 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T14:43:55.674 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-02T14:43:55.674 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-02T14:43:55.674 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T14:43:55.674 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T14:43:55.674 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T14:43:55.674 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T14:43:55.674 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T14:43:55.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:43:55.675 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-02T14:43:55.718 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T14:43:55.718 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T14:43:55.718 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T14:43:55.718 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-02T14:43:55.718 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-02T14:43:55.718 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-02T14:43:55.718 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T14:43:55.718 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-02T14:43:55.718 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-02T14:43:55.718 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T14:43:55.718 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-02T14:43:55.719 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-02T14:43:55.719 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T14:43:55.719 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-02T14:43:55.719 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T14:43:55.719 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-02T14:43:55.719 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-02T14:43:55.719 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41890sec preferred_lft 41890sec 2024-04-02T14:43:55.719 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-02T14:43:55.719 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-02T14:43:55.720 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T14:43:55.720 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T14:43:55.720 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-02T14:43:55.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-04-02T14:43:55.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-04-02T14:43:55.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-04-02T14:43:55.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T14:43:55.721 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-02T14:43:55.756 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:55 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:55.666+0000 7f370b2fa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T14:43:55.821 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:55 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:55.567+0000 7f0d0d271200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T14:43:55.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T14:43:55.867 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:55 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:55.384+0000 7ff03da17200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T14:43:55.867 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:55 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:55.619+0000 7ff03da17200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T14:43:56.090 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:56 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:56.012+0000 7f0d0d271200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T14:43:56.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T14:43:56.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:43:56.258 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T14:43:56.333 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-02T14:43:56.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:43:56.337 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-02T14:43:56.394 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-02T14:43:56.394 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T14:43:56.394 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-02T14:43:56.395 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-02T14:43:56.395 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T14:43:56.395 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T14:43:56.395 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-02T14:43:56.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-02T14:43:56.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-02T14:43:56.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-02T14:43:56.395 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-02T14:43:56.419 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:56 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:56.140+0000 7f0d0d271200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T14:43:56.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T14:43:56.691 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:56 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:56.421+0000 7f0d0d271200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:43:56.756 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:56 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:56.467+0000 7f370b2fa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T14:43:56.867 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:56 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:56.415+0000 7ff03da17200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T14:43:57.090 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:56 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:56.694+0000 7f0d0d271200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T14:43:57.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T14:43:57.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:43:57.210 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-02T14:43:57.285 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:56 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:56.978+0000 7ff03da17200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T14:43:57.285 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:57 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:57.075+0000 7ff03da17200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T14:43:57.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:43:57.306 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-04-02T14:43:57.350 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:57 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:57.038+0000 7f370b2fa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T14:43:57.350 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:57 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:57.136+0000 7f370b2fa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T14:43:57.350 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:43:57 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z[37649]: 2024-04-02T14:43:57.353+0000 7f370b2fa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T14:43:57.358 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T14:43:57.358 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-02T14:43:57.358 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T14:43:57.359 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-02T14:43:57.359 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-02T14:43:57.367 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:57 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:57.112+0000 7f0d0d271200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T14:43:57.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T14:43:57.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T14:43:57.556 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-02T14:43:57.556 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T14:43:57.556 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-02T14:43:57.556 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-02T14:43:57.556 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T14:43:57.556 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-02T14:43:57.556 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-02T14:43:57.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-02T14:43:57.557 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-02T14:43:57.617 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:43:57 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y[37523]: 2024-04-02T14:43:57.289+0000 7ff03da17200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T14:43:57.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:57 smithi113 ceph-mon[36135]: Standby manager daemon y restarted 2024-04-02T14:43:57.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:57 smithi113 ceph-mon[36135]: Standby manager daemon y started 2024-04-02T14:43:57.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:57 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T14:43:57.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:57 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:43:57.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:57 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T14:43:57.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:57 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:43:57.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T14:43:57.684 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:57 smithi062 ceph-mon[32116]: Standby manager daemon y restarted 2024-04-02T14:43:57.684 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:57 smithi062 ceph-mon[32116]: Standby manager daemon y started 2024-04-02T14:43:57.684 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:57 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T14:43:57.685 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:57 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:43:57.685 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:57 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T14:43:57.685 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:57 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:43:57.685 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:57 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:57.453+0000 7f0d0d271200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T14:43:57.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:57 smithi204 ceph-mon[35948]: Standby manager daemon y restarted 2024-04-02T14:43:57.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:57 smithi204 ceph-mon[35948]: Standby manager daemon y started 2024-04-02T14:43:57.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:57 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T14:43:57.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:57 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:43:57.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:57 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T14:43:57.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:57 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.113:0/305163990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:43:57.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T14:43:57.904 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T14:43:57.905 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-02T14:43:57.905 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-02T14:43:57.905 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T14:43:57.905 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-02T14:43:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T14:43:58.022 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:57 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:57.785+0000 7f0d0d271200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T14:43:58.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T14:43:58.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T14:43:58.118 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-02T14:43:58.159 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-02T14:43:58.160 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T14:43:58.160 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-02T14:43:58.247 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-02T14:43:58.248 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-02T14:43:58.248 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8 2024-04-02T14:43:58.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:43:58.248 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-02T14:43:58.416 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-02T14:43:58.416 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-02T14:43:58.416 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-02T14:43:58.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:58 smithi062 ceph-mon[32116]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-02T14:43:58.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:58 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T14:43:58.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:58 smithi062 ceph-mon[32116]: Standby manager daemon z restarted 2024-04-02T14:43:58.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:58 smithi062 ceph-mon[32116]: Standby manager daemon z started 2024-04-02T14:43:58.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:58 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:43:58.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:58 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T14:43:58.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:58 smithi062 ceph-mon[32116]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:43:58.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:43:58 smithi062 ceph-mon[32116]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-04-02T14:43:58.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:58 smithi204 ceph-mon[35948]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-02T14:43:58.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:58 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T14:43:58.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:58 smithi204 ceph-mon[35948]: Standby manager daemon z restarted 2024-04-02T14:43:58.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:58 smithi204 ceph-mon[35948]: Standby manager daemon z started 2024-04-02T14:43:58.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:58 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:43:58.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:58 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T14:43:58.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:58 smithi204 ceph-mon[35948]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:43:58.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:43:58 smithi204 ceph-mon[35948]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-04-02T14:43:58.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:58 smithi113 ceph-mon[36135]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-02T14:43:58.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:58 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T14:43:58.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:58 smithi113 ceph-mon[36135]: Standby manager daemon z restarted 2024-04-02T14:43:58.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:58 smithi113 ceph-mon[36135]: Standby manager daemon z started 2024-04-02T14:43:58.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:58 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T14:43:58.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:58 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T14:43:58.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:58 smithi113 ceph-mon[36135]: from='mgr.? 172.21.15.204:0/2410336083' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T14:43:58.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:43:58 smithi113 ceph-mon[36135]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-04-02T14:43:59.119 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-02T14:43:59.119 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-02T14:43:59.119 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-02T14:43:59.120 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.113:3300,172.21.15.204:3300" 2024-04-02T14:43:59.120 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:3300,172.21.15.113:3300,172.21.15.204:3300:/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8 2024-04-02T14:43:59.120 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=9e86b65e-f0fd-11ee-b647-cb9ed24678a4" will pass to kernel 2024-04-02T14:43:59.122 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-02T14:43:59.122 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-02T14:43:59.122 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-02T14:43:59.122 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-02T14:43:59.122 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-02T14:43:59.127 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-02T14:43:59.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:43:59.127 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-02T14:43:59.206 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:58 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:58.891+0000 7f0d0d271200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T14:43:59.222 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T14:43:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T14:43:59.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T14:43:59.342 INFO:teuthology.orchestra.run.smithi062.stdout:client25310 2024-04-02T14:43:59.344 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:43:59.344 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T14:43:59.394 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T14:43:59.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T14:43:59.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T14:43:59.554 INFO:teuthology.orchestra.run.smithi062.stdout:client25310 2024-04-02T14:43:59.555 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T14:43:59.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T14:43:59.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T14:43:59.667 INFO:teuthology.orchestra.run.smithi062.stdout:client25310 2024-04-02T14:43:59.668 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4.client25310/status') 2024-04-02T14:43:59.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T14:43:59.806 INFO:teuthology.orchestra.run.smithi062.stdout:instance: client.25310 (3)172.21.15.62:0/1250263491 2024-04-02T14:43:59.806 INFO:teuthology.orchestra.run.smithi062.stdout:blocklisted: false 2024-04-02T14:43:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2024-04-02T14:43:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2024-04-02T14:43:59.807 INFO:teuthology.orchestra.run.smithi062.stderr:71 bytes copied, 6.771e-05 s, 1.0 MB/s 2024-04-02T14:43:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:43:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T14:43:59.819 INFO:teuthology.run_tasks:Running task exec... 2024-04-02T14:43:59.837 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T14:43:59.837 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-04-02T14:43:59.837 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-02T14:43:59.908 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:59 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:59.634+0000 7f0d0d271200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T14:43:59.908 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:43:59 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:43:59.770+0000 7f0d0d271200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T14:44:00.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.033+0000 7f700ba00640 1 -- 172.21.15.62:0/2874387278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7004105910 msgr2=0x7f700410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:00.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.033+0000 7f700ba00640 1 --2- 172.21.15.62:0/2874387278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7004105910 0x7f700410c1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009900 tx=0x7f6ffc02bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:00.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.034+0000 7f700ba00640 1 -- 172.21.15.62:0/2874387278 shutdown_connections 2024-04-02T14:44:00.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.034+0000 7f700ba00640 1 --2- 172.21.15.62:0/2874387278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7004105910 0x7f700410c1a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:00.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.034+0000 7f700ba00640 1 --2- 172.21.15.62:0/2874387278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7004104f50 0x7f70041053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:00.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.034+0000 7f700ba00640 1 --2- 172.21.15.62:0/2874387278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7004103d60 0x7f7004104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:00.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.034+0000 7f700ba00640 1 -- 172.21.15.62:0/2874387278 >> 172.21.15.62:0/2874387278 conn(0x7f70040ff2f0 msgr2=0x7f7004101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:00.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.034+0000 7f700ba00640 1 -- 172.21.15.62:0/2874387278 shutdown_connections 2024-04-02T14:44:00.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.034+0000 7f700ba00640 1 -- 172.21.15.62:0/2874387278 wait complete. 2024-04-02T14:44:00.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.035+0000 7f700ba00640 1 Processor -- start 2024-04-02T14:44:00.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.035+0000 7f700ba00640 1 -- start start 2024-04-02T14:44:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.035+0000 7f700ba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7004103d60 0x7f700419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.035+0000 7f700ba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7004104f50 0x7f700419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f700ba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7004105910 0x7f70041a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f700ba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f700410fab0 con 0x7f7004104f50 2024-04-02T14:44:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f700ba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f700410f930 con 0x7f7004105910 2024-04-02T14:44:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f700ba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f700410fc30 con 0x7f7004103d60 2024-04-02T14:44:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f7009000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7004103d60 0x7f700419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f7009a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7004105910 0x7f70041a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f7009000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7004103d60 0x7f700419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:42374/0 (socket says 172.21.15.62:42374) 2024-04-02T14:44:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f7009000640 1 -- 172.21.15.62:0/3807193478 learned_addr learned my addr 172.21.15.62:0/3807193478 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f7003e00640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7004104f50 0x7f700419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f7009000640 1 -- 172.21.15.62:0/3807193478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7004105910 msgr2=0x7f70041a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f7009000640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7004105910 0x7f70041a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f7009000640 1 -- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7004104f50 msgr2=0x7f700419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f7009000640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7004104f50 0x7f700419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.036+0000 7f7009000640 1 -- 172.21.15.62:0/3807193478 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70041a1c20 con 0x7f7004103d60 2024-04-02T14:44:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.037+0000 7f7009000640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7004103d60 0x7f700419a000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400ebc0 tx=0x7f6ff400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.037+0000 7f7001600640 1 -- 172.21.15.62:0/3807193478 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4003b70 con 0x7f7004103d60 2024-04-02T14:44:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.037+0000 7f700ba00640 1 -- 172.21.15.62:0/3807193478 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70041a1f10 con 0x7f7004103d60 2024-04-02T14:44:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.037+0000 7f7001600640 1 -- 172.21.15.62:0/3807193478 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6ff4003d10 con 0x7f7004103d60 2024-04-02T14:44:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.037+0000 7f7001600640 1 -- 172.21.15.62:0/3807193478 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4019980 con 0x7f7004103d60 2024-04-02T14:44:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.037+0000 7f700ba00640 1 -- 172.21.15.62:0/3807193478 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70041a23d0 con 0x7f7004103d60 2024-04-02T14:44:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.038+0000 7f700ba00640 1 -- 172.21.15.62:0/3807193478 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f700410afb0 con 0x7f7004103d60 2024-04-02T14:44:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.040+0000 7f7001600640 1 -- 172.21.15.62:0/3807193478 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) ==== 138663+0+0 (secure 0 0 0) 0x7f6ff4004b60 con 0x7f7004103d60 2024-04-02T14:44:00.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.042+0000 7f7001600640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6fe80a3ca0 0x7f6fe80a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:00.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.042+0000 7f7003e00640 1 -- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6fe80a3ca0 msgr2=0x7f6fe80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1995176952 2024-04-02T14:44:00.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.042+0000 7f7001600640 1 -- 172.21.15.62:0/3807193478 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f6ff40d0f20 con 0x7f7004103d60 2024-04-02T14:44:00.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.042+0000 7f7003e00640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6fe80a3ca0 0x7f6fe80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T14:44:00.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.044+0000 7f7001600640 1 -- 172.21.15.62:0/3807193478 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6ff4044e20 con 0x7f7004103d60 2024-04-02T14:44:00.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.109+0000 7f7001600640 1 -- 172.21.15.62:0/3807193478 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f6ff4099840 con 0x7f7004103d60 2024-04-02T14:44:00.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.111+0000 7f7001600640 1 -- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6fe80a3ca0 msgr2=0x7f6fe80a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T14:44:00.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.111+0000 7f7001600640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:6800/1995176952,v1:172.21.15.62:6801/1995176952] conn(0x7f6fe80a3ca0 0x7f6fe80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:00.293 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: Active manager daemon x restarted 2024-04-02T14:44:00.293 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: Activating manager daemon x 2024-04-02T14:44:00.293 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T14:44:00.293 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: mgrmap e22: x(active, starting, since 0.0226802s), standbys: z, y 2024-04-02T14:44:00.293 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:44:00.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:44:00.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:44:00.296 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:44:00.296 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T14:44:00.296 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T14:44:00.296 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T14:44:00.296 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-mon[32116]: Manager daemon x is now available 2024-04-02T14:44:00.296 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:00.081+0000 7f0d0d271200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T14:44:00.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:00.374+0000 7f700ba00640 1 -- 172.21.15.62:0/3807193478 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f700402d370 con 0x7f7004103d60 2024-04-02T14:44:00.399 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: Active manager daemon x restarted 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: Activating manager daemon x 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: mgrmap e22: x(active, starting, since 0.0226802s), standbys: z, y 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:44:00.400 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:44:00.401 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:44:00.402 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:44:00.402 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:44:00.402 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:44:00.402 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:44:00.402 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:44:00.402 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T14:44:00.402 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T14:44:00.402 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T14:44:00.402 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:00 smithi113 ceph-mon[36135]: Manager daemon x is now available 2024-04-02T14:44:00.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: Active manager daemon x restarted 2024-04-02T14:44:00.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: Activating manager daemon x 2024-04-02T14:44:00.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T14:44:00.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: mgrmap e22: x(active, starting, since 0.0226802s), standbys: z, y 2024-04-02T14:44:00.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T14:44:00.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T14:44:00.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T14:44:00.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T14:44:00.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T14:44:00.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:44:00.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T14:44:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T14:44:00.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:00 smithi204 ceph-mon[35948]: Manager daemon x is now available 2024-04-02T14:44:00.590 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:00.431+0000 7f0c63600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:00.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:00.431+0000 7f0c63600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:00.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:00.431+0000 7f0c63600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:00.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:00.431+0000 7f0c63600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:00.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:00.431+0000 7f0c63600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:00.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:00.431+0000 7f0c5e600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:00.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:00.431+0000 7f0c5e600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:00.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:00.431+0000 7f0c5e600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:00.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:00.431+0000 7f0c5e600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:00.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:00.431+0000 7f0c5e600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:01.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.118+0000 7f7001600640 1 -- 172.21.15.62:0/3807193478 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f6ff40ce700 con 0x7f7004103d60 2024-04-02T14:44:01.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.120+0000 7f7001600640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f6fe8052700 0x7f6fe8052ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:01.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.128+0000 7f7003e00640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f6fe8052700 0x7f6fe8052ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:01.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.135+0000 7f7003e00640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f6fe8052700 0x7f6fe8052ae0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f700419b510 tx=0x7f6ff0005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.146+0000 7f7001600640 1 -- 172.21.15.62:0/3807193478 <== mon.1 v2:172.21.15.204:3300/0 9 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f6ff40ab6e0 con 0x7f7004103d60 2024-04-02T14:44:01.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.149+0000 7f700ba00640 1 -- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f6fe8052700 msgr2=0x7f6fe8052ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:01.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.149+0000 7f700ba00640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f6fe8052700 0x7f6fe8052ae0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f700419b510 tx=0x7f6ff0005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:01.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.149+0000 7f700ba00640 1 -- 172.21.15.62:0/3807193478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7004103d60 msgr2=0x7f700419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:01.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.149+0000 7f700ba00640 1 --2- 172.21.15.62:0/3807193478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7004103d60 0x7f700419a000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400ebc0 tx=0x7f6ff400c6a0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.149+0000 7f7009a00640 1 -- 172.21.15.62:0/3807193478 reap_dead start 2024-04-02T14:44:01.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.149+0000 7f700ba00640 1 -- 172.21.15.62:0/3807193478 shutdown_connections 2024-04-02T14:44:01.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.149+0000 7f700ba00640 1 -- 172.21.15.62:0/3807193478 >> 172.21.15.62:0/3807193478 conn(0x7f70040ff2f0 msgr2=0x7f7004108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:01.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.149+0000 7f700ba00640 1 -- 172.21.15.62:0/3807193478 shutdown_connections 2024-04-02T14:44:01.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.149+0000 7f700ba00640 1 -- 172.21.15.62:0/3807193478 wait complete. 2024-04-02T14:44:01.167 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-02T14:44:01.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.362+0000 7f8a69000640 1 -- 172.21.15.62:0/1194460453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a64101970 msgr2=0x7f8a64101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:01.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.362+0000 7f8a69000640 1 --2- 172.21.15.62:0/1194460453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a64101970 0x7f8a64101d90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009900 tx=0x7f8a5402bc40 comp rx=0 tx=0).stop 2024-04-02T14:44:01.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.362+0000 7f8a69000640 1 -- 172.21.15.62:0/1194460453 shutdown_connections 2024-04-02T14:44:01.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.362+0000 7f8a69000640 1 --2- 172.21.15.62:0/1194460453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a64103520 0x7f8a64109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.362+0000 7f8a69000640 1 --2- 172.21.15.62:0/1194460453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a64102b60 0x7f8a64102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.362+0000 7f8a69000640 1 --2- 172.21.15.62:0/1194460453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a64101970 0x7f8a64101d90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.362+0000 7f8a69000640 1 -- 172.21.15.62:0/1194460453 >> 172.21.15.62:0/1194460453 conn(0x7f8a640fcee0 msgr2=0x7f8a640ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:01.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.362+0000 7f8a69000640 1 -- 172.21.15.62:0/1194460453 shutdown_connections 2024-04-02T14:44:01.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.362+0000 7f8a69000640 1 -- 172.21.15.62:0/1194460453 wait complete. 2024-04-02T14:44:01.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.364+0000 7f8a69000640 1 Processor -- start 2024-04-02T14:44:01.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.364+0000 7f8a69000640 1 -- start start 2024-04-02T14:44:01.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.365+0000 7f8a69000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a64101970 0x7f8a64197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:01.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.365+0000 7f8a69000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a64102b60 0x7f8a64198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:01.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.365+0000 7f8a69000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a64103520 0x7f8a6419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:01.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.365+0000 7f8a69000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6410d710 con 0x7f8a64102b60 2024-04-02T14:44:01.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.365+0000 7f8a69000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6410d590 con 0x7f8a64101970 2024-04-02T14:44:01.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.365+0000 7f8a69000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6410d890 con 0x7f8a64103520 2024-04-02T14:44:01.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.365+0000 7f8a63400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a64101970 0x7f8a64197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:01.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.365+0000 7f8a63400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a64101970 0x7f8a64197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:59506/0 (socket says 172.21.15.62:59506) 2024-04-02T14:44:01.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.365+0000 7f8a63400640 1 -- 172.21.15.62:0/4042073449 learned_addr learned my addr 172.21.15.62:0/4042073449 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:01.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.365+0000 7f8a62a00640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a64102b60 0x7f8a64198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:01.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.365+0000 7f8a63e00640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a64103520 0x7f8a6419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:01.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.366+0000 7f8a63400640 1 -- 172.21.15.62:0/4042073449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a64103520 msgr2=0x7f8a6419f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:01.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.366+0000 7f8a63400640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a64103520 0x7f8a6419f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.366+0000 7f8a63400640 1 -- 172.21.15.62:0/4042073449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a64102b60 msgr2=0x7f8a64198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:01.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.366+0000 7f8a63400640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a64102b60 0x7f8a64198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.366+0000 7f8a63400640 1 -- 172.21.15.62:0/4042073449 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6419f950 con 0x7f8a64101970 2024-04-02T14:44:01.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.366+0000 7f8a63400640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a64101970 0x7f8a64197d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8a54000c00 tx=0x7f8a54004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:01.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.366+0000 7f8a5be00640 1 -- 172.21.15.62:0/4042073449 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54043020 con 0x7f8a64101970 2024-04-02T14:44:01.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.366+0000 7f8a5be00640 1 -- 172.21.15.62:0/4042073449 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a54004250 con 0x7f8a64101970 2024-04-02T14:44:01.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.366+0000 7f8a5be00640 1 -- 172.21.15.62:0/4042073449 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54003b90 con 0x7f8a64101970 2024-04-02T14:44:01.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.367+0000 7f8a69000640 1 -- 172.21.15.62:0/4042073449 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6419fbe0 con 0x7f8a64101970 2024-04-02T14:44:01.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.367+0000 7f8a69000640 1 -- 172.21.15.62:0/4042073449 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a641a00e0 con 0x7f8a64101970 2024-04-02T14:44:01.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.368+0000 7f8a69000640 1 -- 172.21.15.62:0/4042073449 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a640500b0 con 0x7f8a64101970 2024-04-02T14:44:01.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.370+0000 7f8a5be00640 1 -- 172.21.15.62:0/4042073449 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f8a54046020 con 0x7f8a64101970 2024-04-02T14:44:01.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.371+0000 7f8a5be00640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a480a3a10 0x7f8a480a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:01.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.371+0000 7f8a5be00640 1 -- 172.21.15.62:0/4042073449 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8a540f2110 con 0x7f8a64101970 2024-04-02T14:44:01.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.371+0000 7f8a62a00640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a480a3a10 0x7f8a480a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:01.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.372+0000 7f8a62a00640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a480a3a10 0x7f8a480a5ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009bd0 tx=0x7f8a50005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:01.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.375+0000 7f8a5be00640 1 -- 172.21.15.62:0/4042073449 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8a5404d020 con 0x7f8a64101970 2024-04-02T14:44:01.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:01 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:44:01.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:01 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T14:44:01.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:01 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T14:44:01.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:01 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T14:44:01.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:01 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T14:44:01.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:01 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3807193478' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T14:44:01.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:01 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T14:44:01.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:01 smithi204 ceph-mon[35948]: mgrmap e23: x(active, since 1.02246s), standbys: z, y 2024-04-02T14:44:01.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:44:01.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T14:44:01.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T14:44:01.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T14:44:01.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T14:44:01.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3807193478' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T14:44:01.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T14:44:01.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-mon[32116]: mgrmap e23: x(active, since 1.02246s), standbys: z, y 2024-04-02T14:44:01.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:01 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:44:01.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:01 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T14:44:01.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:01 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T14:44:01.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:01 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T14:44:01.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:01 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T14:44:01.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:01 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3807193478' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T14:44:01.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:01 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T14:44:01.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:01 smithi113 ceph-mon[36135]: mgrmap e23: x(active, since 1.02246s), standbys: z, y 2024-04-02T14:44:01.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.626+0000 7f8a69000640 1 -- 172.21.15.62:0/4042073449 --> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f8a640620a0 con 0x7f8a480a3a10 2024-04-02T14:44:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.645+0000 7f8a5be00640 1 -- 172.21.15.62:0/4042073449 <== mgr.25273 v2:172.21.15.62:6800/110874546 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f8a640620a0 con 0x7f8a480a3a10 2024-04-02T14:44:01.643 INFO:teuthology.orchestra.run.smithi062.stdout:{} 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 -- 172.21.15.62:0/4042073449 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a480a3a10 msgr2=0x7f8a480a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a480a3a10 0x7f8a480a5ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009bd0 tx=0x7f8a50005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 -- 172.21.15.62:0/4042073449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a64101970 msgr2=0x7f8a64197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a64101970 0x7f8a64197d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8a54000c00 tx=0x7f8a54004a90 comp rx=0 tx=0).stop 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 -- 172.21.15.62:0/4042073449 shutdown_connections 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a480a3a10 0x7f8a480a5ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a64103520 0x7f8a6419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a64102b60 0x7f8a64198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 --2- 172.21.15.62:0/4042073449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a64101970 0x7f8a64197d40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 -- 172.21.15.62:0/4042073449 >> 172.21.15.62:0/4042073449 conn(0x7f8a640fcee0 msgr2=0x7f8a64105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 -- 172.21.15.62:0/4042073449 shutdown_connections 2024-04-02T14:44:01.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.647+0000 7f8a69000640 1 -- 172.21.15.62:0/4042073449 wait complete. 2024-04-02T14:44:01.662 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-02T14:44:01.675 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-02T14:44:01.705 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:44:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.914+0000 7f1d10c00640 1 -- 172.21.15.62:0/972221054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d0c101970 msgr2=0x7f1d0c101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.914+0000 7f1d10c00640 1 --2- 172.21.15.62:0/972221054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d0c101970 0x7f1d0c101d90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009900 tx=0x7f1cfc02bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.914+0000 7f1d10c00640 1 -- 172.21.15.62:0/972221054 shutdown_connections 2024-04-02T14:44:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.914+0000 7f1d10c00640 1 --2- 172.21.15.62:0/972221054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d0c103520 0x7f1d0c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.914+0000 7f1d10c00640 1 --2- 172.21.15.62:0/972221054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102b60 0x7f1d0c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.914+0000 7f1d10c00640 1 --2- 172.21.15.62:0/972221054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d0c101970 0x7f1d0c101d90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.914+0000 7f1d10c00640 1 -- 172.21.15.62:0/972221054 >> 172.21.15.62:0/972221054 conn(0x7f1d0c0fcee0 msgr2=0x7f1d0c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.914+0000 7f1d10c00640 1 -- 172.21.15.62:0/972221054 shutdown_connections 2024-04-02T14:44:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.914+0000 7f1d10c00640 1 -- 172.21.15.62:0/972221054 wait complete. 2024-04-02T14:44:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.915+0000 7f1d10c00640 1 Processor -- start 2024-04-02T14:44:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.915+0000 7f1d10c00640 1 -- start start 2024-04-02T14:44:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.915+0000 7f1d10c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d0c101970 0x7f1d0c0758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.915+0000 7f1d10c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102b60 0x7f1d0c073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.915+0000 7f1d10c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d0c103520 0x7f1d0c074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.915+0000 7f1d10c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c10d6d0 con 0x7f1d0c101970 2024-04-02T14:44:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.915+0000 7f1d10c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c10d550 con 0x7f1d0c103520 2024-04-02T14:44:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.915+0000 7f1d10c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c10d850 con 0x7f1d0c102b60 2024-04-02T14:44:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.916+0000 7f1d0be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d0c103520 0x7f1d0c074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.916+0000 7f1d0b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d0c101970 0x7f1d0c0758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.916+0000 7f1d03e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102b60 0x7f1d0c073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.916+0000 7f1d0be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d0c103520 0x7f1d0c074430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:59546/0 (socket says 172.21.15.62:59546) 2024-04-02T14:44:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.916+0000 7f1d0be00640 1 -- 172.21.15.62:0/2521124946 learned_addr learned my addr 172.21.15.62:0/2521124946 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.916+0000 7f1d0b400640 1 -- 172.21.15.62:0/2521124946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102b60 msgr2=0x7f1d0c073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.916+0000 7f1d0b400640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102b60 0x7f1d0c073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.916+0000 7f1d0b400640 1 -- 172.21.15.62:0/2521124946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d0c103520 msgr2=0x7f1d0c074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.916+0000 7f1d0b400640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d0c103520 0x7f1d0c074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.916+0000 7f1d0b400640 1 -- 172.21.15.62:0/2521124946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0c074cc0 con 0x7f1d0c101970 2024-04-02T14:44:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.916+0000 7f1d0be00640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d0c103520 0x7f1d0c074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.917+0000 7f1d0b400640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d0c101970 0x7f1d0c0758b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc036040 tx=0x7f1cfc004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.917+0000 7f1d08c00640 1 -- 172.21.15.62:0/2521124946 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc043020 con 0x7f1d0c101970 2024-04-02T14:44:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.917+0000 7f1d08c00640 1 -- 172.21.15.62:0/2521124946 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1cfc004310 con 0x7f1d0c101970 2024-04-02T14:44:01.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.917+0000 7f1d10c00640 1 -- 172.21.15.62:0/2521124946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0c1a1ea0 con 0x7f1d0c101970 2024-04-02T14:44:01.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.917+0000 7f1d08c00640 1 -- 172.21.15.62:0/2521124946 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc003af0 con 0x7f1d0c101970 2024-04-02T14:44:01.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.917+0000 7f1d10c00640 1 -- 172.21.15.62:0/2521124946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0c1a22a0 con 0x7f1d0c101970 2024-04-02T14:44:01.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.918+0000 7f1d10c00640 1 -- 172.21.15.62:0/2521124946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d0c108d10 con 0x7f1d0c101970 2024-04-02T14:44:01.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.920+0000 7f1d08c00640 1 -- 172.21.15.62:0/2521124946 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f1cfc046020 con 0x7f1d0c101970 2024-04-02T14:44:01.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.921+0000 7f1d08c00640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1cf00a39b0 0x7f1cf00a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:01.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.921+0000 7f1d08c00640 1 -- 172.21.15.62:0/2521124946 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f1cfc0f2050 con 0x7f1d0c101970 2024-04-02T14:44:01.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.922+0000 7f1d03e00640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1cf00a39b0 0x7f1cf00a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:01.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.922+0000 7f1d03e00640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1cf00a39b0 0x7f1cf00a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c075620 tx=0x7f1cf4005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:01.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:01.926+0000 7f1d08c00640 1 -- 172.21.15.62:0/2521124946 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1cfc04d020 con 0x7f1d0c101970 2024-04-02T14:44:02.090 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:01.633+0000 7f0c5be00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:02.091 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:01.633+0000 7f0c5be00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:02.091 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:01.633+0000 7f0c5be00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:02.091 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:01.633+0000 7f0c5be00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:02.091 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:44:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:44:01.633+0000 7f0c5be00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:44:02.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.136+0000 7f1d10c00640 1 -- 172.21.15.62:0/2521124946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d0c0627a0 con 0x7f1d0c101970 2024-04-02T14:44:02.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.138+0000 7f1d08c00640 1 -- 172.21.15.62:0/2521124946 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22957 (secure 0 0 0) 0x7f1cfc0bec10 con 0x7f1d0c101970 2024-04-02T14:44:02.136 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:44:02.137 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":91,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:44:00.087363+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/129789951":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/79979639":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/566820061":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3400451184":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:44:02.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 -- 172.21.15.62:0/2521124946 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1cf00a39b0 msgr2=0x7f1cf00a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:02.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1cf00a39b0 0x7f1cf00a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c075620 tx=0x7f1cf4005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:02.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 -- 172.21.15.62:0/2521124946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d0c101970 msgr2=0x7f1d0c0758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:02.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d0c101970 0x7f1d0c0758b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc036040 tx=0x7f1cfc004a30 comp rx=0 tx=0).stop 2024-04-02T14:44:02.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 -- 172.21.15.62:0/2521124946 shutdown_connections 2024-04-02T14:44:02.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1cf00a39b0 0x7f1cf00a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d0c103520 0x7f1d0c074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102b60 0x7f1d0c073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 --2- 172.21.15.62:0/2521124946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d0c101970 0x7f1d0c0758b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 -- 172.21.15.62:0/2521124946 >> 172.21.15.62:0/2521124946 conn(0x7f1d0c0fcee0 msgr2=0x7f1d0c105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:02.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 -- 172.21.15.62:0/2521124946 shutdown_connections 2024-04-02T14:44:02.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.141+0000 7f1d10c00640 1 -- 172.21.15.62:0/2521124946 wait complete. 2024-04-02T14:44:02.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T14:41:20.030393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T14:41:20.917405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:44:02.161 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:44:02.256 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-02T14:44:02.256 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:44:02.257 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: [02/Apr/2024:14:44:01] ENGINE Bus STARTING 2024-04-02T14:44:02.257 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.257 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.257 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.257 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.257 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: [02/Apr/2024:14:44:01] ENGINE Serving on https://172.21.15.62:7150 2024-04-02T14:44:02.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: [02/Apr/2024:14:44:01] ENGINE Serving on http://172.21.15.62:8765 2024-04-02T14:44:02.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: [02/Apr/2024:14:44:01] ENGINE Bus STARTED 2024-04-02T14:44:02.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: from='client.25358 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:02.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:02 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2521124946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:02.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.354+0000 7fc8be400640 1 -- 172.21.15.62:0/238036692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b8102580 msgr2=0x7fc8b8104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:02.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.354+0000 7fc8be400640 1 --2- 172.21.15.62:0/238036692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b8102580 0x7fc8b8104970 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009860 tx=0x7fc8a802bb70 comp rx=0 tx=0).stop 2024-04-02T14:44:02.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.355+0000 7fc8be400640 1 -- 172.21.15.62:0/238036692 shutdown_connections 2024-04-02T14:44:02.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.355+0000 7fc8be400640 1 --2- 172.21.15.62:0/238036692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b81077e0 0x7fc8b8109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.355+0000 7fc8be400640 1 --2- 172.21.15.62:0/238036692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8b8104eb0 0x7fc8b81072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.355+0000 7fc8be400640 1 --2- 172.21.15.62:0/238036692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b8102580 0x7fc8b8104970 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.355+0000 7fc8be400640 1 -- 172.21.15.62:0/238036692 >> 172.21.15.62:0/238036692 conn(0x7fc8b80fbf70 msgr2=0x7fc8b80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.355+0000 7fc8be400640 1 -- 172.21.15.62:0/238036692 shutdown_connections 2024-04-02T14:44:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.355+0000 7fc8be400640 1 -- 172.21.15.62:0/238036692 wait complete. 2024-04-02T14:44:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8be400640 1 Processor -- start 2024-04-02T14:44:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8be400640 1 -- start start 2024-04-02T14:44:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8be400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b8102580 0x7fc8b819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8be400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b8104eb0 0x7fc8b819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8be400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8b81077e0 0x7fc8b81a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8be400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b810d8b0 con 0x7fc8b8104eb0 2024-04-02T14:44:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8be400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b810d730 con 0x7fc8b8102580 2024-04-02T14:44:02.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8be400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b810da30 con 0x7fc8b81077e0 2024-04-02T14:44:02.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8b6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b8104eb0 0x7fc8b819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:02.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8b81077e0 0x7fc8b81a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:02.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8b6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b8104eb0 0x7fc8b819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35588/0 (socket says 172.21.15.62:35588) 2024-04-02T14:44:02.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8b6a00640 1 -- 172.21.15.62:0/1663065840 learned_addr learned my addr 172.21.15.62:0/1663065840 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:02.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.356+0000 7fc8b7400640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b8102580 0x7fc8b819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.357+0000 7fc8b7e00640 1 -- 172.21.15.62:0/1663065840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b8102580 msgr2=0x7fc8b819a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.357+0000 7fc8b7e00640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b8102580 0x7fc8b819a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.357+0000 7fc8b7e00640 1 -- 172.21.15.62:0/1663065840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b8104eb0 msgr2=0x7fc8b819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.357+0000 7fc8b7e00640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b8104eb0 0x7fc8b819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.357+0000 7fc8b7e00640 1 -- 172.21.15.62:0/1663065840 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b81a1d40 con 0x7fc8b81077e0 2024-04-02T14:44:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.357+0000 7fc8b7e00640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8b81077e0 0x7fc8b81a16d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ec50 tx=0x7fc8b000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.357+0000 7fc8afe00640 1 -- 172.21.15.62:0/1663065840 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0003b70 con 0x7fc8b81077e0 2024-04-02T14:44:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.357+0000 7fc8afe00640 1 -- 172.21.15.62:0/1663065840 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8b000ee90 con 0x7fc8b81077e0 2024-04-02T14:44:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.357+0000 7fc8be400640 1 -- 172.21.15.62:0/1663065840 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b81a1fd0 con 0x7fc8b81077e0 2024-04-02T14:44:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.357+0000 7fc8afe00640 1 -- 172.21.15.62:0/1663065840 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0010830 con 0x7fc8b81077e0 2024-04-02T14:44:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.357+0000 7fc8be400640 1 -- 172.21.15.62:0/1663065840 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b81a24a0 con 0x7fc8b81077e0 2024-04-02T14:44:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.359+0000 7fc8be400640 1 -- 172.21.15.62:0/1663065840 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b8109330 con 0x7fc8b81077e0 2024-04-02T14:44:02.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.360+0000 7fc8afe00640 1 -- 172.21.15.62:0/1663065840 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7fc8b0004020 con 0x7fc8b81077e0 2024-04-02T14:44:02.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.361+0000 7fc8afe00640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc89c0a39b0 0x7fc89c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:02.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.362+0000 7fc8afe00640 1 -- 172.21.15.62:0/1663065840 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc8b00d06a0 con 0x7fc8b81077e0 2024-04-02T14:44:02.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.362+0000 7fc8b7400640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc89c0a39b0 0x7fc89c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:02.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.363+0000 7fc8b7400640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc89c0a39b0 0x7fc89c0a5e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009830 tx=0x7fc8a8034b50 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:02.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.365+0000 7fc8afe00640 1 -- 172.21.15.62:0/1663065840 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc8b009d260 con 0x7fc8b81077e0 2024-04-02T14:44:02.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-02T14:44:02.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:44:02.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: [02/Apr/2024:14:44:01] ENGINE Bus STARTING 2024-04-02T14:44:02.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: [02/Apr/2024:14:44:01] ENGINE Serving on https://172.21.15.62:7150 2024-04-02T14:44:02.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: [02/Apr/2024:14:44:01] ENGINE Serving on http://172.21.15.62:8765 2024-04-02T14:44:02.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: [02/Apr/2024:14:44:01] ENGINE Bus STARTED 2024-04-02T14:44:02.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: from='client.25358 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:02.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:02 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2521124946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:02.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.566+0000 7fc8afe00640 1 -- 172.21.15.62:0/1663065840 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7fc8b0004310 con 0x7fc8b81077e0 2024-04-02T14:44:02.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.584+0000 7fc8be400640 1 -- 172.21.15.62:0/1663065840 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8b8062700 con 0x7fc8b81077e0 2024-04-02T14:44:02.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.585+0000 7fc8afe00640 1 -- 172.21.15.62:0/1663065840 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fc8b00c7ce0 con 0x7fc8b81077e0 2024-04-02T14:44:02.582 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:44:02.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.587+0000 7fc8be400640 1 -- 172.21.15.62:0/1663065840 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc89c0a39b0 msgr2=0x7fc89c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:02.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.587+0000 7fc8be400640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc89c0a39b0 0x7fc89c0a5e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009830 tx=0x7fc8a8034b50 comp rx=0 tx=0).stop 2024-04-02T14:44:02.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.587+0000 7fc8be400640 1 -- 172.21.15.62:0/1663065840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8b81077e0 msgr2=0x7fc8b81a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:02.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.587+0000 7fc8be400640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8b81077e0 0x7fc8b81a16d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ec50 tx=0x7fc8b000c6a0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.587+0000 7fc8be400640 1 -- 172.21.15.62:0/1663065840 shutdown_connections 2024-04-02T14:44:02.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.587+0000 7fc8be400640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc89c0a39b0 0x7fc89c0a5e70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.587+0000 7fc8be400640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8b81077e0 0x7fc8b81a16d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.587+0000 7fc8be400640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b8104eb0 0x7fc8b819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.587+0000 7fc8be400640 1 --2- 172.21.15.62:0/1663065840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b8102580 0x7fc8b819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.587+0000 7fc8be400640 1 -- 172.21.15.62:0/1663065840 >> 172.21.15.62:0/1663065840 conn(0x7fc8b80fbf70 msgr2=0x7fc8b8101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:02.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.587+0000 7fc8be400640 1 -- 172.21.15.62:0/1663065840 shutdown_connections 2024-04-02T14:44:02.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.588+0000 7fc8be400640 1 -- 172.21.15.62:0/1663065840 wait complete. 2024-04-02T14:44:02.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-02T14:44:02.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:44:02.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: [02/Apr/2024:14:44:01] ENGINE Bus STARTING 2024-04-02T14:44:02.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: [02/Apr/2024:14:44:01] ENGINE Serving on https://172.21.15.62:7150 2024-04-02T14:44:02.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: [02/Apr/2024:14:44:01] ENGINE Serving on http://172.21.15.62:8765 2024-04-02T14:44:02.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: [02/Apr/2024:14:44:01] ENGINE Bus STARTED 2024-04-02T14:44:02.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:02.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: from='client.25358 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:02.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:02 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2521124946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:02.603 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T14:44:02.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.806+0000 7f3f9c000640 1 -- 172.21.15.62:0/3640342673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f94104f30 msgr2=0x7f3f941053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:02.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.806+0000 7f3f9c000640 1 --2- 172.21.15.62:0/3640342673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f94104f30 0x7f3f941053b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009900 tx=0x7f3f8002bc40 comp rx=0 tx=0).stop 2024-04-02T14:44:02.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.806+0000 7f3f9c000640 1 -- 172.21.15.62:0/3640342673 shutdown_connections 2024-04-02T14:44:02.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.806+0000 7f3f9c000640 1 --2- 172.21.15.62:0/3640342673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f941058f0 0x7f3f9410c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.806+0000 7f3f9c000640 1 --2- 172.21.15.62:0/3640342673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f94104f30 0x7f3f941053b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.806+0000 7f3f9c000640 1 --2- 172.21.15.62:0/3640342673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f94103d40 0x7f3f94104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.806+0000 7f3f9c000640 1 -- 172.21.15.62:0/3640342673 >> 172.21.15.62:0/3640342673 conn(0x7f3f940ff2b0 msgr2=0x7f3f941016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:02.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.806+0000 7f3f9c000640 1 -- 172.21.15.62:0/3640342673 shutdown_connections 2024-04-02T14:44:02.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.806+0000 7f3f9c000640 1 -- 172.21.15.62:0/3640342673 wait complete. 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.807+0000 7f3f9c000640 1 Processor -- start 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.807+0000 7f3f9c000640 1 -- start start 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.807+0000 7f3f9c000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f94103d40 0x7f3f9419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.807+0000 7f3f9c000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f94104f30 0x7f3f9419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.807+0000 7f3f9c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f941058f0 0x7f3f941a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.807+0000 7f3f9c000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9410faf0 con 0x7f3f94103d40 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.807+0000 7f3f9c000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9410f970 con 0x7f3f94104f30 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.807+0000 7f3f9c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9410fc70 con 0x7f3f941058f0 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.808+0000 7f3f99600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f94103d40 0x7f3f9419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.808+0000 7f3f99600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f94103d40 0x7f3f9419a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35612/0 (socket says 172.21.15.62:35612) 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.808+0000 7f3f99600640 1 -- 172.21.15.62:0/1137178477 learned_addr learned my addr 172.21.15.62:0/1137178477 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.808+0000 7f3f9a000640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f941058f0 0x7f3f941a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.808+0000 7f3f98c00640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f94104f30 0x7f3f9419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:02.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.808+0000 7f3f98c00640 1 -- 172.21.15.62:0/1137178477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f941058f0 msgr2=0x7f3f941a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:02.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.808+0000 7f3f98c00640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f941058f0 0x7f3f941a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.808+0000 7f3f98c00640 1 -- 172.21.15.62:0/1137178477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f94103d40 msgr2=0x7f3f9419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:02.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.808+0000 7f3f98c00640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f94103d40 0x7f3f9419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:02.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.808+0000 7f3f98c00640 1 -- 172.21.15.62:0/1137178477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f941a1d00 con 0x7f3f94104f30 2024-04-02T14:44:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.809+0000 7f3f98c00640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f94104f30 0x7f3f9419a620 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3f80036040 tx=0x7f3f80004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.809+0000 7f3f92000640 1 -- 172.21.15.62:0/1137178477 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80043020 con 0x7f3f94104f30 2024-04-02T14:44:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.809+0000 7f3f92000640 1 -- 172.21.15.62:0/1137178477 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3f80004250 con 0x7f3f94104f30 2024-04-02T14:44:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.809+0000 7f3f92000640 1 -- 172.21.15.62:0/1137178477 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80003b20 con 0x7f3f94104f30 2024-04-02T14:44:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.809+0000 7f3f9c000640 1 -- 172.21.15.62:0/1137178477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f941a1f90 con 0x7f3f94104f30 2024-04-02T14:44:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.809+0000 7f3f9c000640 1 -- 172.21.15.62:0/1137178477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f941a2450 con 0x7f3f94104f30 2024-04-02T14:44:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.811+0000 7f3f9c000640 1 -- 172.21.15.62:0/1137178477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f9410b000 con 0x7f3f94104f30 2024-04-02T14:44:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.815+0000 7f3f92000640 1 -- 172.21.15.62:0/1137178477 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7f3f80046020 con 0x7f3f94104f30 2024-04-02T14:44:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.816+0000 7f3f92000640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3f7c0a3db0 0x7f3f7c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.816+0000 7f3f92000640 1 -- 172.21.15.62:0/1137178477 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f3f800f2e90 con 0x7f3f94104f30 2024-04-02T14:44:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.816+0000 7f3f99600640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3f7c0a3db0 0x7f3f7c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.817+0000 7f3f99600640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3f7c0a3db0 0x7f3f7c0a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3f880058c0 tx=0x7f3f88005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:02.817+0000 7f3f92000640 1 -- 172.21.15.62:0/1137178477 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3f8004d020 con 0x7f3f94104f30 2024-04-02T14:44:03.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.030+0000 7f3f9c000640 1 -- 172.21.15.62:0/1137178477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3f9402d5e0 con 0x7f3f94104f30 2024-04-02T14:44:03.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.031+0000 7f3f92000640 1 -- 172.21.15.62:0/1137178477 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f3f800bf4c0 con 0x7f3f94104f30 2024-04-02T14:44:03.029 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-02T14:44:03.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.033+0000 7f3f9c000640 1 -- 172.21.15.62:0/1137178477 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3f7c0a3db0 msgr2=0x7f3f7c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.033+0000 7f3f9c000640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3f7c0a3db0 0x7f3f7c0a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3f880058c0 tx=0x7f3f88005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:03.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.033+0000 7f3f9c000640 1 -- 172.21.15.62:0/1137178477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f94104f30 msgr2=0x7f3f9419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.033+0000 7f3f9c000640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f94104f30 0x7f3f9419a620 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3f80036040 tx=0x7f3f80004a90 comp rx=0 tx=0).stop 2024-04-02T14:44:03.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.033+0000 7f3f9c000640 1 -- 172.21.15.62:0/1137178477 shutdown_connections 2024-04-02T14:44:03.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.033+0000 7f3f9c000640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3f7c0a3db0 0x7f3f7c0a6270 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.033+0000 7f3f9c000640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f941058f0 0x7f3f941a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.033+0000 7f3f9c000640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f94104f30 0x7f3f9419a620 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.033+0000 7f3f9c000640 1 --2- 172.21.15.62:0/1137178477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f94103d40 0x7f3f9419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.033+0000 7f3f9c000640 1 -- 172.21.15.62:0/1137178477 >> 172.21.15.62:0/1137178477 conn(0x7f3f940ff2b0 msgr2=0x7f3f941081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:03.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.034+0000 7f3f9c000640 1 -- 172.21.15.62:0/1137178477 shutdown_connections 2024-04-02T14:44:03.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.034+0000 7f3f9c000640 1 -- 172.21.15.62:0/1137178477 wait complete. 2024-04-02T14:44:03.049 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T14:44:03.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.265+0000 7fe3cb400640 1 -- 172.21.15.62:0/3836617185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4101b10 msgr2=0x7fe3c4101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.265+0000 7fe3cb400640 1 --2- 172.21.15.62:0/3836617185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4101b10 0x7fe3c4101f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009900 tx=0x7fe3b802bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:03.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.265+0000 7fe3cb400640 1 -- 172.21.15.62:0/3836617185 shutdown_connections 2024-04-02T14:44:03.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.265+0000 7fe3cb400640 1 --2- 172.21.15.62:0/3836617185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c41036c0 0x7fe3c4109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.265+0000 7fe3cb400640 1 --2- 172.21.15.62:0/3836617185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c4102d00 0x7fe3c4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.265+0000 7fe3cb400640 1 --2- 172.21.15.62:0/3836617185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4101b10 0x7fe3c4101f30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.265+0000 7fe3cb400640 1 -- 172.21.15.62:0/3836617185 >> 172.21.15.62:0/3836617185 conn(0x7fe3c40fd0a0 msgr2=0x7fe3c40ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:03.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.265+0000 7fe3cb400640 1 -- 172.21.15.62:0/3836617185 shutdown_connections 2024-04-02T14:44:03.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.265+0000 7fe3cb400640 1 -- 172.21.15.62:0/3836617185 wait complete. 2024-04-02T14:44:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.266+0000 7fe3cb400640 1 Processor -- start 2024-04-02T14:44:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.266+0000 7fe3cb400640 1 -- start start 2024-04-02T14:44:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3cb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c4101b10 0x7fe3c4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3cb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4102d00 0x7fe3c4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3cb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c41036c0 0x7fe3c419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3cb400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c410d870 con 0x7fe3c41036c0 2024-04-02T14:44:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3cb400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c410d6f0 con 0x7fe3c4101b10 2024-04-02T14:44:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3cb400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c410d9f0 con 0x7fe3c4102d00 2024-04-02T14:44:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4102d00 0x7fe3c4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c4101b10 0x7fe3c4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c4101b10 0x7fe3c4197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41174/0 (socket says 172.21.15.62:41174) 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4102d00 0x7fe3c4198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39104/0 (socket says 172.21.15.62:39104) 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c8a00640 1 -- 172.21.15.62:0/1883013681 learned_addr learned my addr 172.21.15.62:0/1883013681 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c9400640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c41036c0 0x7fe3c419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c8a00640 1 -- 172.21.15.62:0/1883013681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4102d00 msgr2=0x7fe3c4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c8a00640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4102d00 0x7fe3c4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c8a00640 1 -- 172.21.15.62:0/1883013681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c41036c0 msgr2=0x7fe3c419f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c8a00640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c41036c0 0x7fe3c419f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c8a00640 1 -- 172.21.15.62:0/1883013681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c419fb70 con 0x7fe3c4101b10 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c9400640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c41036c0 0x7fe3c419f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.267+0000 7fe3c3e00640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4102d00 0x7fe3c4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.268+0000 7fe3c8a00640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c4101b10 0x7fe3c4197f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000c00 tx=0x7fe3b8004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:03.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.268+0000 7fe3c1600640 1 -- 172.21.15.62:0/1883013681 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8043020 con 0x7fe3c4101b10 2024-04-02T14:44:03.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.268+0000 7fe3c1600640 1 -- 172.21.15.62:0/1883013681 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe3b8004250 con 0x7fe3c4101b10 2024-04-02T14:44:03.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.268+0000 7fe3c1600640 1 -- 172.21.15.62:0/1883013681 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8003c00 con 0x7fe3c4101b10 2024-04-02T14:44:03.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.268+0000 7fe3cb400640 1 -- 172.21.15.62:0/1883013681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c419fe00 con 0x7fe3c4101b10 2024-04-02T14:44:03.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.268+0000 7fe3cb400640 1 -- 172.21.15.62:0/1883013681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c41a02c0 con 0x7fe3c4101b10 2024-04-02T14:44:03.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.269+0000 7fe3cb400640 1 -- 172.21.15.62:0/1883013681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c4108ea0 con 0x7fe3c4101b10 2024-04-02T14:44:03.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.273+0000 7fe3c1600640 1 -- 172.21.15.62:0/1883013681 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7fe3b8046020 con 0x7fe3c4101b10 2024-04-02T14:44:03.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.274+0000 7fe3c1600640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe3ac0a3e30 0x7fe3ac0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:03.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.275+0000 7fe3c1600640 1 -- 172.21.15.62:0/1883013681 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fe3b80f3270 con 0x7fe3c4101b10 2024-04-02T14:44:03.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.275+0000 7fe3c3e00640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe3ac0a3e30 0x7fe3ac0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:03.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.275+0000 7fe3c1600640 1 -- 172.21.15.62:0/1883013681 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe3b804d020 con 0x7fe3c4101b10 2024-04-02T14:44:03.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.276+0000 7fe3c3e00640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe3ac0a3e30 0x7fe3ac0a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4199460 tx=0x7fe3b0005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:03.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.493+0000 7fe3cb400640 1 -- 172.21.15.62:0/1883013681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe3c402d5c0 con 0x7fe3c4101b10 2024-04-02T14:44:03.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.494+0000 7fe3c1600640 1 -- 172.21.15.62:0/1883013681 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fe3b80f3020 con 0x7fe3c4101b10 2024-04-02T14:44:03.491 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-02T14:44:03.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.497+0000 7fe3cb400640 1 -- 172.21.15.62:0/1883013681 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe3ac0a3e30 msgr2=0x7fe3ac0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.497+0000 7fe3cb400640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe3ac0a3e30 0x7fe3ac0a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4199460 tx=0x7fe3b0005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:03.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.497+0000 7fe3cb400640 1 -- 172.21.15.62:0/1883013681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c4101b10 msgr2=0x7fe3c4197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.497+0000 7fe3cb400640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c4101b10 0x7fe3c4197f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000c00 tx=0x7fe3b8004a30 comp rx=0 tx=0).stop 2024-04-02T14:44:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.498+0000 7fe3cb400640 1 -- 172.21.15.62:0/1883013681 shutdown_connections 2024-04-02T14:44:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.498+0000 7fe3cb400640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe3ac0a3e30 0x7fe3ac0a62f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.498+0000 7fe3cb400640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3c41036c0 0x7fe3c419f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.498+0000 7fe3cb400640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4102d00 0x7fe3c4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.498+0000 7fe3cb400640 1 --2- 172.21.15.62:0/1883013681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c4101b10 0x7fe3c4197f50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.498+0000 7fe3cb400640 1 -- 172.21.15.62:0/1883013681 >> 172.21.15.62:0/1883013681 conn(0x7fe3c40fd0a0 msgr2=0x7fe3c4106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.498+0000 7fe3cb400640 1 -- 172.21.15.62:0/1883013681 shutdown_connections 2024-04-02T14:44:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.498+0000 7fe3cb400640 1 -- 172.21.15.62:0/1883013681 wait complete. 2024-04-02T14:44:03.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:44:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.723+0000 7f2154200640 1 -- 172.21.15.62:0/3699747145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214c1025d0 msgr2=0x7f214c108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.723+0000 7f2154200640 1 --2- 172.21.15.62:0/3699747145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214c1025d0 0x7f214c108e60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2144009ca0 tx=0x7f214402c100 comp rx=0 tx=0).stop 2024-04-02T14:44:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.723+0000 7f2154200640 1 -- 172.21.15.62:0/3699747145 shutdown_connections 2024-04-02T14:44:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.723+0000 7f2154200640 1 --2- 172.21.15.62:0/3699747145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214c1025d0 0x7f214c108e60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.723+0000 7f2154200640 1 --2- 172.21.15.62:0/3699747145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c101c10 0x7f214c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.723+0000 7f2154200640 1 --2- 172.21.15.62:0/3699747145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f214c100a20 0x7f214c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.723+0000 7f2154200640 1 -- 172.21.15.62:0/3699747145 >> 172.21.15.62:0/3699747145 conn(0x7f214c0fbf90 msgr2=0x7f214c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.724+0000 7f2154200640 1 -- 172.21.15.62:0/3699747145 shutdown_connections 2024-04-02T14:44:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.724+0000 7f2154200640 1 -- 172.21.15.62:0/3699747145 wait complete. 2024-04-02T14:44:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2154200640 1 Processor -- start 2024-04-02T14:44:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2154200640 1 -- start start 2024-04-02T14:44:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2154200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f214c100a20 0x7f214c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2154200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214c101c10 0x7f214c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2154200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c1025d0 0x7f214c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2154200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f214c10d8c0 con 0x7f214c1025d0 2024-04-02T14:44:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2154200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f214c10d740 con 0x7f214c100a20 2024-04-02T14:44:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2154200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f214c10da40 con 0x7f214c101c10 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2151800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f214c100a20 0x7f214c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2151800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f214c100a20 0x7f214c19a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41194/0 (socket says 172.21.15.62:41194) 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2151800640 1 -- 172.21.15.62:0/582104119 learned_addr learned my addr 172.21.15.62:0/582104119 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2150e00640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214c101c10 0x7f214c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.725+0000 7f2152200640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c1025d0 0x7f214c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.726+0000 7f2151800640 1 -- 172.21.15.62:0/582104119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214c101c10 msgr2=0x7f214c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.726+0000 7f2151800640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214c101c10 0x7f214c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.726+0000 7f2151800640 1 -- 172.21.15.62:0/582104119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c1025d0 msgr2=0x7f214c1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.726+0000 7f2151800640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c1025d0 0x7f214c1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.726+0000 7f2151800640 1 -- 172.21.15.62:0/582104119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c1a1c80 con 0x7f214c100a20 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.726+0000 7f2151800640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f214c100a20 0x7f214c19a060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f214000e9c0 tx=0x7f214000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.726+0000 7f214a000640 1 -- 172.21.15.62:0/582104119 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214000cde0 con 0x7f214c100a20 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.726+0000 7f214a000640 1 -- 172.21.15.62:0/582104119 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2140010930 con 0x7f214c100a20 2024-04-02T14:44:03.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.726+0000 7f2154200640 1 -- 172.21.15.62:0/582104119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214c1a1f70 con 0x7f214c100a20 2024-04-02T14:44:03.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.727+0000 7f214a000640 1 -- 172.21.15.62:0/582104119 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140018ab0 con 0x7f214c100a20 2024-04-02T14:44:03.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.727+0000 7f2154200640 1 -- 172.21.15.62:0/582104119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214c1a2430 con 0x7f214c100a20 2024-04-02T14:44:03.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.728+0000 7f2154200640 1 -- 172.21.15.62:0/582104119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214c1a2690 con 0x7f214c100a20 2024-04-02T14:44:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.730+0000 7f214a000640 1 -- 172.21.15.62:0/582104119 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7f214001f020 con 0x7f214c100a20 2024-04-02T14:44:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.731+0000 7f214a000640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21340a3e30 0x7f21340a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.731+0000 7f214a000640 1 -- 172.21.15.62:0/582104119 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f21400d0a70 con 0x7f214c100a20 2024-04-02T14:44:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.731+0000 7f2150e00640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21340a3e30 0x7f21340a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.732+0000 7f2150e00640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21340a3e30 0x7f21340a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f214c19b570 tx=0x7f2138005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:03.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.739+0000 7f214a000640 1 -- 172.21.15.62:0/582104119 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f214009cff0 con 0x7f214c100a20 2024-04-02T14:44:03.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:03 smithi062 ceph-mon[32116]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:44:03.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:03 smithi062 ceph-mon[32116]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-02T14:44:03.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:03 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1663065840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:44:03.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:03 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:03.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:03 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:03.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:03 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:03.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:03 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:03.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:03 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1137178477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:44:03.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:03 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1883013681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:44:03.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:03 smithi113 ceph-mon[36135]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:44:03.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:03 smithi113 ceph-mon[36135]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-02T14:44:03.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:03 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1663065840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:44:03.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:03 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:03.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:03 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:03.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:03 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:03.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:03 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:03.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:03 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1137178477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:44:03.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:03 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1883013681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:44:03.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.962+0000 7f2154200640 1 -- 172.21.15.62:0/582104119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f214c1a2e40 con 0x7f214c100a20 2024-04-02T14:44:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.965+0000 7f214a000640 1 -- 172.21.15.62:0/582104119 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22957 (secure 0 0 0) 0x7f214009c8f0 con 0x7f214c100a20 2024-04-02T14:44:03.963 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:44:03.963 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":91,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:44:00.087363+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/129789951":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/79979639":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/566820061":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3400451184":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:44:03.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.969+0000 7f2154200640 1 -- 172.21.15.62:0/582104119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21340a3e30 msgr2=0x7f21340a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.969+0000 7f2154200640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21340a3e30 0x7f21340a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f214c19b570 tx=0x7f2138005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:03.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.969+0000 7f2154200640 1 -- 172.21.15.62:0/582104119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f214c100a20 msgr2=0x7f214c19a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:03.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.969+0000 7f2154200640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f214c100a20 0x7f214c19a060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f214000e9c0 tx=0x7f214000ee80 comp rx=0 tx=0).stop 2024-04-02T14:44:03.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.969+0000 7f2154200640 1 -- 172.21.15.62:0/582104119 shutdown_connections 2024-04-02T14:44:03.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.969+0000 7f2154200640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21340a3e30 0x7f21340a62f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.969+0000 7f2154200640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c1025d0 0x7f214c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.969+0000 7f2154200640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214c101c10 0x7f214c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.969+0000 7f2154200640 1 --2- 172.21.15.62:0/582104119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f214c100a20 0x7f214c19a060 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:03.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.969+0000 7f2154200640 1 -- 172.21.15.62:0/582104119 >> 172.21.15.62:0/582104119 conn(0x7f214c0fbf90 msgr2=0x7f214c109820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:03.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.970+0000 7f2154200640 1 -- 172.21.15.62:0/582104119 shutdown_connections 2024-04-02T14:44:03.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:03.970+0000 7f2154200640 1 -- 172.21.15.62:0/582104119 wait complete. 2024-04-02T14:44:03.988 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T14:41:20.030393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T14:41:20.917405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:44:03.988 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:44:04.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:03 smithi204 ceph-mon[35948]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:44:04.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:03 smithi204 ceph-mon[35948]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-02T14:44:04.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:03 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1663065840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:44:04.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:03 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:04.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:03 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:04.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:03 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:04.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:03 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:04.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:03 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1137178477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:44:04.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:03 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1883013681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:44:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.201+0000 7fa160c00640 1 -- 172.21.15.62:0/774546807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa15c104f00 msgr2=0x7fa15c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.201+0000 7fa160c00640 1 --2- 172.21.15.62:0/774546807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa15c104f00 0x7fa15c105380 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa148009860 tx=0x7fa14802bb70 comp rx=0 tx=0).stop 2024-04-02T14:44:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.201+0000 7fa160c00640 1 -- 172.21.15.62:0/774546807 shutdown_connections 2024-04-02T14:44:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.201+0000 7fa160c00640 1 --2- 172.21.15.62:0/774546807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c1058c0 0x7fa15c10c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.201+0000 7fa160c00640 1 --2- 172.21.15.62:0/774546807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa15c104f00 0x7fa15c105380 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.201+0000 7fa160c00640 1 --2- 172.21.15.62:0/774546807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa15c103d10 0x7fa15c104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.201+0000 7fa160c00640 1 -- 172.21.15.62:0/774546807 >> 172.21.15.62:0/774546807 conn(0x7fa15c0ff2c0 msgr2=0x7fa15c1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:04.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.201+0000 7fa160c00640 1 -- 172.21.15.62:0/774546807 shutdown_connections 2024-04-02T14:44:04.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.201+0000 7fa160c00640 1 -- 172.21.15.62:0/774546807 wait complete. 2024-04-02T14:44:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.202+0000 7fa160c00640 1 Processor -- start 2024-04-02T14:44:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa160c00640 1 -- start start 2024-04-02T14:44:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa160c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c103d10 0x7fa15c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa160c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa15c104f00 0x7fa15c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa160c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa15c1058c0 0x7fa15c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa160c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c10fac0 con 0x7fa15c1058c0 2024-04-02T14:44:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa160c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c10f940 con 0x7fa15c104f00 2024-04-02T14:44:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa160c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c10fc40 con 0x7fa15c103d10 2024-04-02T14:44:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa159e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c103d10 0x7fa15c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa159e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c103d10 0x7fa15c19a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39144/0 (socket says 172.21.15.62:39144) 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa159e00640 1 -- 172.21.15.62:0/267871291 learned_addr learned my addr 172.21.15.62:0/267871291 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa159400640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa15c104f00 0x7fa15c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.203+0000 7fa15a800640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa15c1058c0 0x7fa15c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa159e00640 1 -- 172.21.15.62:0/267871291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa15c104f00 msgr2=0x7fa15c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa159e00640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa15c104f00 0x7fa15c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa159e00640 1 -- 172.21.15.62:0/267871291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa15c1058c0 msgr2=0x7fa15c1a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa159e00640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa15c1058c0 0x7fa15c1a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa159e00640 1 -- 172.21.15.62:0/267871291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c1a1d00 con 0x7fa15c103d10 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa159400640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa15c104f00 0x7fa15c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa15a800640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa15c1058c0 0x7fa15c1a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa159e00640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c103d10 0x7fa15c19a0e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00e9a0 tx=0x7fa14c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa152a00640 1 -- 172.21.15.62:0/267871291 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00cd80 con 0x7fa15c103d10 2024-04-02T14:44:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa152a00640 1 -- 172.21.15.62:0/267871291 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa14c010930 con 0x7fa15c103d10 2024-04-02T14:44:04.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa160c00640 1 -- 172.21.15.62:0/267871291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c1a1ff0 con 0x7fa15c103d10 2024-04-02T14:44:04.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.204+0000 7fa152a00640 1 -- 172.21.15.62:0/267871291 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c018a70 con 0x7fa15c103d10 2024-04-02T14:44:04.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.205+0000 7fa160c00640 1 -- 172.21.15.62:0/267871291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c1a2450 con 0x7fa15c103d10 2024-04-02T14:44:04.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.206+0000 7fa160c00640 1 -- 172.21.15.62:0/267871291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa15c10afd0 con 0x7fa15c103d10 2024-04-02T14:44:04.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.207+0000 7fa152a00640 1 -- 172.21.15.62:0/267871291 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7fa14c018c10 con 0x7fa15c103d10 2024-04-02T14:44:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.209+0000 7fa152a00640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa1400a3db0 0x7fa1400a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.209+0000 7fa152a00640 1 -- 172.21.15.62:0/267871291 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fa14c0d0ea0 con 0x7fa15c103d10 2024-04-02T14:44:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.209+0000 7fa159400640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa1400a3db0 0x7fa1400a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:04.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.210+0000 7fa159400640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa1400a3db0 0x7fa1400a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa148009830 tx=0x7fa148005d20 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:04.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.213+0000 7fa152a00640 1 -- 172.21.15.62:0/267871291 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa14c053f20 con 0x7fa15c103d10 2024-04-02T14:44:04.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.421+0000 7fa160c00640 1 -- 172.21.15.62:0/267871291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa15c0627a0 con 0x7fa15c103d10 2024-04-02T14:44:04.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.422+0000 7fa152a00640 1 -- 172.21.15.62:0/267871291 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fa14c09d3c0 con 0x7fa15c103d10 2024-04-02T14:44:04.419 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:44:04.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.424+0000 7fa160c00640 1 -- 172.21.15.62:0/267871291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa1400a3db0 msgr2=0x7fa1400a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:04.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.424+0000 7fa160c00640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa1400a3db0 0x7fa1400a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa148009830 tx=0x7fa148005d20 comp rx=0 tx=0).stop 2024-04-02T14:44:04.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.424+0000 7fa160c00640 1 -- 172.21.15.62:0/267871291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c103d10 msgr2=0x7fa15c19a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:04.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.424+0000 7fa160c00640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c103d10 0x7fa15c19a0e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00e9a0 tx=0x7fa14c00ee60 comp rx=0 tx=0).stop 2024-04-02T14:44:04.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.425+0000 7fa160c00640 1 -- 172.21.15.62:0/267871291 shutdown_connections 2024-04-02T14:44:04.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.425+0000 7fa160c00640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa1400a3db0 0x7fa1400a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.425+0000 7fa160c00640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa15c1058c0 0x7fa15c1a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.425+0000 7fa160c00640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa15c104f00 0x7fa15c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.425+0000 7fa160c00640 1 --2- 172.21.15.62:0/267871291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15c103d10 0x7fa15c19a0e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.425+0000 7fa160c00640 1 -- 172.21.15.62:0/267871291 >> 172.21.15.62:0/267871291 conn(0x7fa15c0ff2c0 msgr2=0x7fa15c1081a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:04.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.425+0000 7fa160c00640 1 -- 172.21.15.62:0/267871291 shutdown_connections 2024-04-02T14:44:04.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.425+0000 7fa160c00640 1 -- 172.21.15.62:0/267871291 wait complete. 2024-04-02T14:44:04.442 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T14:44:04.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.670+0000 7f008c400640 1 -- 172.21.15.62:0/2740375388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0084102580 msgr2=0x7f0084104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:04.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.670+0000 7f008c400640 1 --2- 172.21.15.62:0/2740375388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0084102580 0x7f0084104970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0078009900 tx=0x7f007802bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:04.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.670+0000 7f008c400640 1 -- 172.21.15.62:0/2740375388 shutdown_connections 2024-04-02T14:44:04.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.670+0000 7f008c400640 1 --2- 172.21.15.62:0/2740375388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00841077e0 0x7f0084109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.670+0000 7f008c400640 1 --2- 172.21.15.62:0/2740375388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0084104eb0 0x7f00841072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.670+0000 7f008c400640 1 --2- 172.21.15.62:0/2740375388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0084102580 0x7f0084104970 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.670+0000 7f008c400640 1 -- 172.21.15.62:0/2740375388 >> 172.21.15.62:0/2740375388 conn(0x7f00840fbf90 msgr2=0x7f00840fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:04.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.670+0000 7f008c400640 1 -- 172.21.15.62:0/2740375388 shutdown_connections 2024-04-02T14:44:04.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.670+0000 7f008c400640 1 -- 172.21.15.62:0/2740375388 wait complete. 2024-04-02T14:44:04.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.672+0000 7f008c400640 1 Processor -- start 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.672+0000 7f008c400640 1 -- start start 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.672+0000 7f008c400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0084102580 0x7f008419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.672+0000 7f008c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0084104eb0 0x7f008419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.672+0000 7f008c400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00841077e0 0x7f00841a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.672+0000 7f008c400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f008410d880 con 0x7f0084102580 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.672+0000 7f008c400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f008410d700 con 0x7f00841077e0 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.672+0000 7f008c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f008410da00 con 0x7f0084104eb0 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.672+0000 7f0089a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0084102580 0x7f008419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.672+0000 7f0089a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0084102580 0x7f008419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35692/0 (socket says 172.21.15.62:35692) 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.673+0000 7f0089a00640 1 -- 172.21.15.62:0/1922844062 learned_addr learned my addr 172.21.15.62:0/1922844062 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.673+0000 7f0089000640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0084104eb0 0x7f008419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.673+0000 7f008a400640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00841077e0 0x7f00841a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.673+0000 7f0089a00640 1 -- 172.21.15.62:0/1922844062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0084104eb0 msgr2=0x7f008419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:04.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.673+0000 7f0089a00640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0084104eb0 0x7f008419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.673+0000 7f0089a00640 1 -- 172.21.15.62:0/1922844062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00841077e0 msgr2=0x7f00841a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:04.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.673+0000 7f0089a00640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00841077e0 0x7f00841a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.673+0000 7f0089a00640 1 -- 172.21.15.62:0/1922844062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00841a1d30 con 0x7f0084102580 2024-04-02T14:44:04.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.673+0000 7f008a400640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00841077e0 0x7f00841a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:04.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.673+0000 7f0089000640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0084104eb0 0x7f008419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:04.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.673+0000 7f0089a00640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0084102580 0x7f008419a110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0078009230 tx=0x7f0078004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.674+0000 7f0082000640 1 -- 172.21.15.62:0/1922844062 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078043020 con 0x7f0084102580 2024-04-02T14:44:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.674+0000 7f0082000640 1 -- 172.21.15.62:0/1922844062 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0078004250 con 0x7f0084102580 2024-04-02T14:44:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.674+0000 7f008c400640 1 -- 172.21.15.62:0/1922844062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00841a1fc0 con 0x7f0084102580 2024-04-02T14:44:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.674+0000 7f0082000640 1 -- 172.21.15.62:0/1922844062 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078003b50 con 0x7f0084102580 2024-04-02T14:44:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.674+0000 7f008c400640 1 -- 172.21.15.62:0/1922844062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00841a2420 con 0x7f0084102580 2024-04-02T14:44:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.675+0000 7f008c400640 1 -- 172.21.15.62:0/1922844062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0084109330 con 0x7f0084102580 2024-04-02T14:44:04.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.678+0000 7f0082000640 1 -- 172.21.15.62:0/1922844062 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7f0078046020 con 0x7f0084102580 2024-04-02T14:44:04.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.680+0000 7f0082000640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f006c0a3db0 0x7f006c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:04.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.680+0000 7f0082000640 1 -- 172.21.15.62:0/1922844062 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f00780f2920 con 0x7f0084102580 2024-04-02T14:44:04.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.680+0000 7f0089000640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f006c0a3db0 0x7f006c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:04.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.681+0000 7f0089000640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f006c0a3db0 0x7f006c0a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f008419b620 tx=0x7f0070005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:04.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.684+0000 7f0082000640 1 -- 172.21.15.62:0/1922844062 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f007804d020 con 0x7f0084102580 2024-04-02T14:44:04.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.911+0000 7f008c400640 1 -- 172.21.15.62:0/1922844062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f008402d5c0 con 0x7f0084102580 2024-04-02T14:44:04.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.912+0000 7f0082000640 1 -- 172.21.15.62:0/1922844062 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f00780befb0 con 0x7f0084102580 2024-04-02T14:44:04.911 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-02T14:44:04.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.916+0000 7f008c400640 1 -- 172.21.15.62:0/1922844062 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f006c0a3db0 msgr2=0x7f006c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:04.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.916+0000 7f008c400640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f006c0a3db0 0x7f006c0a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f008419b620 tx=0x7f0070005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:04.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.916+0000 7f008c400640 1 -- 172.21.15.62:0/1922844062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0084102580 msgr2=0x7f008419a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:04.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.916+0000 7f008c400640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0084102580 0x7f008419a110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0078009230 tx=0x7f0078004ab0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.916+0000 7f008c400640 1 -- 172.21.15.62:0/1922844062 shutdown_connections 2024-04-02T14:44:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.917+0000 7f008c400640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f006c0a3db0 0x7f006c0a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.917+0000 7f008c400640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00841077e0 0x7f00841a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.917+0000 7f008c400640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0084104eb0 0x7f008419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.917+0000 7f008c400640 1 --2- 172.21.15.62:0/1922844062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0084102580 0x7f008419a110 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.917+0000 7f008c400640 1 -- 172.21.15.62:0/1922844062 >> 172.21.15.62:0/1922844062 conn(0x7f00840fbf90 msgr2=0x7f0084101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.917+0000 7f008c400640 1 -- 172.21.15.62:0/1922844062 shutdown_connections 2024-04-02T14:44:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:04.917+0000 7f008c400640 1 -- 172.21.15.62:0/1922844062 wait complete. 2024-04-02T14:44:04.938 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T14:44:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.151+0000 7f7f0dc00640 1 -- 172.21.15.62:0/807975566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f08104f30 msgr2=0x7f7f081053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.151+0000 7f7f0dc00640 1 --2- 172.21.15.62:0/807975566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f08104f30 0x7f7f081053b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009900 tx=0x7f7ef402bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.151+0000 7f7f0dc00640 1 -- 172.21.15.62:0/807975566 shutdown_connections 2024-04-02T14:44:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.151+0000 7f7f0dc00640 1 --2- 172.21.15.62:0/807975566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f081058f0 0x7f7f0810c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.151+0000 7f7f0dc00640 1 --2- 172.21.15.62:0/807975566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f08104f30 0x7f7f081053b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.151+0000 7f7f0dc00640 1 --2- 172.21.15.62:0/807975566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f08103d40 0x7f7f08104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.151+0000 7f7f0dc00640 1 -- 172.21.15.62:0/807975566 >> 172.21.15.62:0/807975566 conn(0x7f7f080ff2b0 msgr2=0x7f7f081016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.151+0000 7f7f0dc00640 1 -- 172.21.15.62:0/807975566 shutdown_connections 2024-04-02T14:44:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.151+0000 7f7f0dc00640 1 -- 172.21.15.62:0/807975566 wait complete. 2024-04-02T14:44:05.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.152+0000 7f7f0dc00640 1 Processor -- start 2024-04-02T14:44:05.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.152+0000 7f7f0dc00640 1 -- start start 2024-04-02T14:44:05.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.152+0000 7f7f0dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f08103d40 0x7f7f0819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:05.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f0dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f08104f30 0x7f7f0819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:05.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f0dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f081058f0 0x7f7f081a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:05.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f0dc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0810faa0 con 0x7f7f08103d40 2024-04-02T14:44:05.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f0dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0810f920 con 0x7f7f08104f30 2024-04-02T14:44:05.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f0dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0810fc20 con 0x7f7f081058f0 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f06a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f08103d40 0x7f7f0819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f06a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f08103d40 0x7f7f0819a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35698/0 (socket says 172.21.15.62:35698) 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f06a00640 1 -- 172.21.15.62:0/2780282288 learned_addr learned my addr 172.21.15.62:0/2780282288 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f07400640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f081058f0 0x7f7f081a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f06000640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f08104f30 0x7f7f0819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f06a00640 1 -- 172.21.15.62:0/2780282288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f081058f0 msgr2=0x7f7f081a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f06a00640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f081058f0 0x7f7f081a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f06a00640 1 -- 172.21.15.62:0/2780282288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f08104f30 msgr2=0x7f7f0819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f06a00640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f08104f30 0x7f7f0819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.153+0000 7f7f06a00640 1 -- 172.21.15.62:0/2780282288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f081a1cc0 con 0x7f7f08103d40 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.154+0000 7f7f06a00640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f08103d40 0x7f7f0819a0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800e9c0 tx=0x7f7ef800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.154+0000 7f7eff400640 1 -- 172.21.15.62:0/2780282288 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800cd80 con 0x7f7f08103d40 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.154+0000 7f7f0dc00640 1 -- 172.21.15.62:0/2780282288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f081a1fb0 con 0x7f7f08103d40 2024-04-02T14:44:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.154+0000 7f7eff400640 1 -- 172.21.15.62:0/2780282288 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7ef80035f0 con 0x7f7f08103d40 2024-04-02T14:44:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.154+0000 7f7eff400640 1 -- 172.21.15.62:0/2780282288 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8010690 con 0x7f7f08103d40 2024-04-02T14:44:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.154+0000 7f7f0dc00640 1 -- 172.21.15.62:0/2780282288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f081a2420 con 0x7f7f08103d40 2024-04-02T14:44:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.156+0000 7f7f0dc00640 1 -- 172.21.15.62:0/2780282288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f0810b060 con 0x7f7f08103d40 2024-04-02T14:44:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.157+0000 7f7eff400640 1 -- 172.21.15.62:0/2780282288 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f7ef80037a0 con 0x7f7f08103d40 2024-04-02T14:44:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.158+0000 7f7eff400640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7eec0a3ec0 0x7f7eec0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.159+0000 7f7eff400640 1 -- 172.21.15.62:0/2780282288 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f7ef80d0e40 con 0x7f7f08103d40 2024-04-02T14:44:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.159+0000 7f7f06000640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7eec0a3ec0 0x7f7eec0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.160+0000 7f7f06000640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7eec0a3ec0 0x7f7eec0a6380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4000c00 tx=0x7f7ef4005fd0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.166+0000 7f7eff400640 1 -- 172.21.15.62:0/2780282288 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7ef809d280 con 0x7f7f08103d40 2024-04-02T14:44:05.215 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/582104119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi204 to 174.1M 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: Unable to set osd_memory_target on smithi204 to 182606233: error parsing value: Value '182606233' is below minimum 939524096 2024-04-02T14:44:05.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi113 to 174.1M 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: Unable to set osd_memory_target on smithi113 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/267871291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:44:05.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:04 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1922844062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/582104119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi204 to 174.1M 2024-04-02T14:44:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: Unable to set osd_memory_target on smithi204 to 182606233: error parsing value: Value '182606233' is below minimum 939524096 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi113 to 174.1M 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: Unable to set osd_memory_target on smithi113 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.281 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.282 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/267871291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:44:05.282 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:04 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1922844062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:44:05.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/582104119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:05.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi204 to 174.1M 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: Unable to set osd_memory_target on smithi204 to 182606233: error parsing value: Value '182606233' is below minimum 939524096 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:05.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi113 to 174.1M 2024-04-02T14:44:05.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: Unable to set osd_memory_target on smithi113 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T14:44:05.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:44:05.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:05.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/267871291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:44:05.370 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:04 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1922844062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:44:05.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.374+0000 7f7f0dc00640 1 -- 172.21.15.62:0/2780282288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7f0802d620 con 0x7f7f08103d40 2024-04-02T14:44:05.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.375+0000 7f7eff400640 1 -- 172.21.15.62:0/2780282288 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f7ef809cb80 con 0x7f7f08103d40 2024-04-02T14:44:05.373 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-02T14:44:05.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.378+0000 7f7f0dc00640 1 -- 172.21.15.62:0/2780282288 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7eec0a3ec0 msgr2=0x7f7eec0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:05.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.378+0000 7f7f0dc00640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7eec0a3ec0 0x7f7eec0a6380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4000c00 tx=0x7f7ef4005fd0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.378+0000 7f7f0dc00640 1 -- 172.21.15.62:0/2780282288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f08103d40 msgr2=0x7f7f0819a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:05.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.378+0000 7f7f0dc00640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f08103d40 0x7f7f0819a0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800e9c0 tx=0x7f7ef800ee80 comp rx=0 tx=0).stop 2024-04-02T14:44:05.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.378+0000 7f7f0dc00640 1 -- 172.21.15.62:0/2780282288 shutdown_connections 2024-04-02T14:44:05.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.378+0000 7f7f0dc00640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7eec0a3ec0 0x7f7eec0a6380 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.378+0000 7f7f0dc00640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f081058f0 0x7f7f081a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.378+0000 7f7f0dc00640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f08104f30 0x7f7f0819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.378+0000 7f7f0dc00640 1 --2- 172.21.15.62:0/2780282288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f08103d40 0x7f7f0819a0a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.378+0000 7f7f0dc00640 1 -- 172.21.15.62:0/2780282288 >> 172.21.15.62:0/2780282288 conn(0x7f7f080ff2b0 msgr2=0x7f7f08108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:05.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.378+0000 7f7f0dc00640 1 -- 172.21.15.62:0/2780282288 shutdown_connections 2024-04-02T14:44:05.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.379+0000 7f7f0dc00640 1 -- 172.21.15.62:0/2780282288 wait complete. 2024-04-02T14:44:05.395 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-02T14:44:05.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:44:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.590+0000 7facdfe00640 1 -- 172.21.15.62:0/3901498667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face0101960 msgr2=0x7face0101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.590+0000 7facdfe00640 1 --2- 172.21.15.62:0/3901498667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face0101960 0x7face0101d80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7facd0009900 tx=0x7facd002bc50 comp rx=0 tx=0).stop 2024-04-02T14:44:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.590+0000 7facdfe00640 1 -- 172.21.15.62:0/3901498667 shutdown_connections 2024-04-02T14:44:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.590+0000 7facdfe00640 1 --2- 172.21.15.62:0/3901498667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7face0103510 0x7face0109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.590+0000 7facdfe00640 1 --2- 172.21.15.62:0/3901498667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face0102b50 0x7face0102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.590+0000 7facdfe00640 1 --2- 172.21.15.62:0/3901498667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face0101960 0x7face0101d80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.590+0000 7facdfe00640 1 -- 172.21.15.62:0/3901498667 >> 172.21.15.62:0/3901498667 conn(0x7face00fced0 msgr2=0x7face00ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.590+0000 7facdfe00640 1 -- 172.21.15.62:0/3901498667 shutdown_connections 2024-04-02T14:44:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.590+0000 7facdfe00640 1 -- 172.21.15.62:0/3901498667 wait complete. 2024-04-02T14:44:05.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.591+0000 7facdfe00640 1 Processor -- start 2024-04-02T14:44:05.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.591+0000 7facdfe00640 1 -- start start 2024-04-02T14:44:05.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.592+0000 7facdfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7face0101960 0x7face00758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:05.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.592+0000 7facdfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face0102b50 0x7face0073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:05.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.592+0000 7facdfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face0103510 0x7face0074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:05.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.592+0000 7facdfe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7face010d710 con 0x7face0101960 2024-04-02T14:44:05.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.592+0000 7facdfe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7face010d590 con 0x7face0102b50 2024-04-02T14:44:05.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.592+0000 7facdfe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7face010d890 con 0x7face0103510 2024-04-02T14:44:05.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.592+0000 7facdea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7face0101960 0x7face00758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:05.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.592+0000 7facdf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face0103510 0x7face0074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:05.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.592+0000 7facde000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face0102b50 0x7face0073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:05.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.592+0000 7facdea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7face0101960 0x7face00758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35714/0 (socket says 172.21.15.62:35714) 2024-04-02T14:44:05.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.592+0000 7facdea00640 1 -- 172.21.15.62:0/4150771145 learned_addr learned my addr 172.21.15.62:0/4150771145 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:05.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7facdf400640 1 -- 172.21.15.62:0/4150771145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face0102b50 msgr2=0x7face0073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:05.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7facdf400640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face0102b50 0x7face0073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7facdf400640 1 -- 172.21.15.62:0/4150771145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7face0101960 msgr2=0x7face00758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7facdf400640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7face0101960 0x7face00758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7facdf400640 1 -- 172.21.15.62:0/4150771145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face0074cc0 con 0x7face0103510 2024-04-02T14:44:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7facde000640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face0102b50 0x7face0073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7facdf400640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face0103510 0x7face0074430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7facd8009fd0 tx=0x7facd800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7face4a00640 1 -- 172.21.15.62:0/4150771145 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd800ce20 con 0x7face0103510 2024-04-02T14:44:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7face4a00640 1 -- 172.21.15.62:0/4150771145 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7facd8004b20 con 0x7face0103510 2024-04-02T14:44:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7facdfe00640 1 -- 172.21.15.62:0/4150771145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face01a1e30 con 0x7face0103510 2024-04-02T14:44:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7face4a00640 1 -- 172.21.15.62:0/4150771145 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8010790 con 0x7face0103510 2024-04-02T14:44:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.593+0000 7facdfe00640 1 -- 172.21.15.62:0/4150771145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face01a2240 con 0x7face0103510 2024-04-02T14:44:05.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.595+0000 7facdfe00640 1 -- 172.21.15.62:0/4150771145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face0108c80 con 0x7face0103510 2024-04-02T14:44:05.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.597+0000 7face4a00640 1 -- 172.21.15.62:0/4150771145 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7facd8044020 con 0x7face0103510 2024-04-02T14:44:05.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.598+0000 7face4a00640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7facc40a3e40 0x7facc40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:05.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.598+0000 7face4a00640 1 -- 172.21.15.62:0/4150771145 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7facd80d14d0 con 0x7face0103510 2024-04-02T14:44:05.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.598+0000 7facdea00640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7facc40a3e40 0x7facc40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:05.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.599+0000 7facdea00640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7facc40a3e40 0x7facc40a6300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7facd002ca50 tx=0x7facd0009560 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:05.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.600+0000 7face4a00640 1 -- 172.21.15.62:0/4150771145 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7facd809d910 con 0x7face0103510 2024-04-02T14:44:05.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.846+0000 7facdfe00640 1 -- 172.21.15.62:0/4150771145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7face002d370 con 0x7face0103510 2024-04-02T14:44:05.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.848+0000 7face4a00640 1 -- 172.21.15.62:0/4150771145 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7facd809d210 con 0x7face0103510 2024-04-02T14:44:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:44:05.845 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:44:00.377074+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584,24535,24533,24553,24565]},"id":1}]} 2024-04-02T14:44:05.846 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-04-02T14:44:05.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.850+0000 7facdfe00640 1 -- 172.21.15.62:0/4150771145 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7facc40a3e40 msgr2=0x7facc40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:05.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.850+0000 7facdfe00640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7facc40a3e40 0x7facc40a6300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7facd002ca50 tx=0x7facd0009560 comp rx=0 tx=0).stop 2024-04-02T14:44:05.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.850+0000 7facdfe00640 1 -- 172.21.15.62:0/4150771145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face0103510 msgr2=0x7face0074430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:05.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.850+0000 7facdfe00640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face0103510 0x7face0074430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7facd8009fd0 tx=0x7facd800ef10 comp rx=0 tx=0).stop 2024-04-02T14:44:05.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.850+0000 7facdfe00640 1 -- 172.21.15.62:0/4150771145 shutdown_connections 2024-04-02T14:44:05.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.850+0000 7facdfe00640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7facc40a3e40 0x7facc40a6300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.850+0000 7facdfe00640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face0103510 0x7face0074430 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.850+0000 7facdfe00640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face0102b50 0x7face0073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.850+0000 7facdfe00640 1 --2- 172.21.15.62:0/4150771145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7face0101960 0x7face00758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:05.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.850+0000 7facdfe00640 1 -- 172.21.15.62:0/4150771145 >> 172.21.15.62:0/4150771145 conn(0x7face00fced0 msgr2=0x7face0105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:05.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.851+0000 7facdfe00640 1 -- 172.21.15.62:0/4150771145 shutdown_connections 2024-04-02T14:44:05.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:05.851+0000 7facdfe00640 1 -- 172.21.15.62:0/4150771145 wait complete. 2024-04-02T14:44:05.869 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-02T14:44:05.869 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-02T14:44:05.869 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T14:44:06.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.100+0000 7fadf1a00640 1 -- 172.21.15.62:0/1422992306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadec1058f0 msgr2=0x7fadec10c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:06.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.100+0000 7fadf1a00640 1 --2- 172.21.15.62:0/1422992306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadec1058f0 0x7fadec10c180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fade4009be0 tx=0x7fade402bef0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.100+0000 7fadf1a00640 1 -- 172.21.15.62:0/1422992306 shutdown_connections 2024-04-02T14:44:06.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.100+0000 7fadf1a00640 1 --2- 172.21.15.62:0/1422992306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadec1058f0 0x7fadec10c180 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.100+0000 7fadf1a00640 1 --2- 172.21.15.62:0/1422992306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadec104f30 0x7fadec1053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.100+0000 7fadf1a00640 1 --2- 172.21.15.62:0/1422992306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadec103d40 0x7fadec104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.100+0000 7fadf1a00640 1 -- 172.21.15.62:0/1422992306 >> 172.21.15.62:0/1422992306 conn(0x7fadec0ff2f0 msgr2=0x7fadec101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:06.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.101+0000 7fadf1a00640 1 -- 172.21.15.62:0/1422992306 shutdown_connections 2024-04-02T14:44:06.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.101+0000 7fadf1a00640 1 -- 172.21.15.62:0/1422992306 wait complete. 2024-04-02T14:44:06.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadf1a00640 1 Processor -- start 2024-04-02T14:44:06.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadf1a00640 1 -- start start 2024-04-02T14:44:06.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadf1a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadec103d40 0x7fadec19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadf1a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadec104f30 0x7fadec19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadf1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadec1058f0 0x7fadec1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadf1a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fadec10fb40 con 0x7fadec103d40 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadf1a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fadec10f9c0 con 0x7fadec104f30 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadf1a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fadec10fcc0 con 0x7fadec1058f0 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadeaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadec103d40 0x7fadec19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadea000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadec104f30 0x7fadec19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadeaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadec103d40 0x7fadec19a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35728/0 (socket says 172.21.15.62:35728) 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.102+0000 7fadeaa00640 1 -- 172.21.15.62:0/2743858774 learned_addr learned my addr 172.21.15.62:0/2743858774 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.103+0000 7fadeb400640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadec1058f0 0x7fadec1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.103+0000 7fadeaa00640 1 -- 172.21.15.62:0/2743858774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadec1058f0 msgr2=0x7fadec1a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.103+0000 7fadeaa00640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadec1058f0 0x7fadec1a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.103+0000 7fadeaa00640 1 -- 172.21.15.62:0/2743858774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadec104f30 msgr2=0x7fadec19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:06.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.103+0000 7fadeaa00640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadec104f30 0x7fadec19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.103+0000 7fadeaa00640 1 -- 172.21.15.62:0/2743858774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadec1a1cd0 con 0x7fadec103d40 2024-04-02T14:44:06.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.103+0000 7fadea000640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadec104f30 0x7fadec19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:06.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.103+0000 7fadeaa00640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadec103d40 0x7fadec19a0b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faddc00e9f0 tx=0x7faddc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:06.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.104+0000 7fade3400640 1 -- 172.21.15.62:0/2743858774 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc00ccc0 con 0x7fadec103d40 2024-04-02T14:44:06.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.104+0000 7fade3400640 1 -- 172.21.15.62:0/2743858774 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faddc0035f0 con 0x7fadec103d40 2024-04-02T14:44:06.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.104+0000 7fadf1a00640 1 -- 172.21.15.62:0/2743858774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadec1a1fc0 con 0x7fadec103d40 2024-04-02T14:44:06.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.104+0000 7fade3400640 1 -- 172.21.15.62:0/2743858774 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc00ce60 con 0x7fadec103d40 2024-04-02T14:44:06.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.104+0000 7fadf1a00640 1 -- 172.21.15.62:0/2743858774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadec1a2420 con 0x7fadec103d40 2024-04-02T14:44:06.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.105+0000 7fadf1a00640 1 -- 172.21.15.62:0/2743858774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadec10af90 con 0x7fadec103d40 2024-04-02T14:44:06.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.107+0000 7fade3400640 1 -- 172.21.15.62:0/2743858774 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7faddc0037a0 con 0x7fadec103d40 2024-04-02T14:44:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.109+0000 7fade3400640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fadd00a3e40 0x7fadd00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.110+0000 7fade3400640 1 -- 172.21.15.62:0/2743858774 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7faddc0d0d10 con 0x7fadec103d40 2024-04-02T14:44:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.110+0000 7fadea000640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fadd00a3e40 0x7fadd00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:06.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.111+0000 7fadea000640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fadd00a3e40 0x7fadd00a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fadec19b5c0 tx=0x7fadd8005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:06.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.114+0000 7fade3400640 1 -- 172.21.15.62:0/2743858774 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7faddc0d6020 con 0x7fadec103d40 2024-04-02T14:44:06.329 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:06 smithi062 ceph-mon[32116]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-02T14:44:06.330 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:06 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2780282288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:44:06.330 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:06 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/4150771145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:44:06.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.347+0000 7fadf1a00640 1 -- 172.21.15.62:0/2743858774 --> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fadec061f40 con 0x7fadd00a3e40 2024-04-02T14:44:06.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.357+0000 7fade3400640 1 -- 172.21.15.62:0/2743858774 <== mgr.25273 v2:172.21.15.62:6800/110874546 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257789 (secure 0 0 0) 0x7fadec061f40 con 0x7fadd00a3e40 2024-04-02T14:44:06.358 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:44:06.359 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 -- 172.21.15.62:0/2743858774 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fadd00a3e40 msgr2=0x7fadd00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fadd00a3e40 0x7fadd00a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fadec19b5c0 tx=0x7fadd8005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 -- 172.21.15.62:0/2743858774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadec103d40 msgr2=0x7fadec19a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadec103d40 0x7fadec19a0b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faddc00e9f0 tx=0x7faddc00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 -- 172.21.15.62:0/2743858774 shutdown_connections 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fadd00a3e40 0x7fadd00a6300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadec1058f0 0x7fadec1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadec104f30 0x7fadec19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 --2- 172.21.15.62:0/2743858774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadec103d40 0x7fadec19a0b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 -- 172.21.15.62:0/2743858774 >> 172.21.15.62:0/2743858774 conn(0x7fadec0ff2f0 msgr2=0x7fadec108160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 -- 172.21.15.62:0/2743858774 shutdown_connections 2024-04-02T14:44:06.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.362+0000 7fadf1a00640 1 -- 172.21.15.62:0/2743858774 wait complete. 2024-04-02T14:44:06.366 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:06 smithi113 ceph-mon[36135]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-02T14:44:06.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:06 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2780282288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:44:06.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:06 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/4150771145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:44:06.378 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-02T14:44:06.118787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188191,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1773,"num_read_kb":1051,"num_write":550,"num_write_kb":4398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181749,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":934,"ondisk_log_size":934,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":405704,"kb_used_data":14736,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124536120,"statfs":{"total":1151940427776,"available":1151524986880,"internally_reserved":0,"allocated":15089664,"data_stored":8619016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.975999"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396472+0000","last_change":"2024-04-02T14:41:22.546468+0000","last_active":"2024-04-02T14:44:00.396472+0000","last_peered":"2024-04-02T14:44:00.396472+0000","last_clean":"2024-04-02T14:44:00.396472+0000","last_became_active":"2024-04-02T14:41:22.545991+0000","last_became_peered":"2024-04-02T14:41:22.545991+0000","last_unstale":"2024-04-02T14:44:00.396472+0000","last_undegraded":"2024-04-02T14:44:00.396472+0000","last_fullsized":"2024-04-02T14:44:00.396472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:34:24.520404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398615+0000","last_change":"2024-04-02T14:41:22.547738+0000","last_active":"2024-04-02T14:44:00.398615+0000","last_peered":"2024-04-02T14:44:00.398615+0000","last_clean":"2024-04-02T14:44:00.398615+0000","last_became_active":"2024-04-02T14:41:22.542372+0000","last_became_peered":"2024-04-02T14:41:22.542372+0000","last_unstale":"2024-04-02T14:44:00.398615+0000","last_undegraded":"2024-04-02T14:44:00.398615+0000","last_fullsized":"2024-04-02T14:44:00.398615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:07:26.951136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397831+0000","last_change":"2024-04-02T14:41:22.540173+0000","last_active":"2024-04-02T14:44:00.397831+0000","last_peered":"2024-04-02T14:44:00.397831+0000","last_clean":"2024-04-02T14:44:00.397831+0000","last_became_active":"2024-04-02T14:41:22.536279+0000","last_became_peered":"2024-04-02T14:41:22.536279+0000","last_unstale":"2024-04-02T14:44:00.397831+0000","last_undegraded":"2024-04-02T14:44:00.397831+0000","last_fullsized":"2024-04-02T14:44:00.397831+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:43:40.826934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099445+0000","last_change":"2024-04-02T14:42:21.251675+0000","last_active":"2024-04-02T14:44:00.099445+0000","last_peered":"2024-04-02T14:44:00.099445+0000","last_clean":"2024-04-02T14:44:00.099445+0000","last_became_active":"2024-04-02T14:42:21.250446+0000","last_became_peered":"2024-04-02T14:42:21.250446+0000","last_unstale":"2024-04-02T14:44:00.099445+0000","last_undegraded":"2024-04-02T14:44:00.099445+0000","last_fullsized":"2024-04-02T14:44:00.099445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:11.005276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238025+0000","last_change":"2024-04-02T14:41:22.554301+0000","last_active":"2024-04-02T14:44:00.238025+0000","last_peered":"2024-04-02T14:44:00.238025+0000","last_clean":"2024-04-02T14:44:00.238025+0000","last_became_active":"2024-04-02T14:41:22.554028+0000","last_became_peered":"2024-04-02T14:41:22.554028+0000","last_unstale":"2024-04-02T14:44:00.238025+0000","last_undegraded":"2024-04-02T14:44:00.238025+0000","last_fullsized":"2024-04-02T14:44:00.238025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:51:59.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253420+0000","last_change":"2024-04-02T14:41:22.543934+0000","last_active":"2024-04-02T14:44:00.253420+0000","last_peered":"2024-04-02T14:44:00.253420+0000","last_clean":"2024-04-02T14:44:00.253420+0000","last_became_active":"2024-04-02T14:41:22.543392+0000","last_became_peered":"2024-04-02T14:41:22.543392+0000","last_unstale":"2024-04-02T14:44:00.253420+0000","last_undegraded":"2024-04-02T14:44:00.253420+0000","last_fullsized":"2024-04-02T14:44:00.253420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:28:30.234356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391852+0000","last_change":"2024-04-02T14:41:22.547460+0000","last_active":"2024-04-02T14:44:00.391852+0000","last_peered":"2024-04-02T14:44:00.391852+0000","last_clean":"2024-04-02T14:44:00.391852+0000","last_became_active":"2024-04-02T14:41:22.547123+0000","last_became_peered":"2024-04-02T14:41:22.547123+0000","last_unstale":"2024-04-02T14:44:00.391852+0000","last_undegraded":"2024-04-02T14:44:00.391852+0000","last_fullsized":"2024-04-02T14:44:00.391852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:24:34.302530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239097+0000","last_change":"2024-04-02T14:41:22.551407+0000","last_active":"2024-04-02T14:44:00.239097+0000","last_peered":"2024-04-02T14:44:00.239097+0000","last_clean":"2024-04-02T14:44:00.239097+0000","last_became_active":"2024-04-02T14:41:22.538304+0000","last_became_peered":"2024-04-02T14:41:22.538304+0000","last_unstale":"2024-04-02T14:44:00.239097+0000","last_undegraded":"2024-04-02T14:44:00.239097+0000","last_fullsized":"2024-04-02T14:44:00.239097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:27.933380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362921+0000","last_change":"2024-04-02T14:41:22.523931+0000","last_active":"2024-04-02T14:44:00.362921+0000","last_peered":"2024-04-02T14:44:00.362921+0000","last_clean":"2024-04-02T14:44:00.362921+0000","last_became_active":"2024-04-02T14:41:22.523622+0000","last_became_peered":"2024-04-02T14:41:22.523622+0000","last_unstale":"2024-04-02T14:44:00.362921+0000","last_undegraded":"2024-04-02T14:44:00.362921+0000","last_fullsized":"2024-04-02T14:44:00.362921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:17.808290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253719+0000","last_change":"2024-04-02T14:41:22.525791+0000","last_active":"2024-04-02T14:44:00.253719+0000","last_peered":"2024-04-02T14:44:00.253719+0000","last_clean":"2024-04-02T14:44:00.253719+0000","last_became_active":"2024-04-02T14:41:22.525434+0000","last_became_peered":"2024-04-02T14:41:22.525434+0000","last_unstale":"2024-04-02T14:44:00.253719+0000","last_undegraded":"2024-04-02T14:44:00.253719+0000","last_fullsized":"2024-04-02T14:44:00.253719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:59.905337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392238+0000","last_change":"2024-04-02T14:41:22.527240+0000","last_active":"2024-04-02T14:44:00.392238+0000","last_peered":"2024-04-02T14:44:00.392238+0000","last_clean":"2024-04-02T14:44:00.392238+0000","last_became_active":"2024-04-02T14:41:22.526412+0000","last_became_peered":"2024-04-02T14:41:22.526412+0000","last_unstale":"2024-04-02T14:44:00.392238+0000","last_undegraded":"2024-04-02T14:44:00.392238+0000","last_fullsized":"2024-04-02T14:44:00.392238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:47.397012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396098+0000","last_change":"2024-04-02T14:41:22.527281+0000","last_active":"2024-04-02T14:44:00.396098+0000","last_peered":"2024-04-02T14:44:00.396098+0000","last_clean":"2024-04-02T14:44:00.396098+0000","last_became_active":"2024-04-02T14:41:22.526946+0000","last_became_peered":"2024-04-02T14:41:22.526946+0000","last_unstale":"2024-04-02T14:44:00.396098+0000","last_undegraded":"2024-04-02T14:44:00.396098+0000","last_fullsized":"2024-04-02T14:44:00.396098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:42.798282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398248+0000","last_change":"2024-04-02T14:41:22.534079+0000","last_active":"2024-04-02T14:44:00.398248+0000","last_peered":"2024-04-02T14:44:00.398248+0000","last_clean":"2024-04-02T14:44:00.398248+0000","last_became_active":"2024-04-02T14:41:22.532233+0000","last_became_peered":"2024-04-02T14:41:22.532233+0000","last_unstale":"2024-04-02T14:44:00.398248+0000","last_undegraded":"2024-04-02T14:44:00.398248+0000","last_fullsized":"2024-04-02T14:44:00.398248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:27:47.122933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397389+0000","last_change":"2024-04-02T14:41:22.544936+0000","last_active":"2024-04-02T14:44:00.397389+0000","last_peered":"2024-04-02T14:44:00.397389+0000","last_clean":"2024-04-02T14:44:00.397389+0000","last_became_active":"2024-04-02T14:41:22.541805+0000","last_became_peered":"2024-04-02T14:41:22.541805+0000","last_unstale":"2024-04-02T14:44:00.397389+0000","last_undegraded":"2024-04-02T14:44:00.397389+0000","last_fullsized":"2024-04-02T14:44:00.397389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:23.992807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397232+0000","last_change":"2024-04-02T14:41:22.534860+0000","last_active":"2024-04-02T14:44:00.397232+0000","last_peered":"2024-04-02T14:44:00.397232+0000","last_clean":"2024-04-02T14:44:00.397232+0000","last_became_active":"2024-04-02T14:41:22.534147+0000","last_became_peered":"2024-04-02T14:41:22.534147+0000","last_unstale":"2024-04-02T14:44:00.397232+0000","last_undegraded":"2024-04-02T14:44:00.397232+0000","last_fullsized":"2024-04-02T14:44:00.397232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:25.249364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362410+0000","last_change":"2024-04-02T14:41:22.537654+0000","last_active":"2024-04-02T14:44:00.362410+0000","last_peered":"2024-04-02T14:44:00.362410+0000","last_clean":"2024-04-02T14:44:00.362410+0000","last_became_active":"2024-04-02T14:41:22.537304+0000","last_became_peered":"2024-04-02T14:41:22.537304+0000","last_unstale":"2024-04-02T14:44:00.362410+0000","last_undegraded":"2024-04-02T14:44:00.362410+0000","last_fullsized":"2024-04-02T14:44:00.362410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:25.599724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396524+0000","last_change":"2024-04-02T14:41:22.546775+0000","last_active":"2024-04-02T14:44:00.396524+0000","last_peered":"2024-04-02T14:44:00.396524+0000","last_clean":"2024-04-02T14:44:00.396524+0000","last_became_active":"2024-04-02T14:41:22.546100+0000","last_became_peered":"2024-04-02T14:41:22.546100+0000","last_unstale":"2024-04-02T14:44:00.396524+0000","last_undegraded":"2024-04-02T14:44:00.396524+0000","last_fullsized":"2024-04-02T14:44:00.396524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:42:02.559335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099394+0000","last_change":"2024-04-02T14:41:22.543988+0000","last_active":"2024-04-02T14:44:00.099394+0000","last_peered":"2024-04-02T14:44:00.099394+0000","last_clean":"2024-04-02T14:44:00.099394+0000","last_became_active":"2024-04-02T14:41:22.543036+0000","last_became_peered":"2024-04-02T14:41:22.543036+0000","last_unstale":"2024-04-02T14:44:00.099394+0000","last_undegraded":"2024-04-02T14:44:00.099394+0000","last_fullsized":"2024-04-02T14:44:00.099394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:29.470929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098748+0000","last_change":"2024-04-02T14:41:22.543954+0000","last_active":"2024-04-02T14:44:00.098748+0000","last_peered":"2024-04-02T14:44:00.098748+0000","last_clean":"2024-04-02T14:44:00.098748+0000","last_became_active":"2024-04-02T14:41:22.543624+0000","last_became_peered":"2024-04-02T14:41:22.543624+0000","last_unstale":"2024-04-02T14:44:00.098748+0000","last_undegraded":"2024-04-02T14:44:00.098748+0000","last_fullsized":"2024-04-02T14:44:00.098748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:46.933877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":188,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.390351+0000","last_change":"2024-04-02T14:41:21.477341+0000","last_active":"2024-04-02T14:44:00.390351+0000","last_peered":"2024-04-02T14:44:00.390351+0000","last_clean":"2024-04-02T14:44:00.390351+0000","last_became_active":"2024-04-02T14:41:21.476708+0000","last_became_peered":"2024-04-02T14:41:21.476708+0000","last_unstale":"2024-04-02T14:44:00.390351+0000","last_undegraded":"2024-04-02T14:44:00.390351+0000","last_fullsized":"2024-04-02T14:44:00.390351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:34.867570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.241083+0000","last_change":"2024-04-02T14:41:22.551293+0000","last_active":"2024-04-02T14:44:00.241083+0000","last_peered":"2024-04-02T14:44:00.241083+0000","last_clean":"2024-04-02T14:44:00.241083+0000","last_became_active":"2024-04-02T14:41:22.547819+0000","last_became_peered":"2024-04-02T14:41:22.547819+0000","last_unstale":"2024-04-02T14:44:00.241083+0000","last_undegraded":"2024-04-02T14:44:00.241083+0000","last_fullsized":"2024-04-02T14:44:00.241083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:40.393005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.241214+0000","last_change":"2024-04-02T14:41:21.487096+0000","last_active":"2024-04-02T14:44:00.241214+0000","last_peered":"2024-04-02T14:44:00.241214+0000","last_clean":"2024-04-02T14:44:00.241214+0000","last_became_active":"2024-04-02T14:41:21.485883+0000","last_became_peered":"2024-04-02T14:41:21.485883+0000","last_unstale":"2024-04-02T14:44:00.241214+0000","last_undegraded":"2024-04-02T14:44:00.241214+0000","last_fullsized":"2024-04-02T14:44:00.241214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:24.017671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398946+0000","last_change":"2024-04-02T14:41:22.549234+0000","last_active":"2024-04-02T14:44:00.398946+0000","last_peered":"2024-04-02T14:44:00.398946+0000","last_clean":"2024-04-02T14:44:00.398946+0000","last_became_active":"2024-04-02T14:41:22.548855+0000","last_became_peered":"2024-04-02T14:41:22.548855+0000","last_unstale":"2024-04-02T14:44:00.398946+0000","last_undegraded":"2024-04-02T14:44:00.398946+0000","last_fullsized":"2024-04-02T14:44:00.398946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:05.047383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238406+0000","last_change":"2024-04-02T14:41:21.471673+0000","last_active":"2024-04-02T14:44:00.238406+0000","last_peered":"2024-04-02T14:44:00.238406+0000","last_clean":"2024-04-02T14:44:00.238406+0000","last_became_active":"2024-04-02T14:41:21.471292+0000","last_became_peered":"2024-04-02T14:41:21.471292+0000","last_unstale":"2024-04-02T14:44:00.238406+0000","last_undegraded":"2024-04-02T14:44:00.238406+0000","last_fullsized":"2024-04-02T14:44:00.238406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:42:55.665938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362654+0000","last_change":"2024-04-02T14:41:22.537665+0000","last_active":"2024-04-02T14:44:00.362654+0000","last_peered":"2024-04-02T14:44:00.362654+0000","last_clean":"2024-04-02T14:44:00.362654+0000","last_became_active":"2024-04-02T14:41:22.537274+0000","last_became_peered":"2024-04-02T14:41:22.537274+0000","last_unstale":"2024-04-02T14:44:00.362654+0000","last_undegraded":"2024-04-02T14:44:00.362654+0000","last_fullsized":"2024-04-02T14:44:00.362654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:14:39.200735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238373+0000","last_change":"2024-04-02T14:41:21.466547+0000","last_active":"2024-04-02T14:44:00.238373+0000","last_peered":"2024-04-02T14:44:00.238373+0000","last_clean":"2024-04-02T14:44:00.238373+0000","last_became_active":"2024-04-02T14:41:21.466063+0000","last_became_peered":"2024-04-02T14:41:21.466063+0000","last_unstale":"2024-04-02T14:44:00.238373+0000","last_undegraded":"2024-04-02T14:44:00.238373+0000","last_fullsized":"2024-04-02T14:44:00.238373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:39.201543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250276+0000","last_change":"2024-04-02T14:41:22.523984+0000","last_active":"2024-04-02T14:44:00.250276+0000","last_peered":"2024-04-02T14:44:00.250276+0000","last_clean":"2024-04-02T14:44:00.250276+0000","last_became_active":"2024-04-02T14:41:22.523524+0000","last_became_peered":"2024-04-02T14:41:22.523524+0000","last_unstale":"2024-04-02T14:44:00.250276+0000","last_undegraded":"2024-04-02T14:44:00.250276+0000","last_fullsized":"2024-04-02T14:44:00.250276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:31.542588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":196,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.440836+0000","last_change":"2024-04-02T14:42:21.255204+0000","last_active":"2024-04-02T14:44:00.440836+0000","last_peered":"2024-04-02T14:44:00.440836+0000","last_clean":"2024-04-02T14:44:00.440836+0000","last_became_active":"2024-04-02T14:42:21.248702+0000","last_became_peered":"2024-04-02T14:42:21.248702+0000","last_unstale":"2024-04-02T14:44:00.440836+0000","last_undegraded":"2024-04-02T14:44:00.440836+0000","last_fullsized":"2024-04-02T14:44:00.440836+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.723036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":151,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238112+0000","last_change":"2024-04-02T14:41:22.544107+0000","last_active":"2024-04-02T14:44:00.238112+0000","last_peered":"2024-04-02T14:44:00.238112+0000","last_clean":"2024-04-02T14:44:00.238112+0000","last_became_active":"2024-04-02T14:41:22.543739+0000","last_became_peered":"2024-04-02T14:41:22.543739+0000","last_unstale":"2024-04-02T14:44:00.238112+0000","last_undegraded":"2024-04-02T14:44:00.238112+0000","last_fullsized":"2024-04-02T14:44:00.238112+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:49:08.921668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253407+0000","last_change":"2024-04-02T14:41:21.479714+0000","last_active":"2024-04-02T14:44:00.253407+0000","last_peered":"2024-04-02T14:44:00.253407+0000","last_clean":"2024-04-02T14:44:00.253407+0000","last_became_active":"2024-04-02T14:41:21.479329+0000","last_became_peered":"2024-04-02T14:41:21.479329+0000","last_unstale":"2024-04-02T14:44:00.253407+0000","last_undegraded":"2024-04-02T14:44:00.253407+0000","last_fullsized":"2024-04-02T14:44:00.253407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:09.021017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099863+0000","last_change":"2024-04-02T14:42:21.254332+0000","last_active":"2024-04-02T14:44:00.099863+0000","last_peered":"2024-04-02T14:44:00.099863+0000","last_clean":"2024-04-02T14:44:00.099863+0000","last_became_active":"2024-04-02T14:42:21.251266+0000","last_became_peered":"2024-04-02T14:42:21.251266+0000","last_unstale":"2024-04-02T14:44:00.099863+0000","last_undegraded":"2024-04-02T14:44:00.099863+0000","last_fullsized":"2024-04-02T14:44:00.099863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:51:18.343098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396860+0000","last_change":"2024-04-02T14:41:21.484054+0000","last_active":"2024-04-02T14:44:00.396860+0000","last_peered":"2024-04-02T14:44:00.396860+0000","last_clean":"2024-04-02T14:44:00.396860+0000","last_became_active":"2024-04-02T14:41:21.483688+0000","last_became_peered":"2024-04-02T14:41:21.483688+0000","last_unstale":"2024-04-02T14:44:00.396860+0000","last_undegraded":"2024-04-02T14:44:00.396860+0000","last_fullsized":"2024-04-02T14:44:00.396860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:24.114180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238925+0000","last_change":"2024-04-02T14:41:22.553274+0000","last_active":"2024-04-02T14:44:00.238925+0000","last_peered":"2024-04-02T14:44:00.238925+0000","last_clean":"2024-04-02T14:44:00.238925+0000","last_became_active":"2024-04-02T14:41:22.545412+0000","last_became_peered":"2024-04-02T14:41:22.545412+0000","last_unstale":"2024-04-02T14:44:00.238925+0000","last_undegraded":"2024-04-02T14:44:00.238925+0000","last_fullsized":"2024-04-02T14:44:00.238925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:21.917119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.249831+0000","last_change":"2024-04-02T14:41:21.468582+0000","last_active":"2024-04-02T14:44:00.249831+0000","last_peered":"2024-04-02T14:44:00.249831+0000","last_clean":"2024-04-02T14:44:00.249831+0000","last_became_active":"2024-04-02T14:41:21.468081+0000","last_became_peered":"2024-04-02T14:41:21.468081+0000","last_unstale":"2024-04-02T14:44:00.249831+0000","last_undegraded":"2024-04-02T14:44:00.249831+0000","last_fullsized":"2024-04-02T14:44:00.249831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:19.593081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099196+0000","last_change":"2024-04-02T14:41:22.555627+0000","last_active":"2024-04-02T14:44:00.099196+0000","last_peered":"2024-04-02T14:44:00.099196+0000","last_clean":"2024-04-02T14:44:00.099196+0000","last_became_active":"2024-04-02T14:41:22.555171+0000","last_became_peered":"2024-04-02T14:41:22.555171+0000","last_unstale":"2024-04-02T14:44:00.099196+0000","last_undegraded":"2024-04-02T14:44:00.099196+0000","last_fullsized":"2024-04-02T14:44:00.099196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:21:28.161869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":330,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.403886+0000","last_change":"2024-04-02T14:41:21.475004+0000","last_active":"2024-04-02T14:44:00.403886+0000","last_peered":"2024-04-02T14:44:00.403886+0000","last_clean":"2024-04-02T14:44:00.403886+0000","last_became_active":"2024-04-02T14:41:21.474621+0000","last_became_peered":"2024-04-02T14:41:21.474621+0000","last_unstale":"2024-04-02T14:44:00.403886+0000","last_undegraded":"2024-04-02T14:44:00.403886+0000","last_fullsized":"2024-04-02T14:44:00.403886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:38.779276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":302,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362877+0000","last_change":"2024-04-02T14:41:22.543100+0000","last_active":"2024-04-02T14:44:00.362877+0000","last_peered":"2024-04-02T14:44:00.362877+0000","last_clean":"2024-04-02T14:44:00.362877+0000","last_became_active":"2024-04-02T14:41:22.542697+0000","last_became_peered":"2024-04-02T14:41:22.542697+0000","last_unstale":"2024-04-02T14:44:00.362877+0000","last_undegraded":"2024-04-02T14:44:00.362877+0000","last_fullsized":"2024-04-02T14:44:00.362877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:50:52.412879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392108+0000","last_change":"2024-04-02T14:41:21.468626+0000","last_active":"2024-04-02T14:44:00.392108+0000","last_peered":"2024-04-02T14:44:00.392108+0000","last_clean":"2024-04-02T14:44:00.392108+0000","last_became_active":"2024-04-02T14:41:21.468041+0000","last_became_peered":"2024-04-02T14:41:21.468041+0000","last_unstale":"2024-04-02T14:44:00.392108+0000","last_undegraded":"2024-04-02T14:44:00.392108+0000","last_fullsized":"2024-04-02T14:44:00.392108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:55.064033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398874+0000","last_change":"2024-04-02T14:41:22.539584+0000","last_active":"2024-04-02T14:44:00.398874+0000","last_peered":"2024-04-02T14:44:00.398874+0000","last_clean":"2024-04-02T14:44:00.398874+0000","last_became_active":"2024-04-02T14:41:22.539218+0000","last_became_peered":"2024-04-02T14:41:22.539218+0000","last_unstale":"2024-04-02T14:44:00.398874+0000","last_undegraded":"2024-04-02T14:44:00.398874+0000","last_fullsized":"2024-04-02T14:44:00.398874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:47:56.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250358+0000","last_change":"2024-04-02T14:41:21.481208+0000","last_active":"2024-04-02T14:44:00.250358+0000","last_peered":"2024-04-02T14:44:00.250358+0000","last_clean":"2024-04-02T14:44:00.250358+0000","last_became_active":"2024-04-02T14:41:21.480782+0000","last_became_peered":"2024-04-02T14:41:21.480782+0000","last_unstale":"2024-04-02T14:44:00.250358+0000","last_undegraded":"2024-04-02T14:44:00.250358+0000","last_fullsized":"2024-04-02T14:44:00.250358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:12.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362685+0000","last_change":"2024-04-02T14:41:22.517397+0000","last_active":"2024-04-02T14:44:00.362685+0000","last_peered":"2024-04-02T14:44:00.362685+0000","last_clean":"2024-04-02T14:44:00.362685+0000","last_became_active":"2024-04-02T14:41:22.517027+0000","last_became_peered":"2024-04-02T14:41:22.517027+0000","last_unstale":"2024-04-02T14:44:00.362685+0000","last_undegraded":"2024-04-02T14:44:00.362685+0000","last_fullsized":"2024-04-02T14:44:00.362685+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:53.129949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":368,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391430+0000","last_change":"2024-04-02T14:41:21.469643+0000","last_active":"2024-04-02T14:44:00.391430+0000","last_peered":"2024-04-02T14:44:00.391430+0000","last_clean":"2024-04-02T14:44:00.391430+0000","last_became_active":"2024-04-02T14:41:21.468730+0000","last_became_peered":"2024-04-02T14:41:21.468730+0000","last_unstale":"2024-04-02T14:44:00.391430+0000","last_undegraded":"2024-04-02T14:44:00.391430+0000","last_fullsized":"2024-04-02T14:44:00.391430+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:10.983551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64646,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":311,"num_read_kb":220,"num_write":37,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238145+0000","last_change":"2024-04-02T14:41:22.525052+0000","last_active":"2024-04-02T14:44:00.238145+0000","last_peered":"2024-04-02T14:44:00.238145+0000","last_clean":"2024-04-02T14:44:00.238145+0000","last_became_active":"2024-04-02T14:41:22.524691+0000","last_became_peered":"2024-04-02T14:41:22.524691+0000","last_unstale":"2024-04-02T14:44:00.238145+0000","last_undegraded":"2024-04-02T14:44:00.238145+0000","last_fullsized":"2024-04-02T14:44:00.238145+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:16.386020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098518+0000","last_change":"2024-04-02T14:41:21.476988+0000","last_active":"2024-04-02T14:44:00.098518+0000","last_peered":"2024-04-02T14:44:00.098518+0000","last_clean":"2024-04-02T14:44:00.098518+0000","last_became_active":"2024-04-02T14:41:21.476549+0000","last_became_peered":"2024-04-02T14:41:21.476549+0000","last_unstale":"2024-04-02T14:44:00.098518+0000","last_undegraded":"2024-04-02T14:44:00.098518+0000","last_fullsized":"2024-04-02T14:44:00.098518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:47.498626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397582+0000","last_change":"2024-04-02T14:41:22.544746+0000","last_active":"2024-04-02T14:44:00.397582+0000","last_peered":"2024-04-02T14:44:00.397582+0000","last_clean":"2024-04-02T14:44:00.397582+0000","last_became_active":"2024-04-02T14:41:22.544334+0000","last_became_peered":"2024-04-02T14:41:22.544334+0000","last_unstale":"2024-04-02T14:44:00.397582+0000","last_undegraded":"2024-04-02T14:44:00.397582+0000","last_fullsized":"2024-04-02T14:44:00.397582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:45:20.094455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250015+0000","last_change":"2024-04-02T14:41:21.472458+0000","last_active":"2024-04-02T14:44:00.250015+0000","last_peered":"2024-04-02T14:44:00.250015+0000","last_clean":"2024-04-02T14:44:00.250015+0000","last_became_active":"2024-04-02T14:41:21.472105+0000","last_became_peered":"2024-04-02T14:41:21.472105+0000","last_unstale":"2024-04-02T14:44:00.250015+0000","last_undegraded":"2024-04-02T14:44:00.250015+0000","last_fullsized":"2024-04-02T14:44:00.250015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:44.075807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392138+0000","last_change":"2024-04-02T14:41:22.537492+0000","last_active":"2024-04-02T14:44:00.392138+0000","last_peered":"2024-04-02T14:44:00.392138+0000","last_clean":"2024-04-02T14:44:00.392138+0000","last_became_active":"2024-04-02T14:41:22.537067+0000","last_became_peered":"2024-04-02T14:41:22.537067+0000","last_unstale":"2024-04-02T14:44:00.392138+0000","last_undegraded":"2024-04-02T14:44:00.392138+0000","last_fullsized":"2024-04-02T14:44:00.392138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:08.704340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":181,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.403962+0000","last_change":"2024-04-02T14:41:21.466017+0000","last_active":"2024-04-02T14:44:00.403962+0000","last_peered":"2024-04-02T14:44:00.403962+0000","last_clean":"2024-04-02T14:44:00.403962+0000","last_became_active":"2024-04-02T14:41:21.465600+0000","last_became_peered":"2024-04-02T14:41:21.465600+0000","last_unstale":"2024-04-02T14:44:00.403962+0000","last_undegraded":"2024-04-02T14:44:00.403962+0000","last_fullsized":"2024-04-02T14:44:00.403962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:52:08.131125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391756+0000","last_change":"2024-04-02T14:41:22.526628+0000","last_active":"2024-04-02T14:44:00.391756+0000","last_peered":"2024-04-02T14:44:00.391756+0000","last_clean":"2024-04-02T14:44:00.391756+0000","last_became_active":"2024-04-02T14:41:22.526283+0000","last_became_peered":"2024-04-02T14:41:22.526283+0000","last_unstale":"2024-04-02T14:44:00.391756+0000","last_undegraded":"2024-04-02T14:44:00.391756+0000","last_fullsized":"2024-04-02T14:44:00.391756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:41.812330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.237864+0000","last_change":"2024-04-02T14:41:21.475853+0000","last_active":"2024-04-02T14:44:00.237864+0000","last_peered":"2024-04-02T14:44:00.237864+0000","last_clean":"2024-04-02T14:44:00.237864+0000","last_became_active":"2024-04-02T14:41:21.475555+0000","last_became_peered":"2024-04-02T14:41:21.475555+0000","last_unstale":"2024-04-02T14:44:00.237864+0000","last_undegraded":"2024-04-02T14:44:00.237864+0000","last_fullsized":"2024-04-02T14:44:00.237864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:49.170906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.249931+0000","last_change":"2024-04-02T14:41:22.523848+0000","last_active":"2024-04-02T14:44:00.249931+0000","last_peered":"2024-04-02T14:44:00.249931+0000","last_clean":"2024-04-02T14:44:00.249931+0000","last_became_active":"2024-04-02T14:41:22.523476+0000","last_became_peered":"2024-04-02T14:41:22.523476+0000","last_unstale":"2024-04-02T14:44:00.249931+0000","last_undegraded":"2024-04-02T14:44:00.249931+0000","last_fullsized":"2024-04-02T14:44:00.249931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:47:46.933264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238399+0000","last_change":"2024-04-02T14:41:21.468556+0000","last_active":"2024-04-02T14:44:00.238399+0000","last_peered":"2024-04-02T14:44:00.238399+0000","last_clean":"2024-04-02T14:44:00.238399+0000","last_became_active":"2024-04-02T14:41:21.468215+0000","last_became_peered":"2024-04-02T14:41:21.468215+0000","last_unstale":"2024-04-02T14:44:00.238399+0000","last_undegraded":"2024-04-02T14:44:00.238399+0000","last_fullsized":"2024-04-02T14:44:00.238399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:04.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396416+0000","last_change":"2024-04-02T14:41:22.524228+0000","last_active":"2024-04-02T14:44:00.396416+0000","last_peered":"2024-04-02T14:44:00.396416+0000","last_clean":"2024-04-02T14:44:00.396416+0000","last_became_active":"2024-04-02T14:41:22.523839+0000","last_became_peered":"2024-04-02T14:41:22.523839+0000","last_unstale":"2024-04-02T14:44:00.396416+0000","last_undegraded":"2024-04-02T14:44:00.396416+0000","last_fullsized":"2024-04-02T14:44:00.396416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:13.094123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099113+0000","last_change":"2024-04-02T14:42:21.260431+0000","last_active":"2024-04-02T14:44:00.099113+0000","last_peered":"2024-04-02T14:44:00.099113+0000","last_clean":"2024-04-02T14:44:00.099113+0000","last_became_active":"2024-04-02T14:42:21.259060+0000","last_became_peered":"2024-04-02T14:42:21.259060+0000","last_unstale":"2024-04-02T14:44:00.099113+0000","last_undegraded":"2024-04-02T14:44:00.099113+0000","last_fullsized":"2024-04-02T14:44:00.099113+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:07.753909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253595+0000","last_change":"2024-04-02T14:41:22.542847+0000","last_active":"2024-04-02T14:44:00.253595+0000","last_peered":"2024-04-02T14:44:00.253595+0000","last_clean":"2024-04-02T14:44:00.253595+0000","last_became_active":"2024-04-02T14:41:22.542476+0000","last_became_peered":"2024-04-02T14:41:22.542476+0000","last_unstale":"2024-04-02T14:44:00.253595+0000","last_undegraded":"2024-04-02T14:44:00.253595+0000","last_fullsized":"2024-04-02T14:44:00.253595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:27:12.102931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392293+0000","last_change":"2024-04-02T14:41:21.483382+0000","last_active":"2024-04-02T14:44:00.392293+0000","last_peered":"2024-04-02T14:44:00.392293+0000","last_clean":"2024-04-02T14:44:00.392293+0000","last_became_active":"2024-04-02T14:41:21.482996+0000","last_became_peered":"2024-04-02T14:41:21.482996+0000","last_unstale":"2024-04-02T14:44:00.392293+0000","last_undegraded":"2024-04-02T14:44:00.392293+0000","last_fullsized":"2024-04-02T14:44:00.392293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:37.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253567+0000","last_change":"2024-04-02T14:42:21.257441+0000","last_active":"2024-04-02T14:44:00.253567+0000","last_peered":"2024-04-02T14:44:00.253567+0000","last_clean":"2024-04-02T14:44:00.253567+0000","last_became_active":"2024-04-02T14:42:21.257003+0000","last_became_peered":"2024-04-02T14:42:21.257003+0000","last_unstale":"2024-04-02T14:44:00.253567+0000","last_undegraded":"2024-04-02T14:44:00.253567+0000","last_fullsized":"2024-04-02T14:44:00.253567+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:19:10.793720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250298+0000","last_change":"2024-04-02T14:41:22.525853+0000","last_active":"2024-04-02T14:44:00.250298+0000","last_peered":"2024-04-02T14:44:00.250298+0000","last_clean":"2024-04-02T14:44:00.250298+0000","last_became_active":"2024-04-02T14:41:22.525553+0000","last_became_peered":"2024-04-02T14:41:22.525553+0000","last_unstale":"2024-04-02T14:44:00.250298+0000","last_undegraded":"2024-04-02T14:44:00.250298+0000","last_fullsized":"2024-04-02T14:44:00.250298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:25:30.340974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.403468+0000","last_change":"2024-04-02T14:41:21.490632+0000","last_active":"2024-04-02T14:44:00.403468+0000","last_peered":"2024-04-02T14:44:00.403468+0000","last_clean":"2024-04-02T14:44:00.403468+0000","last_became_active":"2024-04-02T14:41:21.490208+0000","last_became_peered":"2024-04-02T14:41:21.490208+0000","last_unstale":"2024-04-02T14:44:00.403468+0000","last_undegraded":"2024-04-02T14:44:00.403468+0000","last_fullsized":"2024-04-02T14:44:00.403468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:18.566261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239701+0000","last_change":"2024-04-02T14:41:22.544872+0000","last_active":"2024-04-02T14:44:00.239701+0000","last_peered":"2024-04-02T14:44:00.239701+0000","last_clean":"2024-04-02T14:44:00.239701+0000","last_became_active":"2024-04-02T14:41:22.544268+0000","last_became_peered":"2024-04-02T14:41:22.544268+0000","last_unstale":"2024-04-02T14:44:00.239701+0000","last_undegraded":"2024-04-02T14:44:00.239701+0000","last_fullsized":"2024-04-02T14:44:00.239701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:54.963084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396149+0000","last_change":"2024-04-02T14:41:21.474411+0000","last_active":"2024-04-02T14:44:00.396149+0000","last_peered":"2024-04-02T14:44:00.396149+0000","last_clean":"2024-04-02T14:44:00.396149+0000","last_became_active":"2024-04-02T14:41:21.473950+0000","last_became_peered":"2024-04-02T14:41:21.473950+0000","last_unstale":"2024-04-02T14:44:00.396149+0000","last_undegraded":"2024-04-02T14:44:00.396149+0000","last_fullsized":"2024-04-02T14:44:00.396149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:46:05.892446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391825+0000","last_change":"2024-04-02T14:40:03.537813+0000","last_active":"2024-04-02T14:44:00.391825+0000","last_peered":"2024-04-02T14:44:00.391825+0000","last_clean":"2024-04-02T14:44:00.391825+0000","last_became_active":"2024-04-02T14:40:03.537223+0000","last_became_peered":"2024-04-02T14:40:03.537223+0000","last_unstale":"2024-04-02T14:44:00.391825+0000","last_undegraded":"2024-04-02T14:44:00.391825+0000","last_fullsized":"2024-04-02T14:44:00.391825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_clean_scrub_stamp":"2024-04-02T14:36:11.612815+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:19:10.346096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396548+0000","last_change":"2024-04-02T14:41:22.537906+0000","last_active":"2024-04-02T14:44:00.396548+0000","last_peered":"2024-04-02T14:44:00.396548+0000","last_clean":"2024-04-02T14:44:00.396548+0000","last_became_active":"2024-04-02T14:41:22.537238+0000","last_became_peered":"2024-04-02T14:41:22.537238+0000","last_unstale":"2024-04-02T14:44:00.396548+0000","last_undegraded":"2024-04-02T14:44:00.396548+0000","last_fullsized":"2024-04-02T14:44:00.396548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:57.089563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391955+0000","last_change":"2024-04-02T14:41:21.466830+0000","last_active":"2024-04-02T14:44:00.391955+0000","last_peered":"2024-04-02T14:44:00.391955+0000","last_clean":"2024-04-02T14:44:00.391955+0000","last_became_active":"2024-04-02T14:41:21.466490+0000","last_became_peered":"2024-04-02T14:41:21.466490+0000","last_unstale":"2024-04-02T14:44:00.391955+0000","last_undegraded":"2024-04-02T14:44:00.391955+0000","last_fullsized":"2024-04-02T14:44:00.391955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:15.597854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238180+0000","last_change":"2024-04-02T14:41:21.481139+0000","last_active":"2024-04-02T14:44:00.238180+0000","last_peered":"2024-04-02T14:44:00.238180+0000","last_clean":"2024-04-02T14:44:00.238180+0000","last_became_active":"2024-04-02T14:41:21.480754+0000","last_became_peered":"2024-04-02T14:41:21.480754+0000","last_unstale":"2024-04-02T14:44:00.238180+0000","last_undegraded":"2024-04-02T14:44:00.238180+0000","last_fullsized":"2024-04-02T14:44:00.238180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:03.716223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397939+0000","last_change":"2024-04-02T14:41:22.548483+0000","last_active":"2024-04-02T14:44:00.397939+0000","last_peered":"2024-04-02T14:44:00.397939+0000","last_clean":"2024-04-02T14:44:00.397939+0000","last_became_active":"2024-04-02T14:41:22.547463+0000","last_became_peered":"2024-04-02T14:41:22.547463+0000","last_unstale":"2024-04-02T14:44:00.397939+0000","last_undegraded":"2024-04-02T14:44:00.397939+0000","last_fullsized":"2024-04-02T14:44:00.397939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:33:58.076844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'49","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.390830+0000","last_change":"2024-04-02T14:41:21.491105+0000","last_active":"2024-04-02T14:44:00.390830+0000","last_peered":"2024-04-02T14:44:00.390830+0000","last_clean":"2024-04-02T14:44:00.390830+0000","last_became_active":"2024-04-02T14:41:21.490635+0000","last_became_peered":"2024-04-02T14:41:21.490635+0000","last_unstale":"2024-04-02T14:44:00.390830+0000","last_undegraded":"2024-04-02T14:44:00.390830+0000","last_fullsized":"2024-04-02T14:44:00.390830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:16.531064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239871+0000","last_change":"2024-04-02T14:41:22.553497+0000","last_active":"2024-04-02T14:44:00.239871+0000","last_peered":"2024-04-02T14:44:00.239871+0000","last_clean":"2024-04-02T14:44:00.239871+0000","last_became_active":"2024-04-02T14:41:22.550474+0000","last_became_peered":"2024-04-02T14:41:22.550474+0000","last_unstale":"2024-04-02T14:44:00.239871+0000","last_undegraded":"2024-04-02T14:44:00.239871+0000","last_fullsized":"2024-04-02T14:44:00.239871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:52.212454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396225+0000","last_change":"2024-04-02T14:41:21.479516+0000","last_active":"2024-04-02T14:44:00.396225+0000","last_peered":"2024-04-02T14:44:00.396225+0000","last_clean":"2024-04-02T14:44:00.396225+0000","last_became_active":"2024-04-02T14:41:21.478841+0000","last_became_peered":"2024-04-02T14:41:21.478841+0000","last_unstale":"2024-04-02T14:44:00.396225+0000","last_undegraded":"2024-04-02T14:44:00.396225+0000","last_fullsized":"2024-04-02T14:44:00.396225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:39.222316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398623+0000","last_change":"2024-04-02T14:41:22.550095+0000","last_active":"2024-04-02T14:44:00.398623+0000","last_peered":"2024-04-02T14:44:00.398623+0000","last_clean":"2024-04-02T14:44:00.398623+0000","last_became_active":"2024-04-02T14:41:22.547438+0000","last_became_peered":"2024-04-02T14:41:22.547438+0000","last_unstale":"2024-04-02T14:44:00.398623+0000","last_undegraded":"2024-04-02T14:44:00.398623+0000","last_fullsized":"2024-04-02T14:44:00.398623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:03:03.668289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362487+0000","last_change":"2024-04-02T14:41:21.482926+0000","last_active":"2024-04-02T14:44:00.362487+0000","last_peered":"2024-04-02T14:44:00.362487+0000","last_clean":"2024-04-02T14:44:00.362487+0000","last_became_active":"2024-04-02T14:41:21.482517+0000","last_became_peered":"2024-04-02T14:41:21.482517+0000","last_unstale":"2024-04-02T14:44:00.362487+0000","last_undegraded":"2024-04-02T14:44:00.362487+0000","last_fullsized":"2024-04-02T14:44:00.362487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:43.052390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.237874+0000","last_change":"2024-04-02T14:41:22.527758+0000","last_active":"2024-04-02T14:44:00.237874+0000","last_peered":"2024-04-02T14:44:00.237874+0000","last_clean":"2024-04-02T14:44:00.237874+0000","last_became_active":"2024-04-02T14:41:22.526954+0000","last_became_peered":"2024-04-02T14:41:22.526954+0000","last_unstale":"2024-04-02T14:44:00.237874+0000","last_undegraded":"2024-04-02T14:44:00.237874+0000","last_fullsized":"2024-04-02T14:44:00.237874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:34.988661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239213+0000","last_change":"2024-04-02T14:41:21.487534+0000","last_active":"2024-04-02T14:44:00.239213+0000","last_peered":"2024-04-02T14:44:00.239213+0000","last_clean":"2024-04-02T14:44:00.239213+0000","last_became_active":"2024-04-02T14:41:21.487168+0000","last_became_peered":"2024-04-02T14:41:21.487168+0000","last_unstale":"2024-04-02T14:44:00.239213+0000","last_undegraded":"2024-04-02T14:44:00.239213+0000","last_fullsized":"2024-04-02T14:44:00.239213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:03:20.636028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396152+0000","last_change":"2024-04-02T14:41:22.539246+0000","last_active":"2024-04-02T14:44:00.396152+0000","last_peered":"2024-04-02T14:44:00.396152+0000","last_clean":"2024-04-02T14:44:00.396152+0000","last_became_active":"2024-04-02T14:41:22.538864+0000","last_became_peered":"2024-04-02T14:41:22.538864+0000","last_unstale":"2024-04-02T14:44:00.396152+0000","last_undegraded":"2024-04-02T14:44:00.396152+0000","last_fullsized":"2024-04-02T14:44:00.396152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:40:13.418752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098701+0000","last_change":"2024-04-02T14:41:21.477043+0000","last_active":"2024-04-02T14:44:00.098701+0000","last_peered":"2024-04-02T14:44:00.098701+0000","last_clean":"2024-04-02T14:44:00.098701+0000","last_became_active":"2024-04-02T14:41:21.476580+0000","last_became_peered":"2024-04-02T14:41:21.476580+0000","last_unstale":"2024-04-02T14:44:00.098701+0000","last_undegraded":"2024-04-02T14:44:00.098701+0000","last_fullsized":"2024-04-02T14:44:00.098701+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:18.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239852+0000","last_change":"2024-04-02T14:41:22.549758+0000","last_active":"2024-04-02T14:44:00.239852+0000","last_peered":"2024-04-02T14:44:00.239852+0000","last_clean":"2024-04-02T14:44:00.239852+0000","last_became_active":"2024-04-02T14:41:22.545582+0000","last_became_peered":"2024-04-02T14:41:22.545582+0000","last_unstale":"2024-04-02T14:44:00.239852+0000","last_undegraded":"2024-04-02T14:44:00.239852+0000","last_fullsized":"2024-04-02T14:44:00.239852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:42.311921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396174+0000","last_change":"2024-04-02T14:42:21.253221+0000","last_active":"2024-04-02T14:44:00.396174+0000","last_peered":"2024-04-02T14:44:00.396174+0000","last_clean":"2024-04-02T14:44:00.396174+0000","last_became_active":"2024-04-02T14:42:21.247890+0000","last_became_peered":"2024-04-02T14:42:21.247890+0000","last_unstale":"2024-04-02T14:44:00.396174+0000","last_undegraded":"2024-04-02T14:44:00.396174+0000","last_fullsized":"2024-04-02T14:44:00.396174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:12.571769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392085+0000","last_change":"2024-04-02T14:41:22.531509+0000","last_active":"2024-04-02T14:44:00.392085+0000","last_peered":"2024-04-02T14:44:00.392085+0000","last_clean":"2024-04-02T14:44:00.392085+0000","last_became_active":"2024-04-02T14:41:22.531206+0000","last_became_peered":"2024-04-02T14:41:22.531206+0000","last_unstale":"2024-04-02T14:44:00.392085+0000","last_undegraded":"2024-04-02T14:44:00.392085+0000","last_fullsized":"2024-04-02T14:44:00.392085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:39.099177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098889+0000","last_change":"2024-04-02T14:41:21.485271+0000","last_active":"2024-04-02T14:44:00.098889+0000","last_peered":"2024-04-02T14:44:00.098889+0000","last_clean":"2024-04-02T14:44:00.098889+0000","last_became_active":"2024-04-02T14:41:21.484895+0000","last_became_peered":"2024-04-02T14:41:21.484895+0000","last_unstale":"2024-04-02T14:44:00.098889+0000","last_undegraded":"2024-04-02T14:44:00.098889+0000","last_fullsized":"2024-04-02T14:44:00.098889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.552092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":153,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392205+0000","last_change":"2024-04-02T14:41:22.524853+0000","last_active":"2024-04-02T14:44:00.392205+0000","last_peered":"2024-04-02T14:44:00.392205+0000","last_clean":"2024-04-02T14:44:00.392205+0000","last_became_active":"2024-04-02T14:41:22.524403+0000","last_became_peered":"2024-04-02T14:41:22.524403+0000","last_unstale":"2024-04-02T14:44:00.392205+0000","last_undegraded":"2024-04-02T14:44:00.392205+0000","last_fullsized":"2024-04-02T14:44:00.392205+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:31.322626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.386875+0000","last_change":"2024-04-02T14:41:21.474340+0000","last_active":"2024-04-02T14:44:00.386875+0000","last_peered":"2024-04-02T14:44:00.386875+0000","last_clean":"2024-04-02T14:44:00.386875+0000","last_became_active":"2024-04-02T14:41:21.473934+0000","last_became_peered":"2024-04-02T14:41:21.473934+0000","last_unstale":"2024-04-02T14:44:00.386875+0000","last_undegraded":"2024-04-02T14:44:00.386875+0000","last_fullsized":"2024-04-02T14:44:00.386875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:28:42.711817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391989+0000","last_change":"2024-04-02T14:41:22.543018+0000","last_active":"2024-04-02T14:44:00.391989+0000","last_peered":"2024-04-02T14:44:00.391989+0000","last_clean":"2024-04-02T14:44:00.391989+0000","last_became_active":"2024-04-02T14:41:22.542621+0000","last_became_peered":"2024-04-02T14:41:22.542621+0000","last_unstale":"2024-04-02T14:44:00.391989+0000","last_undegraded":"2024-04-02T14:44:00.391989+0000","last_fullsized":"2024-04-02T14:44:00.391989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:55:03.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398425+0000","last_change":"2024-04-02T14:41:21.493060+0000","last_active":"2024-04-02T14:44:00.398425+0000","last_peered":"2024-04-02T14:44:00.398425+0000","last_clean":"2024-04-02T14:44:00.398425+0000","last_became_active":"2024-04-02T14:41:21.492627+0000","last_became_peered":"2024-04-02T14:41:21.492627+0000","last_unstale":"2024-04-02T14:44:00.398425+0000","last_undegraded":"2024-04-02T14:44:00.398425+0000","last_fullsized":"2024-04-02T14:44:00.398425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:29:10.443866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392362+0000","last_change":"2024-04-02T14:41:22.527742+0000","last_active":"2024-04-02T14:44:00.392362+0000","last_peered":"2024-04-02T14:44:00.392362+0000","last_clean":"2024-04-02T14:44:00.392362+0000","last_became_active":"2024-04-02T14:41:22.527436+0000","last_became_peered":"2024-04-02T14:41:22.527436+0000","last_unstale":"2024-04-02T14:44:00.392362+0000","last_undegraded":"2024-04-02T14:44:00.392362+0000","last_fullsized":"2024-04-02T14:44:00.392362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:04:24.754907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396235+0000","last_change":"2024-04-02T14:41:22.526235+0000","last_active":"2024-04-02T14:44:00.396235+0000","last_peered":"2024-04-02T14:44:00.396235+0000","last_clean":"2024-04-02T14:44:00.396235+0000","last_became_active":"2024-04-02T14:41:22.525879+0000","last_became_peered":"2024-04-02T14:41:22.525879+0000","last_unstale":"2024-04-02T14:44:00.396235+0000","last_undegraded":"2024-04-02T14:44:00.396235+0000","last_fullsized":"2024-04-02T14:44:00.396235+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:20:54.431223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098551+0000","last_change":"2024-04-02T14:41:22.522498+0000","last_active":"2024-04-02T14:44:00.098551+0000","last_peered":"2024-04-02T14:44:00.098551+0000","last_clean":"2024-04-02T14:44:00.098551+0000","last_became_active":"2024-04-02T14:41:22.522162+0000","last_became_peered":"2024-04-02T14:41:22.522162+0000","last_unstale":"2024-04-02T14:44:00.098551+0000","last_undegraded":"2024-04-02T14:44:00.098551+0000","last_fullsized":"2024-04-02T14:44:00.098551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:42:04.701228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098399+0000","last_change":"2024-04-02T14:41:22.543920+0000","last_active":"2024-04-02T14:44:00.098399+0000","last_peered":"2024-04-02T14:44:00.098399+0000","last_clean":"2024-04-02T14:44:00.098399+0000","last_became_active":"2024-04-02T14:41:22.543552+0000","last_became_peered":"2024-04-02T14:41:22.543552+0000","last_unstale":"2024-04-02T14:44:00.098399+0000","last_undegraded":"2024-04-02T14:44:00.098399+0000","last_fullsized":"2024-04-02T14:44:00.098399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:28.761770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398084+0000","last_change":"2024-04-02T14:41:22.550507+0000","last_active":"2024-04-02T14:44:00.398084+0000","last_peered":"2024-04-02T14:44:00.398084+0000","last_clean":"2024-04-02T14:44:00.398084+0000","last_became_active":"2024-04-02T14:41:22.545824+0000","last_became_peered":"2024-04-02T14:41:22.545824+0000","last_unstale":"2024-04-02T14:44:00.398084+0000","last_undegraded":"2024-04-02T14:44:00.398084+0000","last_fullsized":"2024-04-02T14:44:00.398084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:11.175138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398099+0000","last_change":"2024-04-02T14:41:22.541362+0000","last_active":"2024-04-02T14:44:00.398099+0000","last_peered":"2024-04-02T14:44:00.398099+0000","last_clean":"2024-04-02T14:44:00.398099+0000","last_became_active":"2024-04-02T14:41:22.540927+0000","last_became_peered":"2024-04-02T14:41:22.540927+0000","last_unstale":"2024-04-02T14:44:00.398099+0000","last_undegraded":"2024-04-02T14:44:00.398099+0000","last_fullsized":"2024-04-02T14:44:00.398099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:43:00.746274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396335+0000","last_change":"2024-04-02T14:41:22.527146+0000","last_active":"2024-04-02T14:44:00.396335+0000","last_peered":"2024-04-02T14:44:00.396335+0000","last_clean":"2024-04-02T14:44:00.396335+0000","last_became_active":"2024-04-02T14:41:22.526810+0000","last_became_peered":"2024-04-02T14:41:22.526810+0000","last_unstale":"2024-04-02T14:44:00.396335+0000","last_undegraded":"2024-04-02T14:44:00.396335+0000","last_fullsized":"2024-04-02T14:44:00.396335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:18:46.464701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099056+0000","last_change":"2024-04-02T14:41:22.524458+0000","last_active":"2024-04-02T14:44:00.099056+0000","last_peered":"2024-04-02T14:44:00.099056+0000","last_clean":"2024-04-02T14:44:00.099056+0000","last_became_active":"2024-04-02T14:41:22.524138+0000","last_became_peered":"2024-04-02T14:41:22.524138+0000","last_unstale":"2024-04-02T14:44:00.099056+0000","last_undegraded":"2024-04-02T14:44:00.099056+0000","last_fullsized":"2024-04-02T14:44:00.099056+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:50:04.667964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239756+0000","last_change":"2024-04-02T14:42:21.251102+0000","last_active":"2024-04-02T14:44:00.239756+0000","last_peered":"2024-04-02T14:44:00.239756+0000","last_clean":"2024-04-02T14:44:00.239756+0000","last_became_active":"2024-04-02T14:42:21.249134+0000","last_became_peered":"2024-04-02T14:42:21.249134+0000","last_unstale":"2024-04-02T14:44:00.239756+0000","last_undegraded":"2024-04-02T14:44:00.239756+0000","last_fullsized":"2024-04-02T14:44:00.239756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:05:56.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.249896+0000","last_change":"2024-04-02T14:41:22.542319+0000","last_active":"2024-04-02T14:44:00.249896+0000","last_peered":"2024-04-02T14:44:00.249896+0000","last_clean":"2024-04-02T14:44:00.249896+0000","last_became_active":"2024-04-02T14:41:22.541959+0000","last_became_peered":"2024-04-02T14:41:22.541959+0000","last_unstale":"2024-04-02T14:44:00.249896+0000","last_undegraded":"2024-04-02T14:44:00.249896+0000","last_fullsized":"2024-04-02T14:44:00.249896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:49.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239058+0000","last_change":"2024-04-02T14:41:22.549932+0000","last_active":"2024-04-02T14:44:00.239058+0000","last_peered":"2024-04-02T14:44:00.239058+0000","last_clean":"2024-04-02T14:44:00.239058+0000","last_became_active":"2024-04-02T14:41:22.549231+0000","last_became_peered":"2024-04-02T14:41:22.549231+0000","last_unstale":"2024-04-02T14:44:00.239058+0000","last_undegraded":"2024-04-02T14:44:00.239058+0000","last_fullsized":"2024-04-02T14:44:00.239058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:01.485343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362610+0000","last_change":"2024-04-02T14:41:22.544712+0000","last_active":"2024-04-02T14:44:00.362610+0000","last_peered":"2024-04-02T14:44:00.362610+0000","last_clean":"2024-04-02T14:44:00.362610+0000","last_became_active":"2024-04-02T14:41:22.544043+0000","last_became_peered":"2024-04-02T14:41:22.544043+0000","last_unstale":"2024-04-02T14:44:00.362610+0000","last_undegraded":"2024-04-02T14:44:00.362610+0000","last_fullsized":"2024-04-02T14:44:00.362610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:07.931377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398079+0000","last_change":"2024-04-02T14:41:22.542364+0000","last_active":"2024-04-02T14:44:00.398079+0000","last_peered":"2024-04-02T14:44:00.398079+0000","last_clean":"2024-04-02T14:44:00.398079+0000","last_became_active":"2024-04-02T14:41:22.541929+0000","last_became_peered":"2024-04-02T14:41:22.541929+0000","last_unstale":"2024-04-02T14:44:00.398079+0000","last_undegraded":"2024-04-02T14:44:00.398079+0000","last_fullsized":"2024-04-02T14:44:00.398079+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:45.716132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396257+0000","last_change":"2024-04-02T14:41:22.525295+0000","last_active":"2024-04-02T14:44:00.396257+0000","last_peered":"2024-04-02T14:44:00.396257+0000","last_clean":"2024-04-02T14:44:00.396257+0000","last_became_active":"2024-04-02T14:41:22.524484+0000","last_became_peered":"2024-04-02T14:41:22.524484+0000","last_unstale":"2024-04-02T14:44:00.396257+0000","last_undegraded":"2024-04-02T14:44:00.396257+0000","last_fullsized":"2024-04-02T14:44:00.396257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:01.819082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597589,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1637,"num_read_kb":932,"num_write":367,"num_write_kb":1458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1792767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285612,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":478089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547249,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28988,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716164,"statfs":{"total":95995035648,"available":95965351936,"internally_reserved":0,"allocated":1835008,"data_stored":1264973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710775,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36720,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708432,"statfs":{"total":95995035648,"available":95957434368,"internally_reserved":0,"allocated":1413120,"data_stored":871235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972414,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36716,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708436,"statfs":{"total":95995035648,"available":95957438464,"internally_reserved":0,"allocated":1409024,"data_stored":871613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1613824,"data_stored":1071056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430280,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":468838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":542904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1024000,"data_stored":479075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":542893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117080,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":477561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855445,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1011712,"data_stored":479533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691918,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1630208,"data_stored":1071246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":198373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:44:06.383 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T14:44:06.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:06 smithi204 ceph-mon[35948]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-02T14:44:06.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:06 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2780282288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:44:06.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:06 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/4150771145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:44:06.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.610+0000 7f21c6800640 1 -- 172.21.15.62:0/1813869392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c0075b40 msgr2=0x7f21c0073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:06.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.610+0000 7f21c6800640 1 --2- 172.21.15.62:0/1813869392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c0075b40 0x7f21c0073f90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009860 tx=0x7f21b002bb70 comp rx=0 tx=0).stop 2024-04-02T14:44:06.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.610+0000 7f21c6800640 1 -- 172.21.15.62:0/1813869392 shutdown_connections 2024-04-02T14:44:06.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.610+0000 7f21c6800640 1 --2- 172.21.15.62:0/1813869392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c0109a00 0x7f21c010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.610+0000 7f21c6800640 1 --2- 172.21.15.62:0/1813869392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c0074560 0x7f21c00749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.610+0000 7f21c6800640 1 --2- 172.21.15.62:0/1813869392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c0075b40 0x7f21c0073f90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.610+0000 7f21c6800640 1 -- 172.21.15.62:0/1813869392 >> 172.21.15.62:0/1813869392 conn(0x7f21c00fd0b0 msgr2=0x7f21c00ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:06.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.610+0000 7f21c6800640 1 -- 172.21.15.62:0/1813869392 shutdown_connections 2024-04-02T14:44:06.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.610+0000 7f21c6800640 1 -- 172.21.15.62:0/1813869392 wait complete. 2024-04-02T14:44:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21c6800640 1 Processor -- start 2024-04-02T14:44:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21c6800640 1 -- start start 2024-04-02T14:44:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21c6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c0074560 0x7f21c019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21c6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c0075b40 0x7f21c019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21c6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c0109a00 0x7f21c01a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21c6800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f21c010fae0 con 0x7f21c0074560 2024-04-02T14:44:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21c6800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f21c010f960 con 0x7f21c0075b40 2024-04-02T14:44:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21c6800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f21c010fc60 con 0x7f21c0109a00 2024-04-02T14:44:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21bf400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c0074560 0x7f21c019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21bfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c0109a00 0x7f21c01a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:06.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21bfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c0109a00 0x7f21c01a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39244/0 (socket says 172.21.15.62:39244) 2024-04-02T14:44:06.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.613+0000 7f21bfe00640 1 -- 172.21.15.62:0/3926109261 learned_addr learned my addr 172.21.15.62:0/3926109261 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:06.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21bf400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c0074560 0x7f21c019a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35750/0 (socket says 172.21.15.62:35750) 2024-04-02T14:44:06.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.612+0000 7f21bea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c0075b40 0x7f21c019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.613+0000 7f21bfe00640 1 -- 172.21.15.62:0/3926109261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c0075b40 msgr2=0x7f21c019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.613+0000 7f21bfe00640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c0075b40 0x7f21c019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.613+0000 7f21bfe00640 1 -- 172.21.15.62:0/3926109261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c0074560 msgr2=0x7f21c019a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.613+0000 7f21bfe00640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c0074560 0x7f21c019a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.613+0000 7f21bfe00640 1 -- 172.21.15.62:0/3926109261 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c01a1d70 con 0x7f21c0109a00 2024-04-02T14:44:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.613+0000 7f21bea00640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c0075b40 0x7f21c019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.613+0000 7f21bfe00640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c0109a00 0x7f21c01a1700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f21b800eb10 tx=0x7f21b800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.614+0000 7f21b7e00640 1 -- 172.21.15.62:0/3926109261 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b800ce20 con 0x7f21c0109a00 2024-04-02T14:44:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.614+0000 7f21b7e00640 1 -- 172.21.15.62:0/3926109261 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f21b8004b20 con 0x7f21c0109a00 2024-04-02T14:44:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.614+0000 7f21c6800640 1 -- 172.21.15.62:0/3926109261 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c01a2060 con 0x7f21c0109a00 2024-04-02T14:44:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.614+0000 7f21b7e00640 1 -- 172.21.15.62:0/3926109261 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8010760 con 0x7f21c0109a00 2024-04-02T14:44:06.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.614+0000 7f21c6800640 1 -- 172.21.15.62:0/3926109261 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c01a2560 con 0x7f21c0109a00 2024-04-02T14:44:06.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.615+0000 7f21c6800640 1 -- 172.21.15.62:0/3926109261 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c0108730 con 0x7f21c0109a00 2024-04-02T14:44:06.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.617+0000 7f21b7e00640 1 -- 172.21.15.62:0/3926109261 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f21b8005020 con 0x7f21c0109a00 2024-04-02T14:44:06.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.618+0000 7f21b7e00640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21a40a3e40 0x7f21a40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:06.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.618+0000 7f21b7e00640 1 -- 172.21.15.62:0/3926109261 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f21b80d0fe0 con 0x7f21c0109a00 2024-04-02T14:44:06.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.619+0000 7f21bf400640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21a40a3e40 0x7f21a40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:06.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.619+0000 7f21bf400640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21a40a3e40 0x7f21a40a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009830 tx=0x7f21b0034b50 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:06.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.622+0000 7f21b7e00640 1 -- 172.21.15.62:0/3926109261 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f21b809d480 con 0x7f21c0109a00 2024-04-02T14:44:06.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.830+0000 7f21c6800640 1 -- 172.21.15.62:0/3926109261 --> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21c0061f40 con 0x7f21a40a3e40 2024-04-02T14:44:06.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.839+0000 7f21b7e00640 1 -- 172.21.15.62:0/3926109261 <== mgr.25273 v2:172.21.15.62:6800/110874546 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257789 (secure 0 0 0) 0x7f21c0061f40 con 0x7f21a40a3e40 2024-04-02T14:44:06.839 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:44:06.841 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-02T14:44:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.844+0000 7f21c6800640 1 -- 172.21.15.62:0/3926109261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21a40a3e40 msgr2=0x7f21a40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.844+0000 7f21c6800640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21a40a3e40 0x7f21a40a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009830 tx=0x7f21b0034b50 comp rx=0 tx=0).stop 2024-04-02T14:44:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.844+0000 7f21c6800640 1 -- 172.21.15.62:0/3926109261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c0109a00 msgr2=0x7f21c01a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.844+0000 7f21c6800640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c0109a00 0x7f21c01a1700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f21b800eb10 tx=0x7f21b800ef10 comp rx=0 tx=0).stop 2024-04-02T14:44:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.844+0000 7f21c6800640 1 -- 172.21.15.62:0/3926109261 shutdown_connections 2024-04-02T14:44:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.844+0000 7f21c6800640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f21a40a3e40 0x7f21a40a6300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.844+0000 7f21c6800640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c0109a00 0x7f21c01a1700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.844+0000 7f21c6800640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c0075b40 0x7f21c019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.844+0000 7f21c6800640 1 --2- 172.21.15.62:0/3926109261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c0074560 0x7f21c019a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.844+0000 7f21c6800640 1 -- 172.21.15.62:0/3926109261 >> 172.21.15.62:0/3926109261 conn(0x7f21c00fd0b0 msgr2=0x7f21c0105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.845+0000 7f21c6800640 1 -- 172.21.15.62:0/3926109261 shutdown_connections 2024-04-02T14:44:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:06.845+0000 7f21c6800640 1 -- 172.21.15.62:0/3926109261 wait complete. 2024-04-02T14:44:06.863 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-02T14:44:06.118787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188191,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1773,"num_read_kb":1051,"num_write":550,"num_write_kb":4398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181749,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":934,"ondisk_log_size":934,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":405704,"kb_used_data":14736,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124536120,"statfs":{"total":1151940427776,"available":1151524986880,"internally_reserved":0,"allocated":15089664,"data_stored":8619016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.975999"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396472+0000","last_change":"2024-04-02T14:41:22.546468+0000","last_active":"2024-04-02T14:44:00.396472+0000","last_peered":"2024-04-02T14:44:00.396472+0000","last_clean":"2024-04-02T14:44:00.396472+0000","last_became_active":"2024-04-02T14:41:22.545991+0000","last_became_peered":"2024-04-02T14:41:22.545991+0000","last_unstale":"2024-04-02T14:44:00.396472+0000","last_undegraded":"2024-04-02T14:44:00.396472+0000","last_fullsized":"2024-04-02T14:44:00.396472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:34:24.520404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398615+0000","last_change":"2024-04-02T14:41:22.547738+0000","last_active":"2024-04-02T14:44:00.398615+0000","last_peered":"2024-04-02T14:44:00.398615+0000","last_clean":"2024-04-02T14:44:00.398615+0000","last_became_active":"2024-04-02T14:41:22.542372+0000","last_became_peered":"2024-04-02T14:41:22.542372+0000","last_unstale":"2024-04-02T14:44:00.398615+0000","last_undegraded":"2024-04-02T14:44:00.398615+0000","last_fullsized":"2024-04-02T14:44:00.398615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:07:26.951136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397831+0000","last_change":"2024-04-02T14:41:22.540173+0000","last_active":"2024-04-02T14:44:00.397831+0000","last_peered":"2024-04-02T14:44:00.397831+0000","last_clean":"2024-04-02T14:44:00.397831+0000","last_became_active":"2024-04-02T14:41:22.536279+0000","last_became_peered":"2024-04-02T14:41:22.536279+0000","last_unstale":"2024-04-02T14:44:00.397831+0000","last_undegraded":"2024-04-02T14:44:00.397831+0000","last_fullsized":"2024-04-02T14:44:00.397831+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:43:40.826934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099445+0000","last_change":"2024-04-02T14:42:21.251675+0000","last_active":"2024-04-02T14:44:00.099445+0000","last_peered":"2024-04-02T14:44:00.099445+0000","last_clean":"2024-04-02T14:44:00.099445+0000","last_became_active":"2024-04-02T14:42:21.250446+0000","last_became_peered":"2024-04-02T14:42:21.250446+0000","last_unstale":"2024-04-02T14:44:00.099445+0000","last_undegraded":"2024-04-02T14:44:00.099445+0000","last_fullsized":"2024-04-02T14:44:00.099445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:11.005276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238025+0000","last_change":"2024-04-02T14:41:22.554301+0000","last_active":"2024-04-02T14:44:00.238025+0000","last_peered":"2024-04-02T14:44:00.238025+0000","last_clean":"2024-04-02T14:44:00.238025+0000","last_became_active":"2024-04-02T14:41:22.554028+0000","last_became_peered":"2024-04-02T14:41:22.554028+0000","last_unstale":"2024-04-02T14:44:00.238025+0000","last_undegraded":"2024-04-02T14:44:00.238025+0000","last_fullsized":"2024-04-02T14:44:00.238025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:51:59.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253420+0000","last_change":"2024-04-02T14:41:22.543934+0000","last_active":"2024-04-02T14:44:00.253420+0000","last_peered":"2024-04-02T14:44:00.253420+0000","last_clean":"2024-04-02T14:44:00.253420+0000","last_became_active":"2024-04-02T14:41:22.543392+0000","last_became_peered":"2024-04-02T14:41:22.543392+0000","last_unstale":"2024-04-02T14:44:00.253420+0000","last_undegraded":"2024-04-02T14:44:00.253420+0000","last_fullsized":"2024-04-02T14:44:00.253420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:28:30.234356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391852+0000","last_change":"2024-04-02T14:41:22.547460+0000","last_active":"2024-04-02T14:44:00.391852+0000","last_peered":"2024-04-02T14:44:00.391852+0000","last_clean":"2024-04-02T14:44:00.391852+0000","last_became_active":"2024-04-02T14:41:22.547123+0000","last_became_peered":"2024-04-02T14:41:22.547123+0000","last_unstale":"2024-04-02T14:44:00.391852+0000","last_undegraded":"2024-04-02T14:44:00.391852+0000","last_fullsized":"2024-04-02T14:44:00.391852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:24:34.302530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239097+0000","last_change":"2024-04-02T14:41:22.551407+0000","last_active":"2024-04-02T14:44:00.239097+0000","last_peered":"2024-04-02T14:44:00.239097+0000","last_clean":"2024-04-02T14:44:00.239097+0000","last_became_active":"2024-04-02T14:41:22.538304+0000","last_became_peered":"2024-04-02T14:41:22.538304+0000","last_unstale":"2024-04-02T14:44:00.239097+0000","last_undegraded":"2024-04-02T14:44:00.239097+0000","last_fullsized":"2024-04-02T14:44:00.239097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:27.933380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362921+0000","last_change":"2024-04-02T14:41:22.523931+0000","last_active":"2024-04-02T14:44:00.362921+0000","last_peered":"2024-04-02T14:44:00.362921+0000","last_clean":"2024-04-02T14:44:00.362921+0000","last_became_active":"2024-04-02T14:41:22.523622+0000","last_became_peered":"2024-04-02T14:41:22.523622+0000","last_unstale":"2024-04-02T14:44:00.362921+0000","last_undegraded":"2024-04-02T14:44:00.362921+0000","last_fullsized":"2024-04-02T14:44:00.362921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:17.808290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253719+0000","last_change":"2024-04-02T14:41:22.525791+0000","last_active":"2024-04-02T14:44:00.253719+0000","last_peered":"2024-04-02T14:44:00.253719+0000","last_clean":"2024-04-02T14:44:00.253719+0000","last_became_active":"2024-04-02T14:41:22.525434+0000","last_became_peered":"2024-04-02T14:41:22.525434+0000","last_unstale":"2024-04-02T14:44:00.253719+0000","last_undegraded":"2024-04-02T14:44:00.253719+0000","last_fullsized":"2024-04-02T14:44:00.253719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:59.905337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392238+0000","last_change":"2024-04-02T14:41:22.527240+0000","last_active":"2024-04-02T14:44:00.392238+0000","last_peered":"2024-04-02T14:44:00.392238+0000","last_clean":"2024-04-02T14:44:00.392238+0000","last_became_active":"2024-04-02T14:41:22.526412+0000","last_became_peered":"2024-04-02T14:41:22.526412+0000","last_unstale":"2024-04-02T14:44:00.392238+0000","last_undegraded":"2024-04-02T14:44:00.392238+0000","last_fullsized":"2024-04-02T14:44:00.392238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:47.397012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396098+0000","last_change":"2024-04-02T14:41:22.527281+0000","last_active":"2024-04-02T14:44:00.396098+0000","last_peered":"2024-04-02T14:44:00.396098+0000","last_clean":"2024-04-02T14:44:00.396098+0000","last_became_active":"2024-04-02T14:41:22.526946+0000","last_became_peered":"2024-04-02T14:41:22.526946+0000","last_unstale":"2024-04-02T14:44:00.396098+0000","last_undegraded":"2024-04-02T14:44:00.396098+0000","last_fullsized":"2024-04-02T14:44:00.396098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:42.798282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398248+0000","last_change":"2024-04-02T14:41:22.534079+0000","last_active":"2024-04-02T14:44:00.398248+0000","last_peered":"2024-04-02T14:44:00.398248+0000","last_clean":"2024-04-02T14:44:00.398248+0000","last_became_active":"2024-04-02T14:41:22.532233+0000","last_became_peered":"2024-04-02T14:41:22.532233+0000","last_unstale":"2024-04-02T14:44:00.398248+0000","last_undegraded":"2024-04-02T14:44:00.398248+0000","last_fullsized":"2024-04-02T14:44:00.398248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:27:47.122933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397389+0000","last_change":"2024-04-02T14:41:22.544936+0000","last_active":"2024-04-02T14:44:00.397389+0000","last_peered":"2024-04-02T14:44:00.397389+0000","last_clean":"2024-04-02T14:44:00.397389+0000","last_became_active":"2024-04-02T14:41:22.541805+0000","last_became_peered":"2024-04-02T14:41:22.541805+0000","last_unstale":"2024-04-02T14:44:00.397389+0000","last_undegraded":"2024-04-02T14:44:00.397389+0000","last_fullsized":"2024-04-02T14:44:00.397389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:23.992807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397232+0000","last_change":"2024-04-02T14:41:22.534860+0000","last_active":"2024-04-02T14:44:00.397232+0000","last_peered":"2024-04-02T14:44:00.397232+0000","last_clean":"2024-04-02T14:44:00.397232+0000","last_became_active":"2024-04-02T14:41:22.534147+0000","last_became_peered":"2024-04-02T14:41:22.534147+0000","last_unstale":"2024-04-02T14:44:00.397232+0000","last_undegraded":"2024-04-02T14:44:00.397232+0000","last_fullsized":"2024-04-02T14:44:00.397232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:25.249364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362410+0000","last_change":"2024-04-02T14:41:22.537654+0000","last_active":"2024-04-02T14:44:00.362410+0000","last_peered":"2024-04-02T14:44:00.362410+0000","last_clean":"2024-04-02T14:44:00.362410+0000","last_became_active":"2024-04-02T14:41:22.537304+0000","last_became_peered":"2024-04-02T14:41:22.537304+0000","last_unstale":"2024-04-02T14:44:00.362410+0000","last_undegraded":"2024-04-02T14:44:00.362410+0000","last_fullsized":"2024-04-02T14:44:00.362410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:25.599724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396524+0000","last_change":"2024-04-02T14:41:22.546775+0000","last_active":"2024-04-02T14:44:00.396524+0000","last_peered":"2024-04-02T14:44:00.396524+0000","last_clean":"2024-04-02T14:44:00.396524+0000","last_became_active":"2024-04-02T14:41:22.546100+0000","last_became_peered":"2024-04-02T14:41:22.546100+0000","last_unstale":"2024-04-02T14:44:00.396524+0000","last_undegraded":"2024-04-02T14:44:00.396524+0000","last_fullsized":"2024-04-02T14:44:00.396524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:42:02.559335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099394+0000","last_change":"2024-04-02T14:41:22.543988+0000","last_active":"2024-04-02T14:44:00.099394+0000","last_peered":"2024-04-02T14:44:00.099394+0000","last_clean":"2024-04-02T14:44:00.099394+0000","last_became_active":"2024-04-02T14:41:22.543036+0000","last_became_peered":"2024-04-02T14:41:22.543036+0000","last_unstale":"2024-04-02T14:44:00.099394+0000","last_undegraded":"2024-04-02T14:44:00.099394+0000","last_fullsized":"2024-04-02T14:44:00.099394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:29.470929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098748+0000","last_change":"2024-04-02T14:41:22.543954+0000","last_active":"2024-04-02T14:44:00.098748+0000","last_peered":"2024-04-02T14:44:00.098748+0000","last_clean":"2024-04-02T14:44:00.098748+0000","last_became_active":"2024-04-02T14:41:22.543624+0000","last_became_peered":"2024-04-02T14:41:22.543624+0000","last_unstale":"2024-04-02T14:44:00.098748+0000","last_undegraded":"2024-04-02T14:44:00.098748+0000","last_fullsized":"2024-04-02T14:44:00.098748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:46.933877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":188,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.390351+0000","last_change":"2024-04-02T14:41:21.477341+0000","last_active":"2024-04-02T14:44:00.390351+0000","last_peered":"2024-04-02T14:44:00.390351+0000","last_clean":"2024-04-02T14:44:00.390351+0000","last_became_active":"2024-04-02T14:41:21.476708+0000","last_became_peered":"2024-04-02T14:41:21.476708+0000","last_unstale":"2024-04-02T14:44:00.390351+0000","last_undegraded":"2024-04-02T14:44:00.390351+0000","last_fullsized":"2024-04-02T14:44:00.390351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:34.867570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.241083+0000","last_change":"2024-04-02T14:41:22.551293+0000","last_active":"2024-04-02T14:44:00.241083+0000","last_peered":"2024-04-02T14:44:00.241083+0000","last_clean":"2024-04-02T14:44:00.241083+0000","last_became_active":"2024-04-02T14:41:22.547819+0000","last_became_peered":"2024-04-02T14:41:22.547819+0000","last_unstale":"2024-04-02T14:44:00.241083+0000","last_undegraded":"2024-04-02T14:44:00.241083+0000","last_fullsized":"2024-04-02T14:44:00.241083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:40.393005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.241214+0000","last_change":"2024-04-02T14:41:21.487096+0000","last_active":"2024-04-02T14:44:00.241214+0000","last_peered":"2024-04-02T14:44:00.241214+0000","last_clean":"2024-04-02T14:44:00.241214+0000","last_became_active":"2024-04-02T14:41:21.485883+0000","last_became_peered":"2024-04-02T14:41:21.485883+0000","last_unstale":"2024-04-02T14:44:00.241214+0000","last_undegraded":"2024-04-02T14:44:00.241214+0000","last_fullsized":"2024-04-02T14:44:00.241214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:24.017671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398946+0000","last_change":"2024-04-02T14:41:22.549234+0000","last_active":"2024-04-02T14:44:00.398946+0000","last_peered":"2024-04-02T14:44:00.398946+0000","last_clean":"2024-04-02T14:44:00.398946+0000","last_became_active":"2024-04-02T14:41:22.548855+0000","last_became_peered":"2024-04-02T14:41:22.548855+0000","last_unstale":"2024-04-02T14:44:00.398946+0000","last_undegraded":"2024-04-02T14:44:00.398946+0000","last_fullsized":"2024-04-02T14:44:00.398946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:05.047383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238406+0000","last_change":"2024-04-02T14:41:21.471673+0000","last_active":"2024-04-02T14:44:00.238406+0000","last_peered":"2024-04-02T14:44:00.238406+0000","last_clean":"2024-04-02T14:44:00.238406+0000","last_became_active":"2024-04-02T14:41:21.471292+0000","last_became_peered":"2024-04-02T14:41:21.471292+0000","last_unstale":"2024-04-02T14:44:00.238406+0000","last_undegraded":"2024-04-02T14:44:00.238406+0000","last_fullsized":"2024-04-02T14:44:00.238406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:42:55.665938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362654+0000","last_change":"2024-04-02T14:41:22.537665+0000","last_active":"2024-04-02T14:44:00.362654+0000","last_peered":"2024-04-02T14:44:00.362654+0000","last_clean":"2024-04-02T14:44:00.362654+0000","last_became_active":"2024-04-02T14:41:22.537274+0000","last_became_peered":"2024-04-02T14:41:22.537274+0000","last_unstale":"2024-04-02T14:44:00.362654+0000","last_undegraded":"2024-04-02T14:44:00.362654+0000","last_fullsized":"2024-04-02T14:44:00.362654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:14:39.200735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238373+0000","last_change":"2024-04-02T14:41:21.466547+0000","last_active":"2024-04-02T14:44:00.238373+0000","last_peered":"2024-04-02T14:44:00.238373+0000","last_clean":"2024-04-02T14:44:00.238373+0000","last_became_active":"2024-04-02T14:41:21.466063+0000","last_became_peered":"2024-04-02T14:41:21.466063+0000","last_unstale":"2024-04-02T14:44:00.238373+0000","last_undegraded":"2024-04-02T14:44:00.238373+0000","last_fullsized":"2024-04-02T14:44:00.238373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:39.201543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250276+0000","last_change":"2024-04-02T14:41:22.523984+0000","last_active":"2024-04-02T14:44:00.250276+0000","last_peered":"2024-04-02T14:44:00.250276+0000","last_clean":"2024-04-02T14:44:00.250276+0000","last_became_active":"2024-04-02T14:41:22.523524+0000","last_became_peered":"2024-04-02T14:41:22.523524+0000","last_unstale":"2024-04-02T14:44:00.250276+0000","last_undegraded":"2024-04-02T14:44:00.250276+0000","last_fullsized":"2024-04-02T14:44:00.250276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:31.542588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":196,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.440836+0000","last_change":"2024-04-02T14:42:21.255204+0000","last_active":"2024-04-02T14:44:00.440836+0000","last_peered":"2024-04-02T14:44:00.440836+0000","last_clean":"2024-04-02T14:44:00.440836+0000","last_became_active":"2024-04-02T14:42:21.248702+0000","last_became_peered":"2024-04-02T14:42:21.248702+0000","last_unstale":"2024-04-02T14:44:00.440836+0000","last_undegraded":"2024-04-02T14:44:00.440836+0000","last_fullsized":"2024-04-02T14:44:00.440836+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.723036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":151,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238112+0000","last_change":"2024-04-02T14:41:22.544107+0000","last_active":"2024-04-02T14:44:00.238112+0000","last_peered":"2024-04-02T14:44:00.238112+0000","last_clean":"2024-04-02T14:44:00.238112+0000","last_became_active":"2024-04-02T14:41:22.543739+0000","last_became_peered":"2024-04-02T14:41:22.543739+0000","last_unstale":"2024-04-02T14:44:00.238112+0000","last_undegraded":"2024-04-02T14:44:00.238112+0000","last_fullsized":"2024-04-02T14:44:00.238112+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:49:08.921668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253407+0000","last_change":"2024-04-02T14:41:21.479714+0000","last_active":"2024-04-02T14:44:00.253407+0000","last_peered":"2024-04-02T14:44:00.253407+0000","last_clean":"2024-04-02T14:44:00.253407+0000","last_became_active":"2024-04-02T14:41:21.479329+0000","last_became_peered":"2024-04-02T14:41:21.479329+0000","last_unstale":"2024-04-02T14:44:00.253407+0000","last_undegraded":"2024-04-02T14:44:00.253407+0000","last_fullsized":"2024-04-02T14:44:00.253407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:09.021017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099863+0000","last_change":"2024-04-02T14:42:21.254332+0000","last_active":"2024-04-02T14:44:00.099863+0000","last_peered":"2024-04-02T14:44:00.099863+0000","last_clean":"2024-04-02T14:44:00.099863+0000","last_became_active":"2024-04-02T14:42:21.251266+0000","last_became_peered":"2024-04-02T14:42:21.251266+0000","last_unstale":"2024-04-02T14:44:00.099863+0000","last_undegraded":"2024-04-02T14:44:00.099863+0000","last_fullsized":"2024-04-02T14:44:00.099863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:51:18.343098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396860+0000","last_change":"2024-04-02T14:41:21.484054+0000","last_active":"2024-04-02T14:44:00.396860+0000","last_peered":"2024-04-02T14:44:00.396860+0000","last_clean":"2024-04-02T14:44:00.396860+0000","last_became_active":"2024-04-02T14:41:21.483688+0000","last_became_peered":"2024-04-02T14:41:21.483688+0000","last_unstale":"2024-04-02T14:44:00.396860+0000","last_undegraded":"2024-04-02T14:44:00.396860+0000","last_fullsized":"2024-04-02T14:44:00.396860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:24.114180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238925+0000","last_change":"2024-04-02T14:41:22.553274+0000","last_active":"2024-04-02T14:44:00.238925+0000","last_peered":"2024-04-02T14:44:00.238925+0000","last_clean":"2024-04-02T14:44:00.238925+0000","last_became_active":"2024-04-02T14:41:22.545412+0000","last_became_peered":"2024-04-02T14:41:22.545412+0000","last_unstale":"2024-04-02T14:44:00.238925+0000","last_undegraded":"2024-04-02T14:44:00.238925+0000","last_fullsized":"2024-04-02T14:44:00.238925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:21.917119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.249831+0000","last_change":"2024-04-02T14:41:21.468582+0000","last_active":"2024-04-02T14:44:00.249831+0000","last_peered":"2024-04-02T14:44:00.249831+0000","last_clean":"2024-04-02T14:44:00.249831+0000","last_became_active":"2024-04-02T14:41:21.468081+0000","last_became_peered":"2024-04-02T14:41:21.468081+0000","last_unstale":"2024-04-02T14:44:00.249831+0000","last_undegraded":"2024-04-02T14:44:00.249831+0000","last_fullsized":"2024-04-02T14:44:00.249831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:19.593081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099196+0000","last_change":"2024-04-02T14:41:22.555627+0000","last_active":"2024-04-02T14:44:00.099196+0000","last_peered":"2024-04-02T14:44:00.099196+0000","last_clean":"2024-04-02T14:44:00.099196+0000","last_became_active":"2024-04-02T14:41:22.555171+0000","last_became_peered":"2024-04-02T14:41:22.555171+0000","last_unstale":"2024-04-02T14:44:00.099196+0000","last_undegraded":"2024-04-02T14:44:00.099196+0000","last_fullsized":"2024-04-02T14:44:00.099196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:21:28.161869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":330,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.403886+0000","last_change":"2024-04-02T14:41:21.475004+0000","last_active":"2024-04-02T14:44:00.403886+0000","last_peered":"2024-04-02T14:44:00.403886+0000","last_clean":"2024-04-02T14:44:00.403886+0000","last_became_active":"2024-04-02T14:41:21.474621+0000","last_became_peered":"2024-04-02T14:41:21.474621+0000","last_unstale":"2024-04-02T14:44:00.403886+0000","last_undegraded":"2024-04-02T14:44:00.403886+0000","last_fullsized":"2024-04-02T14:44:00.403886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:38.779276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":302,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362877+0000","last_change":"2024-04-02T14:41:22.543100+0000","last_active":"2024-04-02T14:44:00.362877+0000","last_peered":"2024-04-02T14:44:00.362877+0000","last_clean":"2024-04-02T14:44:00.362877+0000","last_became_active":"2024-04-02T14:41:22.542697+0000","last_became_peered":"2024-04-02T14:41:22.542697+0000","last_unstale":"2024-04-02T14:44:00.362877+0000","last_undegraded":"2024-04-02T14:44:00.362877+0000","last_fullsized":"2024-04-02T14:44:00.362877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:50:52.412879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392108+0000","last_change":"2024-04-02T14:41:21.468626+0000","last_active":"2024-04-02T14:44:00.392108+0000","last_peered":"2024-04-02T14:44:00.392108+0000","last_clean":"2024-04-02T14:44:00.392108+0000","last_became_active":"2024-04-02T14:41:21.468041+0000","last_became_peered":"2024-04-02T14:41:21.468041+0000","last_unstale":"2024-04-02T14:44:00.392108+0000","last_undegraded":"2024-04-02T14:44:00.392108+0000","last_fullsized":"2024-04-02T14:44:00.392108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:55.064033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398874+0000","last_change":"2024-04-02T14:41:22.539584+0000","last_active":"2024-04-02T14:44:00.398874+0000","last_peered":"2024-04-02T14:44:00.398874+0000","last_clean":"2024-04-02T14:44:00.398874+0000","last_became_active":"2024-04-02T14:41:22.539218+0000","last_became_peered":"2024-04-02T14:41:22.539218+0000","last_unstale":"2024-04-02T14:44:00.398874+0000","last_undegraded":"2024-04-02T14:44:00.398874+0000","last_fullsized":"2024-04-02T14:44:00.398874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:47:56.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250358+0000","last_change":"2024-04-02T14:41:21.481208+0000","last_active":"2024-04-02T14:44:00.250358+0000","last_peered":"2024-04-02T14:44:00.250358+0000","last_clean":"2024-04-02T14:44:00.250358+0000","last_became_active":"2024-04-02T14:41:21.480782+0000","last_became_peered":"2024-04-02T14:41:21.480782+0000","last_unstale":"2024-04-02T14:44:00.250358+0000","last_undegraded":"2024-04-02T14:44:00.250358+0000","last_fullsized":"2024-04-02T14:44:00.250358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:12.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362685+0000","last_change":"2024-04-02T14:41:22.517397+0000","last_active":"2024-04-02T14:44:00.362685+0000","last_peered":"2024-04-02T14:44:00.362685+0000","last_clean":"2024-04-02T14:44:00.362685+0000","last_became_active":"2024-04-02T14:41:22.517027+0000","last_became_peered":"2024-04-02T14:41:22.517027+0000","last_unstale":"2024-04-02T14:44:00.362685+0000","last_undegraded":"2024-04-02T14:44:00.362685+0000","last_fullsized":"2024-04-02T14:44:00.362685+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:53.129949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":368,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391430+0000","last_change":"2024-04-02T14:41:21.469643+0000","last_active":"2024-04-02T14:44:00.391430+0000","last_peered":"2024-04-02T14:44:00.391430+0000","last_clean":"2024-04-02T14:44:00.391430+0000","last_became_active":"2024-04-02T14:41:21.468730+0000","last_became_peered":"2024-04-02T14:41:21.468730+0000","last_unstale":"2024-04-02T14:44:00.391430+0000","last_undegraded":"2024-04-02T14:44:00.391430+0000","last_fullsized":"2024-04-02T14:44:00.391430+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:10.983551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64646,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":311,"num_read_kb":220,"num_write":37,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238145+0000","last_change":"2024-04-02T14:41:22.525052+0000","last_active":"2024-04-02T14:44:00.238145+0000","last_peered":"2024-04-02T14:44:00.238145+0000","last_clean":"2024-04-02T14:44:00.238145+0000","last_became_active":"2024-04-02T14:41:22.524691+0000","last_became_peered":"2024-04-02T14:41:22.524691+0000","last_unstale":"2024-04-02T14:44:00.238145+0000","last_undegraded":"2024-04-02T14:44:00.238145+0000","last_fullsized":"2024-04-02T14:44:00.238145+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:16.386020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098518+0000","last_change":"2024-04-02T14:41:21.476988+0000","last_active":"2024-04-02T14:44:00.098518+0000","last_peered":"2024-04-02T14:44:00.098518+0000","last_clean":"2024-04-02T14:44:00.098518+0000","last_became_active":"2024-04-02T14:41:21.476549+0000","last_became_peered":"2024-04-02T14:41:21.476549+0000","last_unstale":"2024-04-02T14:44:00.098518+0000","last_undegraded":"2024-04-02T14:44:00.098518+0000","last_fullsized":"2024-04-02T14:44:00.098518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:47.498626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397582+0000","last_change":"2024-04-02T14:41:22.544746+0000","last_active":"2024-04-02T14:44:00.397582+0000","last_peered":"2024-04-02T14:44:00.397582+0000","last_clean":"2024-04-02T14:44:00.397582+0000","last_became_active":"2024-04-02T14:41:22.544334+0000","last_became_peered":"2024-04-02T14:41:22.544334+0000","last_unstale":"2024-04-02T14:44:00.397582+0000","last_undegraded":"2024-04-02T14:44:00.397582+0000","last_fullsized":"2024-04-02T14:44:00.397582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:45:20.094455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250015+0000","last_change":"2024-04-02T14:41:21.472458+0000","last_active":"2024-04-02T14:44:00.250015+0000","last_peered":"2024-04-02T14:44:00.250015+0000","last_clean":"2024-04-02T14:44:00.250015+0000","last_became_active":"2024-04-02T14:41:21.472105+0000","last_became_peered":"2024-04-02T14:41:21.472105+0000","last_unstale":"2024-04-02T14:44:00.250015+0000","last_undegraded":"2024-04-02T14:44:00.250015+0000","last_fullsized":"2024-04-02T14:44:00.250015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:44.075807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392138+0000","last_change":"2024-04-02T14:41:22.537492+0000","last_active":"2024-04-02T14:44:00.392138+0000","last_peered":"2024-04-02T14:44:00.392138+0000","last_clean":"2024-04-02T14:44:00.392138+0000","last_became_active":"2024-04-02T14:41:22.537067+0000","last_became_peered":"2024-04-02T14:41:22.537067+0000","last_unstale":"2024-04-02T14:44:00.392138+0000","last_undegraded":"2024-04-02T14:44:00.392138+0000","last_fullsized":"2024-04-02T14:44:00.392138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:08.704340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":181,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.403962+0000","last_change":"2024-04-02T14:41:21.466017+0000","last_active":"2024-04-02T14:44:00.403962+0000","last_peered":"2024-04-02T14:44:00.403962+0000","last_clean":"2024-04-02T14:44:00.403962+0000","last_became_active":"2024-04-02T14:41:21.465600+0000","last_became_peered":"2024-04-02T14:41:21.465600+0000","last_unstale":"2024-04-02T14:44:00.403962+0000","last_undegraded":"2024-04-02T14:44:00.403962+0000","last_fullsized":"2024-04-02T14:44:00.403962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:52:08.131125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391756+0000","last_change":"2024-04-02T14:41:22.526628+0000","last_active":"2024-04-02T14:44:00.391756+0000","last_peered":"2024-04-02T14:44:00.391756+0000","last_clean":"2024-04-02T14:44:00.391756+0000","last_became_active":"2024-04-02T14:41:22.526283+0000","last_became_peered":"2024-04-02T14:41:22.526283+0000","last_unstale":"2024-04-02T14:44:00.391756+0000","last_undegraded":"2024-04-02T14:44:00.391756+0000","last_fullsized":"2024-04-02T14:44:00.391756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:41.812330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.237864+0000","last_change":"2024-04-02T14:41:21.475853+0000","last_active":"2024-04-02T14:44:00.237864+0000","last_peered":"2024-04-02T14:44:00.237864+0000","last_clean":"2024-04-02T14:44:00.237864+0000","last_became_active":"2024-04-02T14:41:21.475555+0000","last_became_peered":"2024-04-02T14:41:21.475555+0000","last_unstale":"2024-04-02T14:44:00.237864+0000","last_undegraded":"2024-04-02T14:44:00.237864+0000","last_fullsized":"2024-04-02T14:44:00.237864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:49.170906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.249931+0000","last_change":"2024-04-02T14:41:22.523848+0000","last_active":"2024-04-02T14:44:00.249931+0000","last_peered":"2024-04-02T14:44:00.249931+0000","last_clean":"2024-04-02T14:44:00.249931+0000","last_became_active":"2024-04-02T14:41:22.523476+0000","last_became_peered":"2024-04-02T14:41:22.523476+0000","last_unstale":"2024-04-02T14:44:00.249931+0000","last_undegraded":"2024-04-02T14:44:00.249931+0000","last_fullsized":"2024-04-02T14:44:00.249931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:47:46.933264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238399+0000","last_change":"2024-04-02T14:41:21.468556+0000","last_active":"2024-04-02T14:44:00.238399+0000","last_peered":"2024-04-02T14:44:00.238399+0000","last_clean":"2024-04-02T14:44:00.238399+0000","last_became_active":"2024-04-02T14:41:21.468215+0000","last_became_peered":"2024-04-02T14:41:21.468215+0000","last_unstale":"2024-04-02T14:44:00.238399+0000","last_undegraded":"2024-04-02T14:44:00.238399+0000","last_fullsized":"2024-04-02T14:44:00.238399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:04.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396416+0000","last_change":"2024-04-02T14:41:22.524228+0000","last_active":"2024-04-02T14:44:00.396416+0000","last_peered":"2024-04-02T14:44:00.396416+0000","last_clean":"2024-04-02T14:44:00.396416+0000","last_became_active":"2024-04-02T14:41:22.523839+0000","last_became_peered":"2024-04-02T14:41:22.523839+0000","last_unstale":"2024-04-02T14:44:00.396416+0000","last_undegraded":"2024-04-02T14:44:00.396416+0000","last_fullsized":"2024-04-02T14:44:00.396416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:13.094123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099113+0000","last_change":"2024-04-02T14:42:21.260431+0000","last_active":"2024-04-02T14:44:00.099113+0000","last_peered":"2024-04-02T14:44:00.099113+0000","last_clean":"2024-04-02T14:44:00.099113+0000","last_became_active":"2024-04-02T14:42:21.259060+0000","last_became_peered":"2024-04-02T14:42:21.259060+0000","last_unstale":"2024-04-02T14:44:00.099113+0000","last_undegraded":"2024-04-02T14:44:00.099113+0000","last_fullsized":"2024-04-02T14:44:00.099113+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:07.753909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253595+0000","last_change":"2024-04-02T14:41:22.542847+0000","last_active":"2024-04-02T14:44:00.253595+0000","last_peered":"2024-04-02T14:44:00.253595+0000","last_clean":"2024-04-02T14:44:00.253595+0000","last_became_active":"2024-04-02T14:41:22.542476+0000","last_became_peered":"2024-04-02T14:41:22.542476+0000","last_unstale":"2024-04-02T14:44:00.253595+0000","last_undegraded":"2024-04-02T14:44:00.253595+0000","last_fullsized":"2024-04-02T14:44:00.253595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:27:12.102931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392293+0000","last_change":"2024-04-02T14:41:21.483382+0000","last_active":"2024-04-02T14:44:00.392293+0000","last_peered":"2024-04-02T14:44:00.392293+0000","last_clean":"2024-04-02T14:44:00.392293+0000","last_became_active":"2024-04-02T14:41:21.482996+0000","last_became_peered":"2024-04-02T14:41:21.482996+0000","last_unstale":"2024-04-02T14:44:00.392293+0000","last_undegraded":"2024-04-02T14:44:00.392293+0000","last_fullsized":"2024-04-02T14:44:00.392293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:37.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253567+0000","last_change":"2024-04-02T14:42:21.257441+0000","last_active":"2024-04-02T14:44:00.253567+0000","last_peered":"2024-04-02T14:44:00.253567+0000","last_clean":"2024-04-02T14:44:00.253567+0000","last_became_active":"2024-04-02T14:42:21.257003+0000","last_became_peered":"2024-04-02T14:42:21.257003+0000","last_unstale":"2024-04-02T14:44:00.253567+0000","last_undegraded":"2024-04-02T14:44:00.253567+0000","last_fullsized":"2024-04-02T14:44:00.253567+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:19:10.793720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250298+0000","last_change":"2024-04-02T14:41:22.525853+0000","last_active":"2024-04-02T14:44:00.250298+0000","last_peered":"2024-04-02T14:44:00.250298+0000","last_clean":"2024-04-02T14:44:00.250298+0000","last_became_active":"2024-04-02T14:41:22.525553+0000","last_became_peered":"2024-04-02T14:41:22.525553+0000","last_unstale":"2024-04-02T14:44:00.250298+0000","last_undegraded":"2024-04-02T14:44:00.250298+0000","last_fullsized":"2024-04-02T14:44:00.250298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:25:30.340974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.403468+0000","last_change":"2024-04-02T14:41:21.490632+0000","last_active":"2024-04-02T14:44:00.403468+0000","last_peered":"2024-04-02T14:44:00.403468+0000","last_clean":"2024-04-02T14:44:00.403468+0000","last_became_active":"2024-04-02T14:41:21.490208+0000","last_became_peered":"2024-04-02T14:41:21.490208+0000","last_unstale":"2024-04-02T14:44:00.403468+0000","last_undegraded":"2024-04-02T14:44:00.403468+0000","last_fullsized":"2024-04-02T14:44:00.403468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:18.566261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239701+0000","last_change":"2024-04-02T14:41:22.544872+0000","last_active":"2024-04-02T14:44:00.239701+0000","last_peered":"2024-04-02T14:44:00.239701+0000","last_clean":"2024-04-02T14:44:00.239701+0000","last_became_active":"2024-04-02T14:41:22.544268+0000","last_became_peered":"2024-04-02T14:41:22.544268+0000","last_unstale":"2024-04-02T14:44:00.239701+0000","last_undegraded":"2024-04-02T14:44:00.239701+0000","last_fullsized":"2024-04-02T14:44:00.239701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:54.963084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396149+0000","last_change":"2024-04-02T14:41:21.474411+0000","last_active":"2024-04-02T14:44:00.396149+0000","last_peered":"2024-04-02T14:44:00.396149+0000","last_clean":"2024-04-02T14:44:00.396149+0000","last_became_active":"2024-04-02T14:41:21.473950+0000","last_became_peered":"2024-04-02T14:41:21.473950+0000","last_unstale":"2024-04-02T14:44:00.396149+0000","last_undegraded":"2024-04-02T14:44:00.396149+0000","last_fullsized":"2024-04-02T14:44:00.396149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:46:05.892446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391825+0000","last_change":"2024-04-02T14:40:03.537813+0000","last_active":"2024-04-02T14:44:00.391825+0000","last_peered":"2024-04-02T14:44:00.391825+0000","last_clean":"2024-04-02T14:44:00.391825+0000","last_became_active":"2024-04-02T14:40:03.537223+0000","last_became_peered":"2024-04-02T14:40:03.537223+0000","last_unstale":"2024-04-02T14:44:00.391825+0000","last_undegraded":"2024-04-02T14:44:00.391825+0000","last_fullsized":"2024-04-02T14:44:00.391825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_clean_scrub_stamp":"2024-04-02T14:36:11.612815+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:19:10.346096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396548+0000","last_change":"2024-04-02T14:41:22.537906+0000","last_active":"2024-04-02T14:44:00.396548+0000","last_peered":"2024-04-02T14:44:00.396548+0000","last_clean":"2024-04-02T14:44:00.396548+0000","last_became_active":"2024-04-02T14:41:22.537238+0000","last_became_peered":"2024-04-02T14:41:22.537238+0000","last_unstale":"2024-04-02T14:44:00.396548+0000","last_undegraded":"2024-04-02T14:44:00.396548+0000","last_fullsized":"2024-04-02T14:44:00.396548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:57.089563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391955+0000","last_change":"2024-04-02T14:41:21.466830+0000","last_active":"2024-04-02T14:44:00.391955+0000","last_peered":"2024-04-02T14:44:00.391955+0000","last_clean":"2024-04-02T14:44:00.391955+0000","last_became_active":"2024-04-02T14:41:21.466490+0000","last_became_peered":"2024-04-02T14:41:21.466490+0000","last_unstale":"2024-04-02T14:44:00.391955+0000","last_undegraded":"2024-04-02T14:44:00.391955+0000","last_fullsized":"2024-04-02T14:44:00.391955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:15.597854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238180+0000","last_change":"2024-04-02T14:41:21.481139+0000","last_active":"2024-04-02T14:44:00.238180+0000","last_peered":"2024-04-02T14:44:00.238180+0000","last_clean":"2024-04-02T14:44:00.238180+0000","last_became_active":"2024-04-02T14:41:21.480754+0000","last_became_peered":"2024-04-02T14:41:21.480754+0000","last_unstale":"2024-04-02T14:44:00.238180+0000","last_undegraded":"2024-04-02T14:44:00.238180+0000","last_fullsized":"2024-04-02T14:44:00.238180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:03.716223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397939+0000","last_change":"2024-04-02T14:41:22.548483+0000","last_active":"2024-04-02T14:44:00.397939+0000","last_peered":"2024-04-02T14:44:00.397939+0000","last_clean":"2024-04-02T14:44:00.397939+0000","last_became_active":"2024-04-02T14:41:22.547463+0000","last_became_peered":"2024-04-02T14:41:22.547463+0000","last_unstale":"2024-04-02T14:44:00.397939+0000","last_undegraded":"2024-04-02T14:44:00.397939+0000","last_fullsized":"2024-04-02T14:44:00.397939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:33:58.076844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'49","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.390830+0000","last_change":"2024-04-02T14:41:21.491105+0000","last_active":"2024-04-02T14:44:00.390830+0000","last_peered":"2024-04-02T14:44:00.390830+0000","last_clean":"2024-04-02T14:44:00.390830+0000","last_became_active":"2024-04-02T14:41:21.490635+0000","last_became_peered":"2024-04-02T14:41:21.490635+0000","last_unstale":"2024-04-02T14:44:00.390830+0000","last_undegraded":"2024-04-02T14:44:00.390830+0000","last_fullsized":"2024-04-02T14:44:00.390830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:16.531064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239871+0000","last_change":"2024-04-02T14:41:22.553497+0000","last_active":"2024-04-02T14:44:00.239871+0000","last_peered":"2024-04-02T14:44:00.239871+0000","last_clean":"2024-04-02T14:44:00.239871+0000","last_became_active":"2024-04-02T14:41:22.550474+0000","last_became_peered":"2024-04-02T14:41:22.550474+0000","last_unstale":"2024-04-02T14:44:00.239871+0000","last_undegraded":"2024-04-02T14:44:00.239871+0000","last_fullsized":"2024-04-02T14:44:00.239871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:52.212454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396225+0000","last_change":"2024-04-02T14:41:21.479516+0000","last_active":"2024-04-02T14:44:00.396225+0000","last_peered":"2024-04-02T14:44:00.396225+0000","last_clean":"2024-04-02T14:44:00.396225+0000","last_became_active":"2024-04-02T14:41:21.478841+0000","last_became_peered":"2024-04-02T14:41:21.478841+0000","last_unstale":"2024-04-02T14:44:00.396225+0000","last_undegraded":"2024-04-02T14:44:00.396225+0000","last_fullsized":"2024-04-02T14:44:00.396225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:39.222316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398623+0000","last_change":"2024-04-02T14:41:22.550095+0000","last_active":"2024-04-02T14:44:00.398623+0000","last_peered":"2024-04-02T14:44:00.398623+0000","last_clean":"2024-04-02T14:44:00.398623+0000","last_became_active":"2024-04-02T14:41:22.547438+0000","last_became_peered":"2024-04-02T14:41:22.547438+0000","last_unstale":"2024-04-02T14:44:00.398623+0000","last_undegraded":"2024-04-02T14:44:00.398623+0000","last_fullsized":"2024-04-02T14:44:00.398623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:03:03.668289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362487+0000","last_change":"2024-04-02T14:41:21.482926+0000","last_active":"2024-04-02T14:44:00.362487+0000","last_peered":"2024-04-02T14:44:00.362487+0000","last_clean":"2024-04-02T14:44:00.362487+0000","last_became_active":"2024-04-02T14:41:21.482517+0000","last_became_peered":"2024-04-02T14:41:21.482517+0000","last_unstale":"2024-04-02T14:44:00.362487+0000","last_undegraded":"2024-04-02T14:44:00.362487+0000","last_fullsized":"2024-04-02T14:44:00.362487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:43.052390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.237874+0000","last_change":"2024-04-02T14:41:22.527758+0000","last_active":"2024-04-02T14:44:00.237874+0000","last_peered":"2024-04-02T14:44:00.237874+0000","last_clean":"2024-04-02T14:44:00.237874+0000","last_became_active":"2024-04-02T14:41:22.526954+0000","last_became_peered":"2024-04-02T14:41:22.526954+0000","last_unstale":"2024-04-02T14:44:00.237874+0000","last_undegraded":"2024-04-02T14:44:00.237874+0000","last_fullsized":"2024-04-02T14:44:00.237874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:34.988661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239213+0000","last_change":"2024-04-02T14:41:21.487534+0000","last_active":"2024-04-02T14:44:00.239213+0000","last_peered":"2024-04-02T14:44:00.239213+0000","last_clean":"2024-04-02T14:44:00.239213+0000","last_became_active":"2024-04-02T14:41:21.487168+0000","last_became_peered":"2024-04-02T14:41:21.487168+0000","last_unstale":"2024-04-02T14:44:00.239213+0000","last_undegraded":"2024-04-02T14:44:00.239213+0000","last_fullsized":"2024-04-02T14:44:00.239213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:03:20.636028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396152+0000","last_change":"2024-04-02T14:41:22.539246+0000","last_active":"2024-04-02T14:44:00.396152+0000","last_peered":"2024-04-02T14:44:00.396152+0000","last_clean":"2024-04-02T14:44:00.396152+0000","last_became_active":"2024-04-02T14:41:22.538864+0000","last_became_peered":"2024-04-02T14:41:22.538864+0000","last_unstale":"2024-04-02T14:44:00.396152+0000","last_undegraded":"2024-04-02T14:44:00.396152+0000","last_fullsized":"2024-04-02T14:44:00.396152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:40:13.418752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098701+0000","last_change":"2024-04-02T14:41:21.477043+0000","last_active":"2024-04-02T14:44:00.098701+0000","last_peered":"2024-04-02T14:44:00.098701+0000","last_clean":"2024-04-02T14:44:00.098701+0000","last_became_active":"2024-04-02T14:41:21.476580+0000","last_became_peered":"2024-04-02T14:41:21.476580+0000","last_unstale":"2024-04-02T14:44:00.098701+0000","last_undegraded":"2024-04-02T14:44:00.098701+0000","last_fullsized":"2024-04-02T14:44:00.098701+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:18.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239852+0000","last_change":"2024-04-02T14:41:22.549758+0000","last_active":"2024-04-02T14:44:00.239852+0000","last_peered":"2024-04-02T14:44:00.239852+0000","last_clean":"2024-04-02T14:44:00.239852+0000","last_became_active":"2024-04-02T14:41:22.545582+0000","last_became_peered":"2024-04-02T14:41:22.545582+0000","last_unstale":"2024-04-02T14:44:00.239852+0000","last_undegraded":"2024-04-02T14:44:00.239852+0000","last_fullsized":"2024-04-02T14:44:00.239852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:42.311921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396174+0000","last_change":"2024-04-02T14:42:21.253221+0000","last_active":"2024-04-02T14:44:00.396174+0000","last_peered":"2024-04-02T14:44:00.396174+0000","last_clean":"2024-04-02T14:44:00.396174+0000","last_became_active":"2024-04-02T14:42:21.247890+0000","last_became_peered":"2024-04-02T14:42:21.247890+0000","last_unstale":"2024-04-02T14:44:00.396174+0000","last_undegraded":"2024-04-02T14:44:00.396174+0000","last_fullsized":"2024-04-02T14:44:00.396174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:12.571769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392085+0000","last_change":"2024-04-02T14:41:22.531509+0000","last_active":"2024-04-02T14:44:00.392085+0000","last_peered":"2024-04-02T14:44:00.392085+0000","last_clean":"2024-04-02T14:44:00.392085+0000","last_became_active":"2024-04-02T14:41:22.531206+0000","last_became_peered":"2024-04-02T14:41:22.531206+0000","last_unstale":"2024-04-02T14:44:00.392085+0000","last_undegraded":"2024-04-02T14:44:00.392085+0000","last_fullsized":"2024-04-02T14:44:00.392085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:39.099177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098889+0000","last_change":"2024-04-02T14:41:21.485271+0000","last_active":"2024-04-02T14:44:00.098889+0000","last_peered":"2024-04-02T14:44:00.098889+0000","last_clean":"2024-04-02T14:44:00.098889+0000","last_became_active":"2024-04-02T14:41:21.484895+0000","last_became_peered":"2024-04-02T14:41:21.484895+0000","last_unstale":"2024-04-02T14:44:00.098889+0000","last_undegraded":"2024-04-02T14:44:00.098889+0000","last_fullsized":"2024-04-02T14:44:00.098889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.552092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":153,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392205+0000","last_change":"2024-04-02T14:41:22.524853+0000","last_active":"2024-04-02T14:44:00.392205+0000","last_peered":"2024-04-02T14:44:00.392205+0000","last_clean":"2024-04-02T14:44:00.392205+0000","last_became_active":"2024-04-02T14:41:22.524403+0000","last_became_peered":"2024-04-02T14:41:22.524403+0000","last_unstale":"2024-04-02T14:44:00.392205+0000","last_undegraded":"2024-04-02T14:44:00.392205+0000","last_fullsized":"2024-04-02T14:44:00.392205+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:31.322626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.386875+0000","last_change":"2024-04-02T14:41:21.474340+0000","last_active":"2024-04-02T14:44:00.386875+0000","last_peered":"2024-04-02T14:44:00.386875+0000","last_clean":"2024-04-02T14:44:00.386875+0000","last_became_active":"2024-04-02T14:41:21.473934+0000","last_became_peered":"2024-04-02T14:41:21.473934+0000","last_unstale":"2024-04-02T14:44:00.386875+0000","last_undegraded":"2024-04-02T14:44:00.386875+0000","last_fullsized":"2024-04-02T14:44:00.386875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:28:42.711817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391989+0000","last_change":"2024-04-02T14:41:22.543018+0000","last_active":"2024-04-02T14:44:00.391989+0000","last_peered":"2024-04-02T14:44:00.391989+0000","last_clean":"2024-04-02T14:44:00.391989+0000","last_became_active":"2024-04-02T14:41:22.542621+0000","last_became_peered":"2024-04-02T14:41:22.542621+0000","last_unstale":"2024-04-02T14:44:00.391989+0000","last_undegraded":"2024-04-02T14:44:00.391989+0000","last_fullsized":"2024-04-02T14:44:00.391989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:55:03.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398425+0000","last_change":"2024-04-02T14:41:21.493060+0000","last_active":"2024-04-02T14:44:00.398425+0000","last_peered":"2024-04-02T14:44:00.398425+0000","last_clean":"2024-04-02T14:44:00.398425+0000","last_became_active":"2024-04-02T14:41:21.492627+0000","last_became_peered":"2024-04-02T14:41:21.492627+0000","last_unstale":"2024-04-02T14:44:00.398425+0000","last_undegraded":"2024-04-02T14:44:00.398425+0000","last_fullsized":"2024-04-02T14:44:00.398425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:29:10.443866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392362+0000","last_change":"2024-04-02T14:41:22.527742+0000","last_active":"2024-04-02T14:44:00.392362+0000","last_peered":"2024-04-02T14:44:00.392362+0000","last_clean":"2024-04-02T14:44:00.392362+0000","last_became_active":"2024-04-02T14:41:22.527436+0000","last_became_peered":"2024-04-02T14:41:22.527436+0000","last_unstale":"2024-04-02T14:44:00.392362+0000","last_undegraded":"2024-04-02T14:44:00.392362+0000","last_fullsized":"2024-04-02T14:44:00.392362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:04:24.754907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396235+0000","last_change":"2024-04-02T14:41:22.526235+0000","last_active":"2024-04-02T14:44:00.396235+0000","last_peered":"2024-04-02T14:44:00.396235+0000","last_clean":"2024-04-02T14:44:00.396235+0000","last_became_active":"2024-04-02T14:41:22.525879+0000","last_became_peered":"2024-04-02T14:41:22.525879+0000","last_unstale":"2024-04-02T14:44:00.396235+0000","last_undegraded":"2024-04-02T14:44:00.396235+0000","last_fullsized":"2024-04-02T14:44:00.396235+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:20:54.431223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098551+0000","last_change":"2024-04-02T14:41:22.522498+0000","last_active":"2024-04-02T14:44:00.098551+0000","last_peered":"2024-04-02T14:44:00.098551+0000","last_clean":"2024-04-02T14:44:00.098551+0000","last_became_active":"2024-04-02T14:41:22.522162+0000","last_became_peered":"2024-04-02T14:41:22.522162+0000","last_unstale":"2024-04-02T14:44:00.098551+0000","last_undegraded":"2024-04-02T14:44:00.098551+0000","last_fullsized":"2024-04-02T14:44:00.098551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:42:04.701228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098399+0000","last_change":"2024-04-02T14:41:22.543920+0000","last_active":"2024-04-02T14:44:00.098399+0000","last_peered":"2024-04-02T14:44:00.098399+0000","last_clean":"2024-04-02T14:44:00.098399+0000","last_became_active":"2024-04-02T14:41:22.543552+0000","last_became_peered":"2024-04-02T14:41:22.543552+0000","last_unstale":"2024-04-02T14:44:00.098399+0000","last_undegraded":"2024-04-02T14:44:00.098399+0000","last_fullsized":"2024-04-02T14:44:00.098399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:28.761770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398084+0000","last_change":"2024-04-02T14:41:22.550507+0000","last_active":"2024-04-02T14:44:00.398084+0000","last_peered":"2024-04-02T14:44:00.398084+0000","last_clean":"2024-04-02T14:44:00.398084+0000","last_became_active":"2024-04-02T14:41:22.545824+0000","last_became_peered":"2024-04-02T14:41:22.545824+0000","last_unstale":"2024-04-02T14:44:00.398084+0000","last_undegraded":"2024-04-02T14:44:00.398084+0000","last_fullsized":"2024-04-02T14:44:00.398084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:11.175138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398099+0000","last_change":"2024-04-02T14:41:22.541362+0000","last_active":"2024-04-02T14:44:00.398099+0000","last_peered":"2024-04-02T14:44:00.398099+0000","last_clean":"2024-04-02T14:44:00.398099+0000","last_became_active":"2024-04-02T14:41:22.540927+0000","last_became_peered":"2024-04-02T14:41:22.540927+0000","last_unstale":"2024-04-02T14:44:00.398099+0000","last_undegraded":"2024-04-02T14:44:00.398099+0000","last_fullsized":"2024-04-02T14:44:00.398099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:43:00.746274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396335+0000","last_change":"2024-04-02T14:41:22.527146+0000","last_active":"2024-04-02T14:44:00.396335+0000","last_peered":"2024-04-02T14:44:00.396335+0000","last_clean":"2024-04-02T14:44:00.396335+0000","last_became_active":"2024-04-02T14:41:22.526810+0000","last_became_peered":"2024-04-02T14:41:22.526810+0000","last_unstale":"2024-04-02T14:44:00.396335+0000","last_undegraded":"2024-04-02T14:44:00.396335+0000","last_fullsized":"2024-04-02T14:44:00.396335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:18:46.464701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099056+0000","last_change":"2024-04-02T14:41:22.524458+0000","last_active":"2024-04-02T14:44:00.099056+0000","last_peered":"2024-04-02T14:44:00.099056+0000","last_clean":"2024-04-02T14:44:00.099056+0000","last_became_active":"2024-04-02T14:41:22.524138+0000","last_became_peered":"2024-04-02T14:41:22.524138+0000","last_unstale":"2024-04-02T14:44:00.099056+0000","last_undegraded":"2024-04-02T14:44:00.099056+0000","last_fullsized":"2024-04-02T14:44:00.099056+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:50:04.667964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239756+0000","last_change":"2024-04-02T14:42:21.251102+0000","last_active":"2024-04-02T14:44:00.239756+0000","last_peered":"2024-04-02T14:44:00.239756+0000","last_clean":"2024-04-02T14:44:00.239756+0000","last_became_active":"2024-04-02T14:42:21.249134+0000","last_became_peered":"2024-04-02T14:42:21.249134+0000","last_unstale":"2024-04-02T14:44:00.239756+0000","last_undegraded":"2024-04-02T14:44:00.239756+0000","last_fullsized":"2024-04-02T14:44:00.239756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:05:56.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.249896+0000","last_change":"2024-04-02T14:41:22.542319+0000","last_active":"2024-04-02T14:44:00.249896+0000","last_peered":"2024-04-02T14:44:00.249896+0000","last_clean":"2024-04-02T14:44:00.249896+0000","last_became_active":"2024-04-02T14:41:22.541959+0000","last_became_peered":"2024-04-02T14:41:22.541959+0000","last_unstale":"2024-04-02T14:44:00.249896+0000","last_undegraded":"2024-04-02T14:44:00.249896+0000","last_fullsized":"2024-04-02T14:44:00.249896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:49.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239058+0000","last_change":"2024-04-02T14:41:22.549932+0000","last_active":"2024-04-02T14:44:00.239058+0000","last_peered":"2024-04-02T14:44:00.239058+0000","last_clean":"2024-04-02T14:44:00.239058+0000","last_became_active":"2024-04-02T14:41:22.549231+0000","last_became_peered":"2024-04-02T14:41:22.549231+0000","last_unstale":"2024-04-02T14:44:00.239058+0000","last_undegraded":"2024-04-02T14:44:00.239058+0000","last_fullsized":"2024-04-02T14:44:00.239058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:01.485343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362610+0000","last_change":"2024-04-02T14:41:22.544712+0000","last_active":"2024-04-02T14:44:00.362610+0000","last_peered":"2024-04-02T14:44:00.362610+0000","last_clean":"2024-04-02T14:44:00.362610+0000","last_became_active":"2024-04-02T14:41:22.544043+0000","last_became_peered":"2024-04-02T14:41:22.544043+0000","last_unstale":"2024-04-02T14:44:00.362610+0000","last_undegraded":"2024-04-02T14:44:00.362610+0000","last_fullsized":"2024-04-02T14:44:00.362610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:07.931377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398079+0000","last_change":"2024-04-02T14:41:22.542364+0000","last_active":"2024-04-02T14:44:00.398079+0000","last_peered":"2024-04-02T14:44:00.398079+0000","last_clean":"2024-04-02T14:44:00.398079+0000","last_became_active":"2024-04-02T14:41:22.541929+0000","last_became_peered":"2024-04-02T14:41:22.541929+0000","last_unstale":"2024-04-02T14:44:00.398079+0000","last_undegraded":"2024-04-02T14:44:00.398079+0000","last_fullsized":"2024-04-02T14:44:00.398079+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:45.716132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396257+0000","last_change":"2024-04-02T14:41:22.525295+0000","last_active":"2024-04-02T14:44:00.396257+0000","last_peered":"2024-04-02T14:44:00.396257+0000","last_clean":"2024-04-02T14:44:00.396257+0000","last_became_active":"2024-04-02T14:41:22.524484+0000","last_became_peered":"2024-04-02T14:41:22.524484+0000","last_unstale":"2024-04-02T14:44:00.396257+0000","last_undegraded":"2024-04-02T14:44:00.396257+0000","last_fullsized":"2024-04-02T14:44:00.396257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:01.819082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597589,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1637,"num_read_kb":932,"num_write":367,"num_write_kb":1458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1792767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285612,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":478089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547249,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28988,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716164,"statfs":{"total":95995035648,"available":95965351936,"internally_reserved":0,"allocated":1835008,"data_stored":1264973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710775,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36720,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708432,"statfs":{"total":95995035648,"available":95957434368,"internally_reserved":0,"allocated":1413120,"data_stored":871235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972414,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36716,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708436,"statfs":{"total":95995035648,"available":95957438464,"internally_reserved":0,"allocated":1409024,"data_stored":871613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1613824,"data_stored":1071056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430280,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":468838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":542904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1024000,"data_stored":479075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":542893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117080,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":477561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855445,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1011712,"data_stored":479533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691918,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1630208,"data_stored":1071246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":198373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:44:06.868 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-02T14:44:06.868 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T14:44:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.098+0000 7fb7a8a00640 1 -- 172.21.15.62:0/391479518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a41025c0 msgr2=0x7fb7a41049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.098+0000 7fb7a8a00640 1 --2- 172.21.15.62:0/391479518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a41025c0 0x7fb7a41049b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb794009900 tx=0x7fb79402bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.098+0000 7fb7a8a00640 1 -- 172.21.15.62:0/391479518 shutdown_connections 2024-04-02T14:44:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.098+0000 7fb7a8a00640 1 --2- 172.21.15.62:0/391479518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7a4107820 0x7fb7a4109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.098+0000 7fb7a8a00640 1 --2- 172.21.15.62:0/391479518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7a4104ef0 0x7fb7a41072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.098+0000 7fb7a8a00640 1 --2- 172.21.15.62:0/391479518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a41025c0 0x7fb7a41049b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.098+0000 7fb7a8a00640 1 -- 172.21.15.62:0/391479518 >> 172.21.15.62:0/391479518 conn(0x7fb7a40fbfb0 msgr2=0x7fb7a40fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.099+0000 7fb7a8a00640 1 -- 172.21.15.62:0/391479518 shutdown_connections 2024-04-02T14:44:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.099+0000 7fb7a8a00640 1 -- 172.21.15.62:0/391479518 wait complete. 2024-04-02T14:44:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.100+0000 7fb7a8a00640 1 Processor -- start 2024-04-02T14:44:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.100+0000 7fb7a8a00640 1 -- start start 2024-04-02T14:44:07.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.101+0000 7fb7a8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7a41025c0 0x7fb7a4101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.101+0000 7fb7a8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7a4104ef0 0x7fb7a4100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.101+0000 7fb7a8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a4107820 0x7fb7a4100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.101+0000 7fb7a8a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a410d8a0 con 0x7fb7a41025c0 2024-04-02T14:44:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.101+0000 7fb7a8a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a410d720 con 0x7fb7a4104ef0 2024-04-02T14:44:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.101+0000 7fb7a8a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a410da20 con 0x7fb7a4107820 2024-04-02T14:44:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.101+0000 7fb7a1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7a4104ef0 0x7fb7a4100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.101+0000 7fb7a1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7a41025c0 0x7fb7a4101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.101+0000 7fb7a1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7a4104ef0 0x7fb7a4100570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41332/0 (socket says 172.21.15.62:41332) 2024-04-02T14:44:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.101+0000 7fb7a1400640 1 -- 172.21.15.62:0/1187815517 learned_addr learned my addr 172.21.15.62:0/1187815517 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.101+0000 7fb7a2800640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a4107820 0x7fb7a4100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:07.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.102+0000 7fb7a2800640 1 -- 172.21.15.62:0/1187815517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7a4104ef0 msgr2=0x7fb7a4100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:07.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.102+0000 7fb7a2800640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7a4104ef0 0x7fb7a4100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.102+0000 7fb7a2800640 1 -- 172.21.15.62:0/1187815517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7a41025c0 msgr2=0x7fb7a4101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.102+0000 7fb7a2800640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7a41025c0 0x7fb7a4101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.102+0000 7fb7a2800640 1 -- 172.21.15.62:0/1187815517 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a4101370 con 0x7fb7a4107820 2024-04-02T14:44:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.102+0000 7fb7a1400640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7a4104ef0 0x7fb7a4100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.102+0000 7fb7a1e00640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7a41025c0 0x7fb7a4101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.102+0000 7fb7a2800640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a4107820 0x7fb7a4100ab0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00ec50 tx=0x7fb79c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.102+0000 7fb79aa00640 1 -- 172.21.15.62:0/1187815517 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c003b70 con 0x7fb7a4107820 2024-04-02T14:44:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.103+0000 7fb79aa00640 1 -- 172.21.15.62:0/1187815517 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb79c003d10 con 0x7fb7a4107820 2024-04-02T14:44:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.103+0000 7fb7a8a00640 1 -- 172.21.15.62:0/1187815517 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a41a2140 con 0x7fb7a4107820 2024-04-02T14:44:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.103+0000 7fb79aa00640 1 -- 172.21.15.62:0/1187815517 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c010920 con 0x7fb7a4107820 2024-04-02T14:44:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.103+0000 7fb7a8a00640 1 -- 172.21.15.62:0/1187815517 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a41a2660 con 0x7fb7a4107820 2024-04-02T14:44:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.105+0000 7fb7a8a00640 1 -- 172.21.15.62:0/1187815517 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a40fdaa0 con 0x7fb7a4107820 2024-04-02T14:44:07.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.106+0000 7fb79aa00640 1 -- 172.21.15.62:0/1187815517 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fb79c004020 con 0x7fb7a4107820 2024-04-02T14:44:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.107+0000 7fb79aa00640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb7880a3ec0 0x7fb7880a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:07.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.108+0000 7fb79aa00640 1 -- 172.21.15.62:0/1187815517 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fb79c0d0fc0 con 0x7fb7a4107820 2024-04-02T14:44:07.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.108+0000 7fb7a1e00640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb7880a3ec0 0x7fb7880a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:07.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.109+0000 7fb7a1e00640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb7880a3ec0 0x7fb7880a6380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb794009230 tx=0x7fb794005fd0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:07.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.111+0000 7fb79aa00640 1 -- 172.21.15.62:0/1187815517 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb79c09d400 con 0x7fb7a4107820 2024-04-02T14:44:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.328+0000 7fb7a8a00640 1 -- 172.21.15.62:0/1187815517 --> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb7a4061f40 con 0x7fb7880a3ec0 2024-04-02T14:44:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.338+0000 7fb79aa00640 1 -- 172.21.15.62:0/1187815517 <== mgr.25273 v2:172.21.15.62:6800/110874546 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257789 (secure 0 0 0) 0x7fb7a4061f40 con 0x7fb7880a3ec0 2024-04-02T14:44:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:44:07.340 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-02T14:44:07.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:07.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:07.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi062 to 176.8M 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: Unable to set osd_memory_target on smithi062 to 185396019: error parsing value: Value '185396019' is below minimum 939524096 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: Updating smithi113:/etc/ceph/ceph.conf 2024-04-02T14:44:07.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-02T14:44:07.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:44:07.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: Updating smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:44:07.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: Updating smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:44:07.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:07 smithi062 ceph-mon[32116]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:44:07.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.342+0000 7fb7a8a00640 1 -- 172.21.15.62:0/1187815517 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb7880a3ec0 msgr2=0x7fb7880a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:07.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.342+0000 7fb7a8a00640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb7880a3ec0 0x7fb7880a6380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb794009230 tx=0x7fb794005fd0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.342+0000 7fb7a8a00640 1 -- 172.21.15.62:0/1187815517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a4107820 msgr2=0x7fb7a4100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:07.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.342+0000 7fb7a8a00640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a4107820 0x7fb7a4100ab0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00ec50 tx=0x7fb79c00c6a0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.343+0000 7fb7a8a00640 1 -- 172.21.15.62:0/1187815517 shutdown_connections 2024-04-02T14:44:07.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.343+0000 7fb7a8a00640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb7880a3ec0 0x7fb7880a6380 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.343+0000 7fb7a8a00640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a4107820 0x7fb7a4100ab0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.343+0000 7fb7a8a00640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7a4104ef0 0x7fb7a4100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.343+0000 7fb7a8a00640 1 --2- 172.21.15.62:0/1187815517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7a41025c0 0x7fb7a4101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.343+0000 7fb7a8a00640 1 -- 172.21.15.62:0/1187815517 >> 172.21.15.62:0/1187815517 conn(0x7fb7a40fbfb0 msgr2=0x7fb7a4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:07.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.343+0000 7fb7a8a00640 1 -- 172.21.15.62:0/1187815517 shutdown_connections 2024-04-02T14:44:07.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.343+0000 7fb7a8a00640 1 -- 172.21.15.62:0/1187815517 wait complete. 2024-04-02T14:44:07.358 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-02T14:44:06.118787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188191,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1773,"num_read_kb":1051,"num_write":550,"num_write_kb":4398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181749,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":934,"ondisk_log_size":934,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":405704,"kb_used_data":14736,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124536120,"statfs":{"total":1151940427776,"available":1151524986880,"internally_reserved":0,"allocated":15089664,"data_stored":8619016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.975999"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396472+0000","last_change":"2024-04-02T14:41:22.546468+0000","last_active":"2024-04-02T14:44:00.396472+0000","last_peered":"2024-04-02T14:44:00.396472+0000","last_clean":"2024-04-02T14:44:00.396472+0000","last_became_active":"2024-04-02T14:41:22.545991+0000","last_became_peered":"2024-04-02T14:41:22.545991+0000","last_unstale":"2024-04-02T14:44:00.396472+0000","last_undegraded":"2024-04-02T14:44:00.396472+0000","last_fullsized":"2024-04-02T14:44:00.396472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:34:24.520404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398615+0000","last_change":"2024-04-02T14:41:22.547738+0000","last_active":"2024-04-02T14:44:00.398615+0000","last_peered":"2024-04-02T14:44:00.398615+0000","last_clean":"2024-04-02T14:44:00.398615+0000","last_became_active":"2024-04-02T14:41:22.542372+0000","last_became_peered":"2024-04-02T14:41:22.542372+0000","last_unstale":"2024-04-02T14:44:00.398615+0000","last_undegraded":"2024-04-02T14:44:00.398615+0000","last_fullsized":"2024-04-02T14:44:00.398615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:07:26.951136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397831+0000","last_change":"2024-04-02T14:41:22.540173+0000","last_active":"2024-04-02T14:44:00.397831+0000","last_peered":"2024-04-02T14:44:00.397831+0000","last_clean":"2024-04-02T14:44:00.397831+0000","last_became_active":"2024-04-02T14:41:22.536279+0000","last_became_peered":"2024-04-02T14:41:22.536279+0000","last_unstale":"2024-04-02T14:44:00.397831+0000","last_undegraded":"2024-04-02T14:44:00.397831+0000","last_fullsized":"2024-04-02T14:44:00.397831+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:43:40.826934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099445+0000","last_change":"2024-04-02T14:42:21.251675+0000","last_active":"2024-04-02T14:44:00.099445+0000","last_peered":"2024-04-02T14:44:00.099445+0000","last_clean":"2024-04-02T14:44:00.099445+0000","last_became_active":"2024-04-02T14:42:21.250446+0000","last_became_peered":"2024-04-02T14:42:21.250446+0000","last_unstale":"2024-04-02T14:44:00.099445+0000","last_undegraded":"2024-04-02T14:44:00.099445+0000","last_fullsized":"2024-04-02T14:44:00.099445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:11.005276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238025+0000","last_change":"2024-04-02T14:41:22.554301+0000","last_active":"2024-04-02T14:44:00.238025+0000","last_peered":"2024-04-02T14:44:00.238025+0000","last_clean":"2024-04-02T14:44:00.238025+0000","last_became_active":"2024-04-02T14:41:22.554028+0000","last_became_peered":"2024-04-02T14:41:22.554028+0000","last_unstale":"2024-04-02T14:44:00.238025+0000","last_undegraded":"2024-04-02T14:44:00.238025+0000","last_fullsized":"2024-04-02T14:44:00.238025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:51:59.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253420+0000","last_change":"2024-04-02T14:41:22.543934+0000","last_active":"2024-04-02T14:44:00.253420+0000","last_peered":"2024-04-02T14:44:00.253420+0000","last_clean":"2024-04-02T14:44:00.253420+0000","last_became_active":"2024-04-02T14:41:22.543392+0000","last_became_peered":"2024-04-02T14:41:22.543392+0000","last_unstale":"2024-04-02T14:44:00.253420+0000","last_undegraded":"2024-04-02T14:44:00.253420+0000","last_fullsized":"2024-04-02T14:44:00.253420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:28:30.234356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391852+0000","last_change":"2024-04-02T14:41:22.547460+0000","last_active":"2024-04-02T14:44:00.391852+0000","last_peered":"2024-04-02T14:44:00.391852+0000","last_clean":"2024-04-02T14:44:00.391852+0000","last_became_active":"2024-04-02T14:41:22.547123+0000","last_became_peered":"2024-04-02T14:41:22.547123+0000","last_unstale":"2024-04-02T14:44:00.391852+0000","last_undegraded":"2024-04-02T14:44:00.391852+0000","last_fullsized":"2024-04-02T14:44:00.391852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:24:34.302530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239097+0000","last_change":"2024-04-02T14:41:22.551407+0000","last_active":"2024-04-02T14:44:00.239097+0000","last_peered":"2024-04-02T14:44:00.239097+0000","last_clean":"2024-04-02T14:44:00.239097+0000","last_became_active":"2024-04-02T14:41:22.538304+0000","last_became_peered":"2024-04-02T14:41:22.538304+0000","last_unstale":"2024-04-02T14:44:00.239097+0000","last_undegraded":"2024-04-02T14:44:00.239097+0000","last_fullsized":"2024-04-02T14:44:00.239097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:27.933380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362921+0000","last_change":"2024-04-02T14:41:22.523931+0000","last_active":"2024-04-02T14:44:00.362921+0000","last_peered":"2024-04-02T14:44:00.362921+0000","last_clean":"2024-04-02T14:44:00.362921+0000","last_became_active":"2024-04-02T14:41:22.523622+0000","last_became_peered":"2024-04-02T14:41:22.523622+0000","last_unstale":"2024-04-02T14:44:00.362921+0000","last_undegraded":"2024-04-02T14:44:00.362921+0000","last_fullsized":"2024-04-02T14:44:00.362921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:17.808290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253719+0000","last_change":"2024-04-02T14:41:22.525791+0000","last_active":"2024-04-02T14:44:00.253719+0000","last_peered":"2024-04-02T14:44:00.253719+0000","last_clean":"2024-04-02T14:44:00.253719+0000","last_became_active":"2024-04-02T14:41:22.525434+0000","last_became_peered":"2024-04-02T14:41:22.525434+0000","last_unstale":"2024-04-02T14:44:00.253719+0000","last_undegraded":"2024-04-02T14:44:00.253719+0000","last_fullsized":"2024-04-02T14:44:00.253719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:59.905337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392238+0000","last_change":"2024-04-02T14:41:22.527240+0000","last_active":"2024-04-02T14:44:00.392238+0000","last_peered":"2024-04-02T14:44:00.392238+0000","last_clean":"2024-04-02T14:44:00.392238+0000","last_became_active":"2024-04-02T14:41:22.526412+0000","last_became_peered":"2024-04-02T14:41:22.526412+0000","last_unstale":"2024-04-02T14:44:00.392238+0000","last_undegraded":"2024-04-02T14:44:00.392238+0000","last_fullsized":"2024-04-02T14:44:00.392238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:47.397012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396098+0000","last_change":"2024-04-02T14:41:22.527281+0000","last_active":"2024-04-02T14:44:00.396098+0000","last_peered":"2024-04-02T14:44:00.396098+0000","last_clean":"2024-04-02T14:44:00.396098+0000","last_became_active":"2024-04-02T14:41:22.526946+0000","last_became_peered":"2024-04-02T14:41:22.526946+0000","last_unstale":"2024-04-02T14:44:00.396098+0000","last_undegraded":"2024-04-02T14:44:00.396098+0000","last_fullsized":"2024-04-02T14:44:00.396098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:42.798282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398248+0000","last_change":"2024-04-02T14:41:22.534079+0000","last_active":"2024-04-02T14:44:00.398248+0000","last_peered":"2024-04-02T14:44:00.398248+0000","last_clean":"2024-04-02T14:44:00.398248+0000","last_became_active":"2024-04-02T14:41:22.532233+0000","last_became_peered":"2024-04-02T14:41:22.532233+0000","last_unstale":"2024-04-02T14:44:00.398248+0000","last_undegraded":"2024-04-02T14:44:00.398248+0000","last_fullsized":"2024-04-02T14:44:00.398248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:27:47.122933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397389+0000","last_change":"2024-04-02T14:41:22.544936+0000","last_active":"2024-04-02T14:44:00.397389+0000","last_peered":"2024-04-02T14:44:00.397389+0000","last_clean":"2024-04-02T14:44:00.397389+0000","last_became_active":"2024-04-02T14:41:22.541805+0000","last_became_peered":"2024-04-02T14:41:22.541805+0000","last_unstale":"2024-04-02T14:44:00.397389+0000","last_undegraded":"2024-04-02T14:44:00.397389+0000","last_fullsized":"2024-04-02T14:44:00.397389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:23.992807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397232+0000","last_change":"2024-04-02T14:41:22.534860+0000","last_active":"2024-04-02T14:44:00.397232+0000","last_peered":"2024-04-02T14:44:00.397232+0000","last_clean":"2024-04-02T14:44:00.397232+0000","last_became_active":"2024-04-02T14:41:22.534147+0000","last_became_peered":"2024-04-02T14:41:22.534147+0000","last_unstale":"2024-04-02T14:44:00.397232+0000","last_undegraded":"2024-04-02T14:44:00.397232+0000","last_fullsized":"2024-04-02T14:44:00.397232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:25.249364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362410+0000","last_change":"2024-04-02T14:41:22.537654+0000","last_active":"2024-04-02T14:44:00.362410+0000","last_peered":"2024-04-02T14:44:00.362410+0000","last_clean":"2024-04-02T14:44:00.362410+0000","last_became_active":"2024-04-02T14:41:22.537304+0000","last_became_peered":"2024-04-02T14:41:22.537304+0000","last_unstale":"2024-04-02T14:44:00.362410+0000","last_undegraded":"2024-04-02T14:44:00.362410+0000","last_fullsized":"2024-04-02T14:44:00.362410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:25.599724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396524+0000","last_change":"2024-04-02T14:41:22.546775+0000","last_active":"2024-04-02T14:44:00.396524+0000","last_peered":"2024-04-02T14:44:00.396524+0000","last_clean":"2024-04-02T14:44:00.396524+0000","last_became_active":"2024-04-02T14:41:22.546100+0000","last_became_peered":"2024-04-02T14:41:22.546100+0000","last_unstale":"2024-04-02T14:44:00.396524+0000","last_undegraded":"2024-04-02T14:44:00.396524+0000","last_fullsized":"2024-04-02T14:44:00.396524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:42:02.559335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099394+0000","last_change":"2024-04-02T14:41:22.543988+0000","last_active":"2024-04-02T14:44:00.099394+0000","last_peered":"2024-04-02T14:44:00.099394+0000","last_clean":"2024-04-02T14:44:00.099394+0000","last_became_active":"2024-04-02T14:41:22.543036+0000","last_became_peered":"2024-04-02T14:41:22.543036+0000","last_unstale":"2024-04-02T14:44:00.099394+0000","last_undegraded":"2024-04-02T14:44:00.099394+0000","last_fullsized":"2024-04-02T14:44:00.099394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:29.470929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098748+0000","last_change":"2024-04-02T14:41:22.543954+0000","last_active":"2024-04-02T14:44:00.098748+0000","last_peered":"2024-04-02T14:44:00.098748+0000","last_clean":"2024-04-02T14:44:00.098748+0000","last_became_active":"2024-04-02T14:41:22.543624+0000","last_became_peered":"2024-04-02T14:41:22.543624+0000","last_unstale":"2024-04-02T14:44:00.098748+0000","last_undegraded":"2024-04-02T14:44:00.098748+0000","last_fullsized":"2024-04-02T14:44:00.098748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:46.933877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":188,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.390351+0000","last_change":"2024-04-02T14:41:21.477341+0000","last_active":"2024-04-02T14:44:00.390351+0000","last_peered":"2024-04-02T14:44:00.390351+0000","last_clean":"2024-04-02T14:44:00.390351+0000","last_became_active":"2024-04-02T14:41:21.476708+0000","last_became_peered":"2024-04-02T14:41:21.476708+0000","last_unstale":"2024-04-02T14:44:00.390351+0000","last_undegraded":"2024-04-02T14:44:00.390351+0000","last_fullsized":"2024-04-02T14:44:00.390351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:34.867570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.241083+0000","last_change":"2024-04-02T14:41:22.551293+0000","last_active":"2024-04-02T14:44:00.241083+0000","last_peered":"2024-04-02T14:44:00.241083+0000","last_clean":"2024-04-02T14:44:00.241083+0000","last_became_active":"2024-04-02T14:41:22.547819+0000","last_became_peered":"2024-04-02T14:41:22.547819+0000","last_unstale":"2024-04-02T14:44:00.241083+0000","last_undegraded":"2024-04-02T14:44:00.241083+0000","last_fullsized":"2024-04-02T14:44:00.241083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:40.393005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.241214+0000","last_change":"2024-04-02T14:41:21.487096+0000","last_active":"2024-04-02T14:44:00.241214+0000","last_peered":"2024-04-02T14:44:00.241214+0000","last_clean":"2024-04-02T14:44:00.241214+0000","last_became_active":"2024-04-02T14:41:21.485883+0000","last_became_peered":"2024-04-02T14:41:21.485883+0000","last_unstale":"2024-04-02T14:44:00.241214+0000","last_undegraded":"2024-04-02T14:44:00.241214+0000","last_fullsized":"2024-04-02T14:44:00.241214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:24.017671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398946+0000","last_change":"2024-04-02T14:41:22.549234+0000","last_active":"2024-04-02T14:44:00.398946+0000","last_peered":"2024-04-02T14:44:00.398946+0000","last_clean":"2024-04-02T14:44:00.398946+0000","last_became_active":"2024-04-02T14:41:22.548855+0000","last_became_peered":"2024-04-02T14:41:22.548855+0000","last_unstale":"2024-04-02T14:44:00.398946+0000","last_undegraded":"2024-04-02T14:44:00.398946+0000","last_fullsized":"2024-04-02T14:44:00.398946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:05.047383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238406+0000","last_change":"2024-04-02T14:41:21.471673+0000","last_active":"2024-04-02T14:44:00.238406+0000","last_peered":"2024-04-02T14:44:00.238406+0000","last_clean":"2024-04-02T14:44:00.238406+0000","last_became_active":"2024-04-02T14:41:21.471292+0000","last_became_peered":"2024-04-02T14:41:21.471292+0000","last_unstale":"2024-04-02T14:44:00.238406+0000","last_undegraded":"2024-04-02T14:44:00.238406+0000","last_fullsized":"2024-04-02T14:44:00.238406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:42:55.665938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362654+0000","last_change":"2024-04-02T14:41:22.537665+0000","last_active":"2024-04-02T14:44:00.362654+0000","last_peered":"2024-04-02T14:44:00.362654+0000","last_clean":"2024-04-02T14:44:00.362654+0000","last_became_active":"2024-04-02T14:41:22.537274+0000","last_became_peered":"2024-04-02T14:41:22.537274+0000","last_unstale":"2024-04-02T14:44:00.362654+0000","last_undegraded":"2024-04-02T14:44:00.362654+0000","last_fullsized":"2024-04-02T14:44:00.362654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:14:39.200735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238373+0000","last_change":"2024-04-02T14:41:21.466547+0000","last_active":"2024-04-02T14:44:00.238373+0000","last_peered":"2024-04-02T14:44:00.238373+0000","last_clean":"2024-04-02T14:44:00.238373+0000","last_became_active":"2024-04-02T14:41:21.466063+0000","last_became_peered":"2024-04-02T14:41:21.466063+0000","last_unstale":"2024-04-02T14:44:00.238373+0000","last_undegraded":"2024-04-02T14:44:00.238373+0000","last_fullsized":"2024-04-02T14:44:00.238373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:39.201543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250276+0000","last_change":"2024-04-02T14:41:22.523984+0000","last_active":"2024-04-02T14:44:00.250276+0000","last_peered":"2024-04-02T14:44:00.250276+0000","last_clean":"2024-04-02T14:44:00.250276+0000","last_became_active":"2024-04-02T14:41:22.523524+0000","last_became_peered":"2024-04-02T14:41:22.523524+0000","last_unstale":"2024-04-02T14:44:00.250276+0000","last_undegraded":"2024-04-02T14:44:00.250276+0000","last_fullsized":"2024-04-02T14:44:00.250276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:31.542588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":196,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.440836+0000","last_change":"2024-04-02T14:42:21.255204+0000","last_active":"2024-04-02T14:44:00.440836+0000","last_peered":"2024-04-02T14:44:00.440836+0000","last_clean":"2024-04-02T14:44:00.440836+0000","last_became_active":"2024-04-02T14:42:21.248702+0000","last_became_peered":"2024-04-02T14:42:21.248702+0000","last_unstale":"2024-04-02T14:44:00.440836+0000","last_undegraded":"2024-04-02T14:44:00.440836+0000","last_fullsized":"2024-04-02T14:44:00.440836+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.723036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":151,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238112+0000","last_change":"2024-04-02T14:41:22.544107+0000","last_active":"2024-04-02T14:44:00.238112+0000","last_peered":"2024-04-02T14:44:00.238112+0000","last_clean":"2024-04-02T14:44:00.238112+0000","last_became_active":"2024-04-02T14:41:22.543739+0000","last_became_peered":"2024-04-02T14:41:22.543739+0000","last_unstale":"2024-04-02T14:44:00.238112+0000","last_undegraded":"2024-04-02T14:44:00.238112+0000","last_fullsized":"2024-04-02T14:44:00.238112+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:49:08.921668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253407+0000","last_change":"2024-04-02T14:41:21.479714+0000","last_active":"2024-04-02T14:44:00.253407+0000","last_peered":"2024-04-02T14:44:00.253407+0000","last_clean":"2024-04-02T14:44:00.253407+0000","last_became_active":"2024-04-02T14:41:21.479329+0000","last_became_peered":"2024-04-02T14:41:21.479329+0000","last_unstale":"2024-04-02T14:44:00.253407+0000","last_undegraded":"2024-04-02T14:44:00.253407+0000","last_fullsized":"2024-04-02T14:44:00.253407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:09.021017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099863+0000","last_change":"2024-04-02T14:42:21.254332+0000","last_active":"2024-04-02T14:44:00.099863+0000","last_peered":"2024-04-02T14:44:00.099863+0000","last_clean":"2024-04-02T14:44:00.099863+0000","last_became_active":"2024-04-02T14:42:21.251266+0000","last_became_peered":"2024-04-02T14:42:21.251266+0000","last_unstale":"2024-04-02T14:44:00.099863+0000","last_undegraded":"2024-04-02T14:44:00.099863+0000","last_fullsized":"2024-04-02T14:44:00.099863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:51:18.343098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396860+0000","last_change":"2024-04-02T14:41:21.484054+0000","last_active":"2024-04-02T14:44:00.396860+0000","last_peered":"2024-04-02T14:44:00.396860+0000","last_clean":"2024-04-02T14:44:00.396860+0000","last_became_active":"2024-04-02T14:41:21.483688+0000","last_became_peered":"2024-04-02T14:41:21.483688+0000","last_unstale":"2024-04-02T14:44:00.396860+0000","last_undegraded":"2024-04-02T14:44:00.396860+0000","last_fullsized":"2024-04-02T14:44:00.396860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:24.114180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238925+0000","last_change":"2024-04-02T14:41:22.553274+0000","last_active":"2024-04-02T14:44:00.238925+0000","last_peered":"2024-04-02T14:44:00.238925+0000","last_clean":"2024-04-02T14:44:00.238925+0000","last_became_active":"2024-04-02T14:41:22.545412+0000","last_became_peered":"2024-04-02T14:41:22.545412+0000","last_unstale":"2024-04-02T14:44:00.238925+0000","last_undegraded":"2024-04-02T14:44:00.238925+0000","last_fullsized":"2024-04-02T14:44:00.238925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:21.917119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.249831+0000","last_change":"2024-04-02T14:41:21.468582+0000","last_active":"2024-04-02T14:44:00.249831+0000","last_peered":"2024-04-02T14:44:00.249831+0000","last_clean":"2024-04-02T14:44:00.249831+0000","last_became_active":"2024-04-02T14:41:21.468081+0000","last_became_peered":"2024-04-02T14:41:21.468081+0000","last_unstale":"2024-04-02T14:44:00.249831+0000","last_undegraded":"2024-04-02T14:44:00.249831+0000","last_fullsized":"2024-04-02T14:44:00.249831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:19.593081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099196+0000","last_change":"2024-04-02T14:41:22.555627+0000","last_active":"2024-04-02T14:44:00.099196+0000","last_peered":"2024-04-02T14:44:00.099196+0000","last_clean":"2024-04-02T14:44:00.099196+0000","last_became_active":"2024-04-02T14:41:22.555171+0000","last_became_peered":"2024-04-02T14:41:22.555171+0000","last_unstale":"2024-04-02T14:44:00.099196+0000","last_undegraded":"2024-04-02T14:44:00.099196+0000","last_fullsized":"2024-04-02T14:44:00.099196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:21:28.161869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":330,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.403886+0000","last_change":"2024-04-02T14:41:21.475004+0000","last_active":"2024-04-02T14:44:00.403886+0000","last_peered":"2024-04-02T14:44:00.403886+0000","last_clean":"2024-04-02T14:44:00.403886+0000","last_became_active":"2024-04-02T14:41:21.474621+0000","last_became_peered":"2024-04-02T14:41:21.474621+0000","last_unstale":"2024-04-02T14:44:00.403886+0000","last_undegraded":"2024-04-02T14:44:00.403886+0000","last_fullsized":"2024-04-02T14:44:00.403886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:38.779276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":302,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362877+0000","last_change":"2024-04-02T14:41:22.543100+0000","last_active":"2024-04-02T14:44:00.362877+0000","last_peered":"2024-04-02T14:44:00.362877+0000","last_clean":"2024-04-02T14:44:00.362877+0000","last_became_active":"2024-04-02T14:41:22.542697+0000","last_became_peered":"2024-04-02T14:41:22.542697+0000","last_unstale":"2024-04-02T14:44:00.362877+0000","last_undegraded":"2024-04-02T14:44:00.362877+0000","last_fullsized":"2024-04-02T14:44:00.362877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:50:52.412879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392108+0000","last_change":"2024-04-02T14:41:21.468626+0000","last_active":"2024-04-02T14:44:00.392108+0000","last_peered":"2024-04-02T14:44:00.392108+0000","last_clean":"2024-04-02T14:44:00.392108+0000","last_became_active":"2024-04-02T14:41:21.468041+0000","last_became_peered":"2024-04-02T14:41:21.468041+0000","last_unstale":"2024-04-02T14:44:00.392108+0000","last_undegraded":"2024-04-02T14:44:00.392108+0000","last_fullsized":"2024-04-02T14:44:00.392108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:55.064033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398874+0000","last_change":"2024-04-02T14:41:22.539584+0000","last_active":"2024-04-02T14:44:00.398874+0000","last_peered":"2024-04-02T14:44:00.398874+0000","last_clean":"2024-04-02T14:44:00.398874+0000","last_became_active":"2024-04-02T14:41:22.539218+0000","last_became_peered":"2024-04-02T14:41:22.539218+0000","last_unstale":"2024-04-02T14:44:00.398874+0000","last_undegraded":"2024-04-02T14:44:00.398874+0000","last_fullsized":"2024-04-02T14:44:00.398874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:47:56.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250358+0000","last_change":"2024-04-02T14:41:21.481208+0000","last_active":"2024-04-02T14:44:00.250358+0000","last_peered":"2024-04-02T14:44:00.250358+0000","last_clean":"2024-04-02T14:44:00.250358+0000","last_became_active":"2024-04-02T14:41:21.480782+0000","last_became_peered":"2024-04-02T14:41:21.480782+0000","last_unstale":"2024-04-02T14:44:00.250358+0000","last_undegraded":"2024-04-02T14:44:00.250358+0000","last_fullsized":"2024-04-02T14:44:00.250358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:12.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362685+0000","last_change":"2024-04-02T14:41:22.517397+0000","last_active":"2024-04-02T14:44:00.362685+0000","last_peered":"2024-04-02T14:44:00.362685+0000","last_clean":"2024-04-02T14:44:00.362685+0000","last_became_active":"2024-04-02T14:41:22.517027+0000","last_became_peered":"2024-04-02T14:41:22.517027+0000","last_unstale":"2024-04-02T14:44:00.362685+0000","last_undegraded":"2024-04-02T14:44:00.362685+0000","last_fullsized":"2024-04-02T14:44:00.362685+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:53.129949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":368,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391430+0000","last_change":"2024-04-02T14:41:21.469643+0000","last_active":"2024-04-02T14:44:00.391430+0000","last_peered":"2024-04-02T14:44:00.391430+0000","last_clean":"2024-04-02T14:44:00.391430+0000","last_became_active":"2024-04-02T14:41:21.468730+0000","last_became_peered":"2024-04-02T14:41:21.468730+0000","last_unstale":"2024-04-02T14:44:00.391430+0000","last_undegraded":"2024-04-02T14:44:00.391430+0000","last_fullsized":"2024-04-02T14:44:00.391430+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:10.983551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64646,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":311,"num_read_kb":220,"num_write":37,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238145+0000","last_change":"2024-04-02T14:41:22.525052+0000","last_active":"2024-04-02T14:44:00.238145+0000","last_peered":"2024-04-02T14:44:00.238145+0000","last_clean":"2024-04-02T14:44:00.238145+0000","last_became_active":"2024-04-02T14:41:22.524691+0000","last_became_peered":"2024-04-02T14:41:22.524691+0000","last_unstale":"2024-04-02T14:44:00.238145+0000","last_undegraded":"2024-04-02T14:44:00.238145+0000","last_fullsized":"2024-04-02T14:44:00.238145+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:16.386020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098518+0000","last_change":"2024-04-02T14:41:21.476988+0000","last_active":"2024-04-02T14:44:00.098518+0000","last_peered":"2024-04-02T14:44:00.098518+0000","last_clean":"2024-04-02T14:44:00.098518+0000","last_became_active":"2024-04-02T14:41:21.476549+0000","last_became_peered":"2024-04-02T14:41:21.476549+0000","last_unstale":"2024-04-02T14:44:00.098518+0000","last_undegraded":"2024-04-02T14:44:00.098518+0000","last_fullsized":"2024-04-02T14:44:00.098518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:47.498626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397582+0000","last_change":"2024-04-02T14:41:22.544746+0000","last_active":"2024-04-02T14:44:00.397582+0000","last_peered":"2024-04-02T14:44:00.397582+0000","last_clean":"2024-04-02T14:44:00.397582+0000","last_became_active":"2024-04-02T14:41:22.544334+0000","last_became_peered":"2024-04-02T14:41:22.544334+0000","last_unstale":"2024-04-02T14:44:00.397582+0000","last_undegraded":"2024-04-02T14:44:00.397582+0000","last_fullsized":"2024-04-02T14:44:00.397582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:45:20.094455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250015+0000","last_change":"2024-04-02T14:41:21.472458+0000","last_active":"2024-04-02T14:44:00.250015+0000","last_peered":"2024-04-02T14:44:00.250015+0000","last_clean":"2024-04-02T14:44:00.250015+0000","last_became_active":"2024-04-02T14:41:21.472105+0000","last_became_peered":"2024-04-02T14:41:21.472105+0000","last_unstale":"2024-04-02T14:44:00.250015+0000","last_undegraded":"2024-04-02T14:44:00.250015+0000","last_fullsized":"2024-04-02T14:44:00.250015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:44.075807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392138+0000","last_change":"2024-04-02T14:41:22.537492+0000","last_active":"2024-04-02T14:44:00.392138+0000","last_peered":"2024-04-02T14:44:00.392138+0000","last_clean":"2024-04-02T14:44:00.392138+0000","last_became_active":"2024-04-02T14:41:22.537067+0000","last_became_peered":"2024-04-02T14:41:22.537067+0000","last_unstale":"2024-04-02T14:44:00.392138+0000","last_undegraded":"2024-04-02T14:44:00.392138+0000","last_fullsized":"2024-04-02T14:44:00.392138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:08.704340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":181,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.403962+0000","last_change":"2024-04-02T14:41:21.466017+0000","last_active":"2024-04-02T14:44:00.403962+0000","last_peered":"2024-04-02T14:44:00.403962+0000","last_clean":"2024-04-02T14:44:00.403962+0000","last_became_active":"2024-04-02T14:41:21.465600+0000","last_became_peered":"2024-04-02T14:41:21.465600+0000","last_unstale":"2024-04-02T14:44:00.403962+0000","last_undegraded":"2024-04-02T14:44:00.403962+0000","last_fullsized":"2024-04-02T14:44:00.403962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:52:08.131125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391756+0000","last_change":"2024-04-02T14:41:22.526628+0000","last_active":"2024-04-02T14:44:00.391756+0000","last_peered":"2024-04-02T14:44:00.391756+0000","last_clean":"2024-04-02T14:44:00.391756+0000","last_became_active":"2024-04-02T14:41:22.526283+0000","last_became_peered":"2024-04-02T14:41:22.526283+0000","last_unstale":"2024-04-02T14:44:00.391756+0000","last_undegraded":"2024-04-02T14:44:00.391756+0000","last_fullsized":"2024-04-02T14:44:00.391756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:41.812330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.237864+0000","last_change":"2024-04-02T14:41:21.475853+0000","last_active":"2024-04-02T14:44:00.237864+0000","last_peered":"2024-04-02T14:44:00.237864+0000","last_clean":"2024-04-02T14:44:00.237864+0000","last_became_active":"2024-04-02T14:41:21.475555+0000","last_became_peered":"2024-04-02T14:41:21.475555+0000","last_unstale":"2024-04-02T14:44:00.237864+0000","last_undegraded":"2024-04-02T14:44:00.237864+0000","last_fullsized":"2024-04-02T14:44:00.237864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:49.170906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.249931+0000","last_change":"2024-04-02T14:41:22.523848+0000","last_active":"2024-04-02T14:44:00.249931+0000","last_peered":"2024-04-02T14:44:00.249931+0000","last_clean":"2024-04-02T14:44:00.249931+0000","last_became_active":"2024-04-02T14:41:22.523476+0000","last_became_peered":"2024-04-02T14:41:22.523476+0000","last_unstale":"2024-04-02T14:44:00.249931+0000","last_undegraded":"2024-04-02T14:44:00.249931+0000","last_fullsized":"2024-04-02T14:44:00.249931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:47:46.933264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238399+0000","last_change":"2024-04-02T14:41:21.468556+0000","last_active":"2024-04-02T14:44:00.238399+0000","last_peered":"2024-04-02T14:44:00.238399+0000","last_clean":"2024-04-02T14:44:00.238399+0000","last_became_active":"2024-04-02T14:41:21.468215+0000","last_became_peered":"2024-04-02T14:41:21.468215+0000","last_unstale":"2024-04-02T14:44:00.238399+0000","last_undegraded":"2024-04-02T14:44:00.238399+0000","last_fullsized":"2024-04-02T14:44:00.238399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:41:04.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396416+0000","last_change":"2024-04-02T14:41:22.524228+0000","last_active":"2024-04-02T14:44:00.396416+0000","last_peered":"2024-04-02T14:44:00.396416+0000","last_clean":"2024-04-02T14:44:00.396416+0000","last_became_active":"2024-04-02T14:41:22.523839+0000","last_became_peered":"2024-04-02T14:41:22.523839+0000","last_unstale":"2024-04-02T14:44:00.396416+0000","last_undegraded":"2024-04-02T14:44:00.396416+0000","last_fullsized":"2024-04-02T14:44:00.396416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:13.094123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099113+0000","last_change":"2024-04-02T14:42:21.260431+0000","last_active":"2024-04-02T14:44:00.099113+0000","last_peered":"2024-04-02T14:44:00.099113+0000","last_clean":"2024-04-02T14:44:00.099113+0000","last_became_active":"2024-04-02T14:42:21.259060+0000","last_became_peered":"2024-04-02T14:42:21.259060+0000","last_unstale":"2024-04-02T14:44:00.099113+0000","last_undegraded":"2024-04-02T14:44:00.099113+0000","last_fullsized":"2024-04-02T14:44:00.099113+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:56:07.753909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253595+0000","last_change":"2024-04-02T14:41:22.542847+0000","last_active":"2024-04-02T14:44:00.253595+0000","last_peered":"2024-04-02T14:44:00.253595+0000","last_clean":"2024-04-02T14:44:00.253595+0000","last_became_active":"2024-04-02T14:41:22.542476+0000","last_became_peered":"2024-04-02T14:41:22.542476+0000","last_unstale":"2024-04-02T14:44:00.253595+0000","last_undegraded":"2024-04-02T14:44:00.253595+0000","last_fullsized":"2024-04-02T14:44:00.253595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:27:12.102931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392293+0000","last_change":"2024-04-02T14:41:21.483382+0000","last_active":"2024-04-02T14:44:00.392293+0000","last_peered":"2024-04-02T14:44:00.392293+0000","last_clean":"2024-04-02T14:44:00.392293+0000","last_became_active":"2024-04-02T14:41:21.482996+0000","last_became_peered":"2024-04-02T14:41:21.482996+0000","last_unstale":"2024-04-02T14:44:00.392293+0000","last_undegraded":"2024-04-02T14:44:00.392293+0000","last_fullsized":"2024-04-02T14:44:00.392293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:37.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.253567+0000","last_change":"2024-04-02T14:42:21.257441+0000","last_active":"2024-04-02T14:44:00.253567+0000","last_peered":"2024-04-02T14:44:00.253567+0000","last_clean":"2024-04-02T14:44:00.253567+0000","last_became_active":"2024-04-02T14:42:21.257003+0000","last_became_peered":"2024-04-02T14:42:21.257003+0000","last_unstale":"2024-04-02T14:44:00.253567+0000","last_undegraded":"2024-04-02T14:44:00.253567+0000","last_fullsized":"2024-04-02T14:44:00.253567+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:19:10.793720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.250298+0000","last_change":"2024-04-02T14:41:22.525853+0000","last_active":"2024-04-02T14:44:00.250298+0000","last_peered":"2024-04-02T14:44:00.250298+0000","last_clean":"2024-04-02T14:44:00.250298+0000","last_became_active":"2024-04-02T14:41:22.525553+0000","last_became_peered":"2024-04-02T14:41:22.525553+0000","last_unstale":"2024-04-02T14:44:00.250298+0000","last_undegraded":"2024-04-02T14:44:00.250298+0000","last_fullsized":"2024-04-02T14:44:00.250298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:25:30.340974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.403468+0000","last_change":"2024-04-02T14:41:21.490632+0000","last_active":"2024-04-02T14:44:00.403468+0000","last_peered":"2024-04-02T14:44:00.403468+0000","last_clean":"2024-04-02T14:44:00.403468+0000","last_became_active":"2024-04-02T14:41:21.490208+0000","last_became_peered":"2024-04-02T14:41:21.490208+0000","last_unstale":"2024-04-02T14:44:00.403468+0000","last_undegraded":"2024-04-02T14:44:00.403468+0000","last_fullsized":"2024-04-02T14:44:00.403468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:18.566261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239701+0000","last_change":"2024-04-02T14:41:22.544872+0000","last_active":"2024-04-02T14:44:00.239701+0000","last_peered":"2024-04-02T14:44:00.239701+0000","last_clean":"2024-04-02T14:44:00.239701+0000","last_became_active":"2024-04-02T14:41:22.544268+0000","last_became_peered":"2024-04-02T14:41:22.544268+0000","last_unstale":"2024-04-02T14:44:00.239701+0000","last_undegraded":"2024-04-02T14:44:00.239701+0000","last_fullsized":"2024-04-02T14:44:00.239701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:54.963084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396149+0000","last_change":"2024-04-02T14:41:21.474411+0000","last_active":"2024-04-02T14:44:00.396149+0000","last_peered":"2024-04-02T14:44:00.396149+0000","last_clean":"2024-04-02T14:44:00.396149+0000","last_became_active":"2024-04-02T14:41:21.473950+0000","last_became_peered":"2024-04-02T14:41:21.473950+0000","last_unstale":"2024-04-02T14:44:00.396149+0000","last_undegraded":"2024-04-02T14:44:00.396149+0000","last_fullsized":"2024-04-02T14:44:00.396149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:46:05.892446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391825+0000","last_change":"2024-04-02T14:40:03.537813+0000","last_active":"2024-04-02T14:44:00.391825+0000","last_peered":"2024-04-02T14:44:00.391825+0000","last_clean":"2024-04-02T14:44:00.391825+0000","last_became_active":"2024-04-02T14:40:03.537223+0000","last_became_peered":"2024-04-02T14:40:03.537223+0000","last_unstale":"2024-04-02T14:44:00.391825+0000","last_undegraded":"2024-04-02T14:44:00.391825+0000","last_fullsized":"2024-04-02T14:44:00.391825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:36:11.612815+0000","last_clean_scrub_stamp":"2024-04-02T14:36:11.612815+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:19:10.346096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396548+0000","last_change":"2024-04-02T14:41:22.537906+0000","last_active":"2024-04-02T14:44:00.396548+0000","last_peered":"2024-04-02T14:44:00.396548+0000","last_clean":"2024-04-02T14:44:00.396548+0000","last_became_active":"2024-04-02T14:41:22.537238+0000","last_became_peered":"2024-04-02T14:41:22.537238+0000","last_unstale":"2024-04-02T14:44:00.396548+0000","last_undegraded":"2024-04-02T14:44:00.396548+0000","last_fullsized":"2024-04-02T14:44:00.396548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:57.089563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391955+0000","last_change":"2024-04-02T14:41:21.466830+0000","last_active":"2024-04-02T14:44:00.391955+0000","last_peered":"2024-04-02T14:44:00.391955+0000","last_clean":"2024-04-02T14:44:00.391955+0000","last_became_active":"2024-04-02T14:41:21.466490+0000","last_became_peered":"2024-04-02T14:41:21.466490+0000","last_unstale":"2024-04-02T14:44:00.391955+0000","last_undegraded":"2024-04-02T14:44:00.391955+0000","last_fullsized":"2024-04-02T14:44:00.391955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:15.597854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.238180+0000","last_change":"2024-04-02T14:41:21.481139+0000","last_active":"2024-04-02T14:44:00.238180+0000","last_peered":"2024-04-02T14:44:00.238180+0000","last_clean":"2024-04-02T14:44:00.238180+0000","last_became_active":"2024-04-02T14:41:21.480754+0000","last_became_peered":"2024-04-02T14:41:21.480754+0000","last_unstale":"2024-04-02T14:44:00.238180+0000","last_undegraded":"2024-04-02T14:44:00.238180+0000","last_fullsized":"2024-04-02T14:44:00.238180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:03.716223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.397939+0000","last_change":"2024-04-02T14:41:22.548483+0000","last_active":"2024-04-02T14:44:00.397939+0000","last_peered":"2024-04-02T14:44:00.397939+0000","last_clean":"2024-04-02T14:44:00.397939+0000","last_became_active":"2024-04-02T14:41:22.547463+0000","last_became_peered":"2024-04-02T14:41:22.547463+0000","last_unstale":"2024-04-02T14:44:00.397939+0000","last_undegraded":"2024-04-02T14:44:00.397939+0000","last_fullsized":"2024-04-02T14:44:00.397939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:33:58.076844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'49","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.390830+0000","last_change":"2024-04-02T14:41:21.491105+0000","last_active":"2024-04-02T14:44:00.390830+0000","last_peered":"2024-04-02T14:44:00.390830+0000","last_clean":"2024-04-02T14:44:00.390830+0000","last_became_active":"2024-04-02T14:41:21.490635+0000","last_became_peered":"2024-04-02T14:41:21.490635+0000","last_unstale":"2024-04-02T14:44:00.390830+0000","last_undegraded":"2024-04-02T14:44:00.390830+0000","last_fullsized":"2024-04-02T14:44:00.390830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:16.531064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239871+0000","last_change":"2024-04-02T14:41:22.553497+0000","last_active":"2024-04-02T14:44:00.239871+0000","last_peered":"2024-04-02T14:44:00.239871+0000","last_clean":"2024-04-02T14:44:00.239871+0000","last_became_active":"2024-04-02T14:41:22.550474+0000","last_became_peered":"2024-04-02T14:41:22.550474+0000","last_unstale":"2024-04-02T14:44:00.239871+0000","last_undegraded":"2024-04-02T14:44:00.239871+0000","last_fullsized":"2024-04-02T14:44:00.239871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:52.212454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396225+0000","last_change":"2024-04-02T14:41:21.479516+0000","last_active":"2024-04-02T14:44:00.396225+0000","last_peered":"2024-04-02T14:44:00.396225+0000","last_clean":"2024-04-02T14:44:00.396225+0000","last_became_active":"2024-04-02T14:41:21.478841+0000","last_became_peered":"2024-04-02T14:41:21.478841+0000","last_unstale":"2024-04-02T14:44:00.396225+0000","last_undegraded":"2024-04-02T14:44:00.396225+0000","last_fullsized":"2024-04-02T14:44:00.396225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:39.222316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398623+0000","last_change":"2024-04-02T14:41:22.550095+0000","last_active":"2024-04-02T14:44:00.398623+0000","last_peered":"2024-04-02T14:44:00.398623+0000","last_clean":"2024-04-02T14:44:00.398623+0000","last_became_active":"2024-04-02T14:41:22.547438+0000","last_became_peered":"2024-04-02T14:41:22.547438+0000","last_unstale":"2024-04-02T14:44:00.398623+0000","last_undegraded":"2024-04-02T14:44:00.398623+0000","last_fullsized":"2024-04-02T14:44:00.398623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:03:03.668289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362487+0000","last_change":"2024-04-02T14:41:21.482926+0000","last_active":"2024-04-02T14:44:00.362487+0000","last_peered":"2024-04-02T14:44:00.362487+0000","last_clean":"2024-04-02T14:44:00.362487+0000","last_became_active":"2024-04-02T14:41:21.482517+0000","last_became_peered":"2024-04-02T14:41:21.482517+0000","last_unstale":"2024-04-02T14:44:00.362487+0000","last_undegraded":"2024-04-02T14:44:00.362487+0000","last_fullsized":"2024-04-02T14:44:00.362487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:43.052390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.237874+0000","last_change":"2024-04-02T14:41:22.527758+0000","last_active":"2024-04-02T14:44:00.237874+0000","last_peered":"2024-04-02T14:44:00.237874+0000","last_clean":"2024-04-02T14:44:00.237874+0000","last_became_active":"2024-04-02T14:41:22.526954+0000","last_became_peered":"2024-04-02T14:41:22.526954+0000","last_unstale":"2024-04-02T14:44:00.237874+0000","last_undegraded":"2024-04-02T14:44:00.237874+0000","last_fullsized":"2024-04-02T14:44:00.237874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:34.988661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239213+0000","last_change":"2024-04-02T14:41:21.487534+0000","last_active":"2024-04-02T14:44:00.239213+0000","last_peered":"2024-04-02T14:44:00.239213+0000","last_clean":"2024-04-02T14:44:00.239213+0000","last_became_active":"2024-04-02T14:41:21.487168+0000","last_became_peered":"2024-04-02T14:41:21.487168+0000","last_unstale":"2024-04-02T14:44:00.239213+0000","last_undegraded":"2024-04-02T14:44:00.239213+0000","last_fullsized":"2024-04-02T14:44:00.239213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:03:20.636028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396152+0000","last_change":"2024-04-02T14:41:22.539246+0000","last_active":"2024-04-02T14:44:00.396152+0000","last_peered":"2024-04-02T14:44:00.396152+0000","last_clean":"2024-04-02T14:44:00.396152+0000","last_became_active":"2024-04-02T14:41:22.538864+0000","last_became_peered":"2024-04-02T14:41:22.538864+0000","last_unstale":"2024-04-02T14:44:00.396152+0000","last_undegraded":"2024-04-02T14:44:00.396152+0000","last_fullsized":"2024-04-02T14:44:00.396152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:40:13.418752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098701+0000","last_change":"2024-04-02T14:41:21.477043+0000","last_active":"2024-04-02T14:44:00.098701+0000","last_peered":"2024-04-02T14:44:00.098701+0000","last_clean":"2024-04-02T14:44:00.098701+0000","last_became_active":"2024-04-02T14:41:21.476580+0000","last_became_peered":"2024-04-02T14:41:21.476580+0000","last_unstale":"2024-04-02T14:44:00.098701+0000","last_undegraded":"2024-04-02T14:44:00.098701+0000","last_fullsized":"2024-04-02T14:44:00.098701+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:41:18.149329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239852+0000","last_change":"2024-04-02T14:41:22.549758+0000","last_active":"2024-04-02T14:44:00.239852+0000","last_peered":"2024-04-02T14:44:00.239852+0000","last_clean":"2024-04-02T14:44:00.239852+0000","last_became_active":"2024-04-02T14:41:22.545582+0000","last_became_peered":"2024-04-02T14:41:22.545582+0000","last_unstale":"2024-04-02T14:44:00.239852+0000","last_undegraded":"2024-04-02T14:44:00.239852+0000","last_fullsized":"2024-04-02T14:44:00.239852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:42.311921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396174+0000","last_change":"2024-04-02T14:42:21.253221+0000","last_active":"2024-04-02T14:44:00.396174+0000","last_peered":"2024-04-02T14:44:00.396174+0000","last_clean":"2024-04-02T14:44:00.396174+0000","last_became_active":"2024-04-02T14:42:21.247890+0000","last_became_peered":"2024-04-02T14:42:21.247890+0000","last_unstale":"2024-04-02T14:44:00.396174+0000","last_undegraded":"2024-04-02T14:44:00.396174+0000","last_fullsized":"2024-04-02T14:44:00.396174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:12.571769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392085+0000","last_change":"2024-04-02T14:41:22.531509+0000","last_active":"2024-04-02T14:44:00.392085+0000","last_peered":"2024-04-02T14:44:00.392085+0000","last_clean":"2024-04-02T14:44:00.392085+0000","last_became_active":"2024-04-02T14:41:22.531206+0000","last_became_peered":"2024-04-02T14:41:22.531206+0000","last_unstale":"2024-04-02T14:44:00.392085+0000","last_undegraded":"2024-04-02T14:44:00.392085+0000","last_fullsized":"2024-04-02T14:44:00.392085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:39.099177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098889+0000","last_change":"2024-04-02T14:41:21.485271+0000","last_active":"2024-04-02T14:44:00.098889+0000","last_peered":"2024-04-02T14:44:00.098889+0000","last_clean":"2024-04-02T14:44:00.098889+0000","last_became_active":"2024-04-02T14:41:21.484895+0000","last_became_peered":"2024-04-02T14:41:21.484895+0000","last_unstale":"2024-04-02T14:44:00.098889+0000","last_undegraded":"2024-04-02T14:44:00.098889+0000","last_fullsized":"2024-04-02T14:44:00.098889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.552092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":153,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392205+0000","last_change":"2024-04-02T14:41:22.524853+0000","last_active":"2024-04-02T14:44:00.392205+0000","last_peered":"2024-04-02T14:44:00.392205+0000","last_clean":"2024-04-02T14:44:00.392205+0000","last_became_active":"2024-04-02T14:41:22.524403+0000","last_became_peered":"2024-04-02T14:41:22.524403+0000","last_unstale":"2024-04-02T14:44:00.392205+0000","last_undegraded":"2024-04-02T14:44:00.392205+0000","last_fullsized":"2024-04-02T14:44:00.392205+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:31.322626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.386875+0000","last_change":"2024-04-02T14:41:21.474340+0000","last_active":"2024-04-02T14:44:00.386875+0000","last_peered":"2024-04-02T14:44:00.386875+0000","last_clean":"2024-04-02T14:44:00.386875+0000","last_became_active":"2024-04-02T14:41:21.473934+0000","last_became_peered":"2024-04-02T14:41:21.473934+0000","last_unstale":"2024-04-02T14:44:00.386875+0000","last_undegraded":"2024-04-02T14:44:00.386875+0000","last_fullsized":"2024-04-02T14:44:00.386875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:28:42.711817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.391989+0000","last_change":"2024-04-02T14:41:22.543018+0000","last_active":"2024-04-02T14:44:00.391989+0000","last_peered":"2024-04-02T14:44:00.391989+0000","last_clean":"2024-04-02T14:44:00.391989+0000","last_became_active":"2024-04-02T14:41:22.542621+0000","last_became_peered":"2024-04-02T14:41:22.542621+0000","last_unstale":"2024-04-02T14:44:00.391989+0000","last_undegraded":"2024-04-02T14:44:00.391989+0000","last_fullsized":"2024-04-02T14:44:00.391989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:55:03.582181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398425+0000","last_change":"2024-04-02T14:41:21.493060+0000","last_active":"2024-04-02T14:44:00.398425+0000","last_peered":"2024-04-02T14:44:00.398425+0000","last_clean":"2024-04-02T14:44:00.398425+0000","last_became_active":"2024-04-02T14:41:21.492627+0000","last_became_peered":"2024-04-02T14:41:21.492627+0000","last_unstale":"2024-04-02T14:44:00.398425+0000","last_undegraded":"2024-04-02T14:44:00.398425+0000","last_fullsized":"2024-04-02T14:44:00.398425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:20.444294+0000","last_clean_scrub_stamp":"2024-04-02T14:41:20.444294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:29:10.443866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.392362+0000","last_change":"2024-04-02T14:41:22.527742+0000","last_active":"2024-04-02T14:44:00.392362+0000","last_peered":"2024-04-02T14:44:00.392362+0000","last_clean":"2024-04-02T14:44:00.392362+0000","last_became_active":"2024-04-02T14:41:22.527436+0000","last_became_peered":"2024-04-02T14:41:22.527436+0000","last_unstale":"2024-04-02T14:44:00.392362+0000","last_undegraded":"2024-04-02T14:44:00.392362+0000","last_fullsized":"2024-04-02T14:44:00.392362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:04:24.754907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396235+0000","last_change":"2024-04-02T14:41:22.526235+0000","last_active":"2024-04-02T14:44:00.396235+0000","last_peered":"2024-04-02T14:44:00.396235+0000","last_clean":"2024-04-02T14:44:00.396235+0000","last_became_active":"2024-04-02T14:41:22.525879+0000","last_became_peered":"2024-04-02T14:41:22.525879+0000","last_unstale":"2024-04-02T14:44:00.396235+0000","last_undegraded":"2024-04-02T14:44:00.396235+0000","last_fullsized":"2024-04-02T14:44:00.396235+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:20:54.431223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098551+0000","last_change":"2024-04-02T14:41:22.522498+0000","last_active":"2024-04-02T14:44:00.098551+0000","last_peered":"2024-04-02T14:44:00.098551+0000","last_clean":"2024-04-02T14:44:00.098551+0000","last_became_active":"2024-04-02T14:41:22.522162+0000","last_became_peered":"2024-04-02T14:41:22.522162+0000","last_unstale":"2024-04-02T14:44:00.098551+0000","last_undegraded":"2024-04-02T14:44:00.098551+0000","last_fullsized":"2024-04-02T14:44:00.098551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:42:04.701228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.098399+0000","last_change":"2024-04-02T14:41:22.543920+0000","last_active":"2024-04-02T14:44:00.098399+0000","last_peered":"2024-04-02T14:44:00.098399+0000","last_clean":"2024-04-02T14:44:00.098399+0000","last_became_active":"2024-04-02T14:41:22.543552+0000","last_became_peered":"2024-04-02T14:41:22.543552+0000","last_unstale":"2024-04-02T14:44:00.098399+0000","last_undegraded":"2024-04-02T14:44:00.098399+0000","last_fullsized":"2024-04-02T14:44:00.098399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:28.761770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398084+0000","last_change":"2024-04-02T14:41:22.550507+0000","last_active":"2024-04-02T14:44:00.398084+0000","last_peered":"2024-04-02T14:44:00.398084+0000","last_clean":"2024-04-02T14:44:00.398084+0000","last_became_active":"2024-04-02T14:41:22.545824+0000","last_became_peered":"2024-04-02T14:41:22.545824+0000","last_unstale":"2024-04-02T14:44:00.398084+0000","last_undegraded":"2024-04-02T14:44:00.398084+0000","last_fullsized":"2024-04-02T14:44:00.398084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:11.175138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398099+0000","last_change":"2024-04-02T14:41:22.541362+0000","last_active":"2024-04-02T14:44:00.398099+0000","last_peered":"2024-04-02T14:44:00.398099+0000","last_clean":"2024-04-02T14:44:00.398099+0000","last_became_active":"2024-04-02T14:41:22.540927+0000","last_became_peered":"2024-04-02T14:41:22.540927+0000","last_unstale":"2024-04-02T14:44:00.398099+0000","last_undegraded":"2024-04-02T14:44:00.398099+0000","last_fullsized":"2024-04-02T14:44:00.398099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:43:00.746274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396335+0000","last_change":"2024-04-02T14:41:22.527146+0000","last_active":"2024-04-02T14:44:00.396335+0000","last_peered":"2024-04-02T14:44:00.396335+0000","last_clean":"2024-04-02T14:44:00.396335+0000","last_became_active":"2024-04-02T14:41:22.526810+0000","last_became_peered":"2024-04-02T14:41:22.526810+0000","last_unstale":"2024-04-02T14:44:00.396335+0000","last_undegraded":"2024-04-02T14:44:00.396335+0000","last_fullsized":"2024-04-02T14:44:00.396335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:18:46.464701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.099056+0000","last_change":"2024-04-02T14:41:22.524458+0000","last_active":"2024-04-02T14:44:00.099056+0000","last_peered":"2024-04-02T14:44:00.099056+0000","last_clean":"2024-04-02T14:44:00.099056+0000","last_became_active":"2024-04-02T14:41:22.524138+0000","last_became_peered":"2024-04-02T14:41:22.524138+0000","last_unstale":"2024-04-02T14:44:00.099056+0000","last_undegraded":"2024-04-02T14:44:00.099056+0000","last_fullsized":"2024-04-02T14:44:00.099056+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:50:04.667964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239756+0000","last_change":"2024-04-02T14:42:21.251102+0000","last_active":"2024-04-02T14:44:00.239756+0000","last_peered":"2024-04-02T14:44:00.239756+0000","last_clean":"2024-04-02T14:44:00.239756+0000","last_became_active":"2024-04-02T14:42:21.249134+0000","last_became_peered":"2024-04-02T14:42:21.249134+0000","last_unstale":"2024-04-02T14:44:00.239756+0000","last_undegraded":"2024-04-02T14:44:00.239756+0000","last_fullsized":"2024-04-02T14:44:00.239756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:05:56.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.249896+0000","last_change":"2024-04-02T14:41:22.542319+0000","last_active":"2024-04-02T14:44:00.249896+0000","last_peered":"2024-04-02T14:44:00.249896+0000","last_clean":"2024-04-02T14:44:00.249896+0000","last_became_active":"2024-04-02T14:41:22.541959+0000","last_became_peered":"2024-04-02T14:41:22.541959+0000","last_unstale":"2024-04-02T14:44:00.249896+0000","last_undegraded":"2024-04-02T14:44:00.249896+0000","last_fullsized":"2024-04-02T14:44:00.249896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:49.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.239058+0000","last_change":"2024-04-02T14:41:22.549932+0000","last_active":"2024-04-02T14:44:00.239058+0000","last_peered":"2024-04-02T14:44:00.239058+0000","last_clean":"2024-04-02T14:44:00.239058+0000","last_became_active":"2024-04-02T14:41:22.549231+0000","last_became_peered":"2024-04-02T14:41:22.549231+0000","last_unstale":"2024-04-02T14:44:00.239058+0000","last_undegraded":"2024-04-02T14:44:00.239058+0000","last_fullsized":"2024-04-02T14:44:00.239058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:01.485343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.362610+0000","last_change":"2024-04-02T14:41:22.544712+0000","last_active":"2024-04-02T14:44:00.362610+0000","last_peered":"2024-04-02T14:44:00.362610+0000","last_clean":"2024-04-02T14:44:00.362610+0000","last_became_active":"2024-04-02T14:41:22.544043+0000","last_became_peered":"2024-04-02T14:41:22.544043+0000","last_unstale":"2024-04-02T14:44:00.362610+0000","last_undegraded":"2024-04-02T14:44:00.362610+0000","last_fullsized":"2024-04-02T14:44:00.362610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:07.931377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.398079+0000","last_change":"2024-04-02T14:41:22.542364+0000","last_active":"2024-04-02T14:44:00.398079+0000","last_peered":"2024-04-02T14:44:00.398079+0000","last_clean":"2024-04-02T14:44:00.398079+0000","last_became_active":"2024-04-02T14:41:22.541929+0000","last_became_peered":"2024-04-02T14:41:22.541929+0000","last_unstale":"2024-04-02T14:44:00.398079+0000","last_undegraded":"2024-04-02T14:44:00.398079+0000","last_fullsized":"2024-04-02T14:44:00.398079+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:45.716132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T14:44:00.396257+0000","last_change":"2024-04-02T14:41:22.525295+0000","last_active":"2024-04-02T14:44:00.396257+0000","last_peered":"2024-04-02T14:44:00.396257+0000","last_clean":"2024-04-02T14:44:00.396257+0000","last_became_active":"2024-04-02T14:41:22.524484+0000","last_became_peered":"2024-04-02T14:41:22.524484+0000","last_unstale":"2024-04-02T14:44:00.396257+0000","last_undegraded":"2024-04-02T14:44:00.396257+0000","last_fullsized":"2024-04-02T14:44:00.396257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:41:21.448149+0000","last_clean_scrub_stamp":"2024-04-02T14:41:21.448149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:01.819082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597589,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1637,"num_read_kb":932,"num_write":367,"num_write_kb":1458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1792767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285612,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":478089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547249,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28988,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716164,"statfs":{"total":95995035648,"available":95965351936,"internally_reserved":0,"allocated":1835008,"data_stored":1264973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710775,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36720,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708432,"statfs":{"total":95995035648,"available":95957434368,"internally_reserved":0,"allocated":1413120,"data_stored":871235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972414,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36716,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708436,"statfs":{"total":95995035648,"available":95957438464,"internally_reserved":0,"allocated":1409024,"data_stored":871613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1613824,"data_stored":1071056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430280,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":468838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":542904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1024000,"data_stored":479075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":542893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117080,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":477561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855445,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1011712,"data_stored":479533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691918,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1630208,"data_stored":1071246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":198373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:44:07.363 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:44:07.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:07.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:07.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.367 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi062 to 176.8M 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: Unable to set osd_memory_target on smithi062 to 185396019: error parsing value: Value '185396019' is below minimum 939524096 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: Updating smithi113:/etc/ceph/ceph.conf 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: Updating smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:44:07.368 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: Updating smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:44:07.369 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:07 smithi113 ceph-mon[36135]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:44:07.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:07.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:07.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi062 to 176.8M 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: Unable to set osd_memory_target on smithi062 to 185396019: error parsing value: Value '185396019' is below minimum 939524096 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: Updating smithi113:/etc/ceph/ceph.conf 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: Updating smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:44:07.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: Updating smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:44:07.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:07 smithi204 ceph-mon[35948]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:44:07.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.590+0000 7f8a51400640 1 -- 172.21.15.62:0/38793195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4c102d30 msgr2=0x7f8a4c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:07.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.590+0000 7f8a51400640 1 --2- 172.21.15.62:0/38793195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4c102d30 0x7f8a4c1031b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009900 tx=0x7f8a3802bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:07.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.590+0000 7f8a51400640 1 -- 172.21.15.62:0/38793195 shutdown_connections 2024-04-02T14:44:07.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.590+0000 7f8a51400640 1 --2- 172.21.15.62:0/38793195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a4c1036f0 0x7f8a4c109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.590+0000 7f8a51400640 1 --2- 172.21.15.62:0/38793195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4c102d30 0x7f8a4c1031b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.590+0000 7f8a51400640 1 --2- 172.21.15.62:0/38793195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c101b40 0x7f8a4c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.590+0000 7f8a51400640 1 -- 172.21.15.62:0/38793195 >> 172.21.15.62:0/38793195 conn(0x7f8a4c0fd0b0 msgr2=0x7f8a4c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:07.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.590+0000 7f8a51400640 1 -- 172.21.15.62:0/38793195 shutdown_connections 2024-04-02T14:44:07.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.590+0000 7f8a51400640 1 -- 172.21.15.62:0/38793195 wait complete. 2024-04-02T14:44:07.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.591+0000 7f8a51400640 1 Processor -- start 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.591+0000 7f8a51400640 1 -- start start 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.591+0000 7f8a51400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c101b40 0x7f8a4c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.591+0000 7f8a51400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4c102d30 0x7f8a4c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.591+0000 7f8a51400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a4c1036f0 0x7f8a4c19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.591+0000 7f8a51400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10d8a0 con 0x7f8a4c101b40 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.591+0000 7f8a51400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10d720 con 0x7f8a4c1036f0 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.591+0000 7f8a51400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10da20 con 0x7f8a4c102d30 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.592+0000 7f8a4a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c101b40 0x7f8a4c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.592+0000 7f8a4a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c101b40 0x7f8a4c197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35788/0 (socket says 172.21.15.62:35788) 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.592+0000 7f8a4a800640 1 -- 172.21.15.62:0/1828248251 learned_addr learned my addr 172.21.15.62:0/1828248251 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.592+0000 7f8a4b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a4c1036f0 0x7f8a4c19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:07.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.592+0000 7f8a49e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4c102d30 0x7f8a4c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:07.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.592+0000 7f8a4b200640 1 -- 172.21.15.62:0/1828248251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4c102d30 msgr2=0x7f8a4c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:07.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.592+0000 7f8a4b200640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4c102d30 0x7f8a4c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.592+0000 7f8a4b200640 1 -- 172.21.15.62:0/1828248251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c101b40 msgr2=0x7f8a4c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:07.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.592+0000 7f8a4b200640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c101b40 0x7f8a4c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.592+0000 7f8a4b200640 1 -- 172.21.15.62:0/1828248251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4c19fb00 con 0x7f8a4c1036f0 2024-04-02T14:44:07.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.593+0000 7f8a4b200640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a4c1036f0 0x7f8a4c19f490 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009fd0 tx=0x7f8a4400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:07.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.593+0000 7f8a43400640 1 -- 172.21.15.62:0/1828248251 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400ce20 con 0x7f8a4c1036f0 2024-04-02T14:44:07.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.593+0000 7f8a43400640 1 -- 172.21.15.62:0/1828248251 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a44004b20 con 0x7f8a4c1036f0 2024-04-02T14:44:07.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.593+0000 7f8a43400640 1 -- 172.21.15.62:0/1828248251 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44010790 con 0x7f8a4c1036f0 2024-04-02T14:44:07.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.593+0000 7f8a51400640 1 -- 172.21.15.62:0/1828248251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4c19fdf0 con 0x7f8a4c1036f0 2024-04-02T14:44:07.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.593+0000 7f8a51400640 1 -- 172.21.15.62:0/1828248251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4c1a0260 con 0x7f8a4c1036f0 2024-04-02T14:44:07.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.594+0000 7f8a51400640 1 -- 172.21.15.62:0/1828248251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a4c108ee0 con 0x7f8a4c1036f0 2024-04-02T14:44:07.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.597+0000 7f8a43400640 1 -- 172.21.15.62:0/1828248251 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f8a44005020 con 0x7f8a4c1036f0 2024-04-02T14:44:07.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.598+0000 7f8a43400640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a300a3e40 0x7f8a300a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:07.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.598+0000 7f8a43400640 1 -- 172.21.15.62:0/1828248251 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8a440d11a0 con 0x7f8a4c1036f0 2024-04-02T14:44:07.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.598+0000 7f8a4a800640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a300a3e40 0x7f8a300a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:07.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.599+0000 7f8a4a800640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a300a3e40 0x7f8a300a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0058c0 tx=0x7f8a3c005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:07.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.602+0000 7f8a43400640 1 -- 172.21.15.62:0/1828248251 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8a4409d640 con 0x7f8a4c1036f0 2024-04-02T14:44:07.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.814+0000 7f8a51400640 1 -- 172.21.15.62:0/1828248251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a4c02d590 con 0x7f8a4c1036f0 2024-04-02T14:44:07.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.815+0000 7f8a43400640 1 -- 172.21.15.62:0/1828248251 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22957 (secure 0 0 0) 0x7f8a4409cf40 con 0x7f8a4c1036f0 2024-04-02T14:44:07.813 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:44:07.813 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":91,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:44:00.087363+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/129789951":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/79979639":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/566820061":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3400451184":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:44:07.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.818+0000 7f8a51400640 1 -- 172.21.15.62:0/1828248251 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a300a3e40 msgr2=0x7f8a300a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:07.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.818+0000 7f8a51400640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a300a3e40 0x7f8a300a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0058c0 tx=0x7f8a3c005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:07.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.818+0000 7f8a51400640 1 -- 172.21.15.62:0/1828248251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a4c1036f0 msgr2=0x7f8a4c19f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:07.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.818+0000 7f8a51400640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a4c1036f0 0x7f8a4c19f490 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009fd0 tx=0x7f8a4400ef10 comp rx=0 tx=0).stop 2024-04-02T14:44:07.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.818+0000 7f8a51400640 1 -- 172.21.15.62:0/1828248251 shutdown_connections 2024-04-02T14:44:07.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.818+0000 7f8a51400640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a300a3e40 0x7f8a300a6300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.818+0000 7f8a51400640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a4c1036f0 0x7f8a4c19f490 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.818+0000 7f8a51400640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4c102d30 0x7f8a4c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.818+0000 7f8a51400640 1 --2- 172.21.15.62:0/1828248251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c101b40 0x7f8a4c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.818+0000 7f8a51400640 1 -- 172.21.15.62:0/1828248251 >> 172.21.15.62:0/1828248251 conn(0x7f8a4c0fd0b0 msgr2=0x7f8a4c1060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.819+0000 7f8a51400640 1 -- 172.21.15.62:0/1828248251 shutdown_connections 2024-04-02T14:44:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:07.819+0000 7f8a51400640 1 -- 172.21.15.62:0/1828248251 wait complete. 2024-04-02T14:44:07.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T14:41:20.030393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T14:41:20.917405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:44:07.838 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:44:08.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.026+0000 7f1694c00640 1 -- 172.21.15.62:0/905683878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1690104460 msgr2=0x7f1690110970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.026+0000 7f1694c00640 1 --2- 172.21.15.62:0/905683878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1690104460 0x7f1690110970 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1688009c10 tx=0x7f168802bef0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.026+0000 7f1694c00640 1 -- 172.21.15.62:0/905683878 shutdown_connections 2024-04-02T14:44:08.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.026+0000 7f1694c00640 1 --2- 172.21.15.62:0/905683878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1690104460 0x7f1690110970 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.026+0000 7f1694c00640 1 --2- 172.21.15.62:0/905683878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690103aa0 0x7f1690103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.026+0000 7f1694c00640 1 --2- 172.21.15.62:0/905683878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1690109ab0 0x7f1690109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.026+0000 7f1694c00640 1 -- 172.21.15.62:0/905683878 >> 172.21.15.62:0/905683878 conn(0x7f16900ff570 msgr2=0x7f1690101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:08.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.026+0000 7f1694c00640 1 -- 172.21.15.62:0/905683878 shutdown_connections 2024-04-02T14:44:08.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.026+0000 7f1694c00640 1 -- 172.21.15.62:0/905683878 wait complete. 2024-04-02T14:44:08.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.028+0000 7f1694c00640 1 Processor -- start 2024-04-02T14:44:08.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.028+0000 7f1694c00640 1 -- start start 2024-04-02T14:44:08.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.029+0000 7f1694c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1690103aa0 0x7f169019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.029+0000 7f1694c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690104460 0x7f169019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.029+0000 7f1694c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1690109ab0 0x7f169019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.029+0000 7f1694c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1690113fb0 con 0x7f1690109ab0 2024-04-02T14:44:08.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.029+0000 7f1694c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1690113e30 con 0x7f1690104460 2024-04-02T14:44:08.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.029+0000 7f1694c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1690114130 con 0x7f1690103aa0 2024-04-02T14:44:08.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.029+0000 7f1686a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690104460 0x7f169019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.029+0000 7f168fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1690109ab0 0x7f169019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.029+0000 7f1686a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690104460 0x7f169019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41354/0 (socket says 172.21.15.62:41354) 2024-04-02T14:44:08.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.029+0000 7f1686a00640 1 -- 172.21.15.62:0/2477766454 learned_addr learned my addr 172.21.15.62:0/2477766454 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:08.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.029+0000 7f168f400640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1690103aa0 0x7f169019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.030+0000 7f1686a00640 1 -- 172.21.15.62:0/2477766454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1690103aa0 msgr2=0x7f169019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.030+0000 7f1686a00640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1690103aa0 0x7f169019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.030+0000 7f1686a00640 1 -- 172.21.15.62:0/2477766454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1690109ab0 msgr2=0x7f169019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.030+0000 7f1686a00640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1690109ab0 0x7f169019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.030+0000 7f1686a00640 1 -- 172.21.15.62:0/2477766454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169019f480 con 0x7f1690104460 2024-04-02T14:44:08.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.030+0000 7f1686a00640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690104460 0x7f169019a9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f167800c600 tx=0x7f167800cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:08.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.030+0000 7f168cc00640 1 -- 172.21.15.62:0/2477766454 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678023070 con 0x7f1690104460 2024-04-02T14:44:08.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.030+0000 7f1694c00640 1 -- 172.21.15.62:0/2477766454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169019f770 con 0x7f1690104460 2024-04-02T14:44:08.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.030+0000 7f168cc00640 1 -- 172.21.15.62:0/2477766454 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f167801e930 con 0x7f1690104460 2024-04-02T14:44:08.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.030+0000 7f168cc00640 1 -- 172.21.15.62:0/2477766454 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167800bbf0 con 0x7f1690104460 2024-04-02T14:44:08.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.031+0000 7f1694c00640 1 -- 172.21.15.62:0/2477766454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1690075f40 con 0x7f1690104460 2024-04-02T14:44:08.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.032+0000 7f1694c00640 1 -- 172.21.15.62:0/2477766454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1690068810 con 0x7f1690104460 2024-04-02T14:44:08.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.033+0000 7f168cc00640 1 -- 172.21.15.62:0/2477766454 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f1678026070 con 0x7f1690104460 2024-04-02T14:44:08.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.036+0000 7f168cc00640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16740a3ec0 0x7f16740a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.036+0000 7f168cc00640 1 -- 172.21.15.62:0/2477766454 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f16780d8b60 con 0x7f1690104460 2024-04-02T14:44:08.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.037+0000 7f168f400640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16740a3ec0 0x7f16740a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.037+0000 7f168f400640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16740a3ec0 0x7f16740a6380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f16800096c0 tx=0x7f1680009260 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:08.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.039+0000 7f168cc00640 1 -- 172.21.15.62:0/2477766454 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f16780a4950 con 0x7f1690104460 2024-04-02T14:44:08.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.272+0000 7f1694c00640 1 -- 172.21.15.62:0/2477766454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1690076260 con 0x7f1690104460 2024-04-02T14:44:08.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.272+0000 7f168cc00640 1 -- 172.21.15.62:0/2477766454 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f16780a8c50 con 0x7f1690104460 2024-04-02T14:44:08.270 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:44:08.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 -- 172.21.15.62:0/2477766454 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16740a3ec0 msgr2=0x7f16740a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16740a3ec0 0x7f16740a6380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f16800096c0 tx=0x7f1680009260 comp rx=0 tx=0).stop 2024-04-02T14:44:08.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 -- 172.21.15.62:0/2477766454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690104460 msgr2=0x7f169019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690104460 0x7f169019a9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f167800c600 tx=0x7f167800cac0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 -- 172.21.15.62:0/2477766454 shutdown_connections 2024-04-02T14:44:08.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16740a3ec0 0x7f16740a6380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1690109ab0 0x7f169019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690104460 0x7f169019a9d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 --2- 172.21.15.62:0/2477766454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1690103aa0 0x7f169019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 -- 172.21.15.62:0/2477766454 >> 172.21.15.62:0/2477766454 conn(0x7f16900ff570 msgr2=0x7f16900ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:08.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 -- 172.21.15.62:0/2477766454 shutdown_connections 2024-04-02T14:44:08.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.275+0000 7f1694c00640 1 -- 172.21.15.62:0/2477766454 wait complete. 2024-04-02T14:44:08.290 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T14:44:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.480+0000 7f9046e00640 1 -- 172.21.15.62:0/1019002448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9040103450 msgr2=0x7f904010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.480+0000 7f9046e00640 1 --2- 172.21.15.62:0/1019002448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9040103450 0x7f904010b930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f902c009900 tx=0x7f902c02bcc0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.480+0000 7f9046e00640 1 -- 172.21.15.62:0/1019002448 shutdown_connections 2024-04-02T14:44:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.480+0000 7f9046e00640 1 --2- 172.21.15.62:0/1019002448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904010bf50 0x7f904010e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.480+0000 7f9046e00640 1 --2- 172.21.15.62:0/1019002448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9040103450 0x7f904010b930 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.480+0000 7f9046e00640 1 --2- 172.21.15.62:0/1019002448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040102b30 0x7f9040102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.480+0000 7f9046e00640 1 -- 172.21.15.62:0/1019002448 >> 172.21.15.62:0/1019002448 conn(0x7f90400fc350 msgr2=0x7f90400fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.480+0000 7f9046e00640 1 -- 172.21.15.62:0/1019002448 shutdown_connections 2024-04-02T14:44:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.480+0000 7f9046e00640 1 -- 172.21.15.62:0/1019002448 wait complete. 2024-04-02T14:44:08.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.481+0000 7f9046e00640 1 Processor -- start 2024-04-02T14:44:08.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.481+0000 7f9046e00640 1 -- start start 2024-04-02T14:44:08.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.482+0000 7f9046e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9040102b30 0x7f904019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.482+0000 7f9046e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040103450 0x7f904019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.482+0000 7f9046e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904010bf50 0x7f904019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.482+0000 7f9046e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9040111e70 con 0x7f904010bf50 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.482+0000 7f9046e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9040111cf0 con 0x7f9040102b30 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.482+0000 7f9046e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9040111ff0 con 0x7f9040103450 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.482+0000 7f9045a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9040102b30 0x7f904019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.482+0000 7f9045a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9040102b30 0x7f904019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41364/0 (socket says 172.21.15.62:41364) 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.482+0000 7f9046400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904010bf50 0x7f904019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.482+0000 7f9045a00640 1 -- 172.21.15.62:0/1561025303 learned_addr learned my addr 172.21.15.62:0/1561025303 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.482+0000 7f9045000640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040103450 0x7f904019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.483+0000 7f9045a00640 1 -- 172.21.15.62:0/1561025303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040103450 msgr2=0x7f904019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.483+0000 7f9045a00640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040103450 0x7f904019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.483+0000 7f9045a00640 1 -- 172.21.15.62:0/1561025303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904010bf50 msgr2=0x7f904019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.483+0000 7f9045a00640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904010bf50 0x7f904019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.483+0000 7f9045a00640 1 -- 172.21.15.62:0/1561025303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904019f510 con 0x7f9040102b30 2024-04-02T14:44:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.483+0000 7f9045a00640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9040102b30 0x7f904019a520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f903400d950 tx=0x7f903400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.483+0000 7f903e000640 1 -- 172.21.15.62:0/1561025303 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034017030 con 0x7f9040102b30 2024-04-02T14:44:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.483+0000 7f903e000640 1 -- 172.21.15.62:0/1561025303 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9034004020 con 0x7f9040102b30 2024-04-02T14:44:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.483+0000 7f9046e00640 1 -- 172.21.15.62:0/1561025303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904019f800 con 0x7f9040102b30 2024-04-02T14:44:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.483+0000 7f903e000640 1 -- 172.21.15.62:0/1561025303 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034004a00 con 0x7f9040102b30 2024-04-02T14:44:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.483+0000 7f9046e00640 1 -- 172.21.15.62:0/1561025303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90401a7090 con 0x7f9040102b30 2024-04-02T14:44:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.484+0000 7f9046e00640 1 -- 172.21.15.62:0/1561025303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9040068810 con 0x7f9040102b30 2024-04-02T14:44:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.487+0000 7f903e000640 1 -- 172.21.15.62:0/1561025303 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f90340041d0 con 0x7f9040102b30 2024-04-02T14:44:08.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.489+0000 7f903e000640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f90280a3ec0 0x7f90280a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.489+0000 7f903e000640 1 -- 172.21.15.62:0/1561025303 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f90340d0390 con 0x7f9040102b30 2024-04-02T14:44:08.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.489+0000 7f9045000640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f90280a3ec0 0x7f90280a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.490+0000 7f9045000640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f90280a3ec0 0x7f90280a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f902c006010 tx=0x7f902c005e40 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:08.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.493+0000 7f903e000640 1 -- 172.21.15.62:0/1561025303 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f903409c7d0 con 0x7f9040102b30 2024-04-02T14:44:08.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:44:08.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: Updating smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:44:08.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: from='client.25411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:44:08.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: Updating smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:44:08.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: Updating smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:44:08.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: from='client.25417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:44:08.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:44:08.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:08 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1828248251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:08.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:44:08.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: Updating smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:44:08.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: from='client.25411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:44:08.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: Updating smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:44:08.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: Updating smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:44:08.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: from='client.25417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:44:08.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:44:08.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:08 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1828248251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:08.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:44:08.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: Updating smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T14:44:08.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: from='client.25411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:44:08.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: Updating smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:44:08.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: Updating smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:44:08.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:08.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: from='client.25417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T14:44:08.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-04-02T14:44:08.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:08 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1828248251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:08.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.703+0000 7f9046e00640 1 -- 172.21.15.62:0/1561025303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90401a72c0 con 0x7f9040102b30 2024-04-02T14:44:08.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.704+0000 7f903e000640 1 -- 172.21.15.62:0/1561025303 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f903409c0d0 con 0x7f9040102b30 2024-04-02T14:44:08.701 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-02T14:44:08.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.706+0000 7f9046e00640 1 -- 172.21.15.62:0/1561025303 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f90280a3ec0 msgr2=0x7f90280a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.706+0000 7f9046e00640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f90280a3ec0 0x7f90280a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f902c006010 tx=0x7f902c005e40 comp rx=0 tx=0).stop 2024-04-02T14:44:08.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.706+0000 7f9046e00640 1 -- 172.21.15.62:0/1561025303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9040102b30 msgr2=0x7f904019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.706+0000 7f9046e00640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9040102b30 0x7f904019a520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f903400d950 tx=0x7f903400de10 comp rx=0 tx=0).stop 2024-04-02T14:44:08.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.707+0000 7f9046e00640 1 -- 172.21.15.62:0/1561025303 shutdown_connections 2024-04-02T14:44:08.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.707+0000 7f9046e00640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f90280a3ec0 0x7f90280a6380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.707+0000 7f9046e00640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904010bf50 0x7f904019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.707+0000 7f9046e00640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040103450 0x7f904019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.707+0000 7f9046e00640 1 --2- 172.21.15.62:0/1561025303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9040102b30 0x7f904019a520 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.707+0000 7f9046e00640 1 -- 172.21.15.62:0/1561025303 >> 172.21.15.62:0/1561025303 conn(0x7f90400fc350 msgr2=0x7f90400fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:08.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.707+0000 7f9046e00640 1 -- 172.21.15.62:0/1561025303 shutdown_connections 2024-04-02T14:44:08.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.707+0000 7f9046e00640 1 -- 172.21.15.62:0/1561025303 wait complete. 2024-04-02T14:44:08.724 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T14:44:08.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.942+0000 7fae6da00640 1 -- 172.21.15.62:0/914872122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae68102b90 msgr2=0x7fae68102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.942+0000 7fae6da00640 1 --2- 172.21.15.62:0/914872122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae68102b90 0x7fae68102f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fae58009900 tx=0x7fae5802bcc0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.942+0000 7fae6da00640 1 -- 172.21.15.62:0/914872122 shutdown_connections 2024-04-02T14:44:08.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.942+0000 7fae6da00640 1 --2- 172.21.15.62:0/914872122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae6810bf60 0x7fae6810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.942+0000 7fae6da00640 1 --2- 172.21.15.62:0/914872122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae681034b0 0x7fae6810b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.942+0000 7fae6da00640 1 --2- 172.21.15.62:0/914872122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae68102b90 0x7fae68102f70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.942+0000 7fae6da00640 1 -- 172.21.15.62:0/914872122 >> 172.21.15.62:0/914872122 conn(0x7fae680fc350 msgr2=0x7fae680fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:08.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.942+0000 7fae6da00640 1 -- 172.21.15.62:0/914872122 shutdown_connections 2024-04-02T14:44:08.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.942+0000 7fae6da00640 1 -- 172.21.15.62:0/914872122 wait complete. 2024-04-02T14:44:08.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.943+0000 7fae6da00640 1 Processor -- start 2024-04-02T14:44:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.943+0000 7fae6da00640 1 -- start start 2024-04-02T14:44:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.943+0000 7fae6da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae68102b90 0x7fae6819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.943+0000 7fae6da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae681034b0 0x7fae6819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae6da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae6810bf60 0x7fae6819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae6da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fae68111e60 con 0x7fae68102b90 2024-04-02T14:44:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae6da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fae68111ce0 con 0x7fae681034b0 2024-04-02T14:44:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae6da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae68111fe0 con 0x7fae6810bf60 2024-04-02T14:44:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae67e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae68102b90 0x7fae6819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae67e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae68102b90 0x7fae6819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35840/0 (socket says 172.21.15.62:35840) 2024-04-02T14:44:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae67e00640 1 -- 172.21.15.62:0/3721668920 learned_addr learned my addr 172.21.15.62:0/3721668920 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae67400640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae681034b0 0x7fae6819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae6d000640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae6810bf60 0x7fae6819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae67e00640 1 -- 172.21.15.62:0/3721668920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae6810bf60 msgr2=0x7fae6819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae67e00640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae6810bf60 0x7fae6819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae67e00640 1 -- 172.21.15.62:0/3721668920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae681034b0 msgr2=0x7fae6819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae67e00640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae681034b0 0x7fae6819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae67e00640 1 -- 172.21.15.62:0/3721668920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6819f520 con 0x7fae68102b90 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.944+0000 7fae67400640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae681034b0 0x7fae6819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.945+0000 7fae67e00640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae68102b90 0x7fae6819a530 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fae58005940 tx=0x7fae58004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.945+0000 7fae64c00640 1 -- 172.21.15.62:0/3721668920 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58043020 con 0x7fae68102b90 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.945+0000 7fae64c00640 1 -- 172.21.15.62:0/3721668920 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae58004250 con 0x7fae68102b90 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.945+0000 7fae64c00640 1 -- 172.21.15.62:0/3721668920 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58003a80 con 0x7fae68102b90 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.945+0000 7fae6da00640 1 -- 172.21.15.62:0/3721668920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6819f7b0 con 0x7fae68102b90 2024-04-02T14:44:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.946+0000 7fae6da00640 1 -- 172.21.15.62:0/3721668920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae681a70a0 con 0x7fae68102b90 2024-04-02T14:44:08.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.947+0000 7fae6da00640 1 -- 172.21.15.62:0/3721668920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae68068810 con 0x7fae68102b90 2024-04-02T14:44:08.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.949+0000 7fae64c00640 1 -- 172.21.15.62:0/3721668920 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fae58046020 con 0x7fae68102b90 2024-04-02T14:44:08.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.950+0000 7fae64c00640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fae4c0a3ec0 0x7fae4c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:08.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.950+0000 7fae64c00640 1 -- 172.21.15.62:0/3721668920 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fae580f2160 con 0x7fae68102b90 2024-04-02T14:44:08.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.950+0000 7fae67400640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fae4c0a3ec0 0x7fae4c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:08.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.951+0000 7fae67400640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fae4c0a3ec0 0x7fae4c0a6380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae540095f0 tx=0x7fae54005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:08.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:08.956+0000 7fae64c00640 1 -- 172.21.15.62:0/3721668920 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fae580be710 con 0x7fae68102b90 2024-04-02T14:44:09.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.162+0000 7fae6da00640 1 -- 172.21.15.62:0/3721668920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fae681a72d0 con 0x7fae68102b90 2024-04-02T14:44:09.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.163+0000 7fae64c00640 1 -- 172.21.15.62:0/3721668920 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fae580be010 con 0x7fae68102b90 2024-04-02T14:44:09.160 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-02T14:44:09.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.166+0000 7fae6da00640 1 -- 172.21.15.62:0/3721668920 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fae4c0a3ec0 msgr2=0x7fae4c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:09.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.166+0000 7fae6da00640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fae4c0a3ec0 0x7fae4c0a6380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae540095f0 tx=0x7fae54005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.166+0000 7fae6da00640 1 -- 172.21.15.62:0/3721668920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae68102b90 msgr2=0x7fae6819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.166+0000 7fae6da00640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae68102b90 0x7fae6819a530 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fae58005940 tx=0x7fae58004970 comp rx=0 tx=0).stop 2024-04-02T14:44:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.166+0000 7fae6da00640 1 -- 172.21.15.62:0/3721668920 shutdown_connections 2024-04-02T14:44:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.166+0000 7fae6da00640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fae4c0a3ec0 0x7fae4c0a6380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.166+0000 7fae6da00640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae6810bf60 0x7fae6819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.166+0000 7fae6da00640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae681034b0 0x7fae6819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.166+0000 7fae6da00640 1 --2- 172.21.15.62:0/3721668920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae68102b90 0x7fae6819a530 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.166+0000 7fae6da00640 1 -- 172.21.15.62:0/3721668920 >> 172.21.15.62:0/3721668920 conn(0x7fae680fc350 msgr2=0x7fae680fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.166+0000 7fae6da00640 1 -- 172.21.15.62:0/3721668920 shutdown_connections 2024-04-02T14:44:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.167+0000 7fae6da00640 1 -- 172.21.15.62:0/3721668920 wait complete. 2024-04-02T14:44:09.183 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:44:09.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.405+0000 7fe217e00640 1 -- 172.21.15.62:0/1027894299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe218104490 msgr2=0x7fe2181109a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:09.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.405+0000 7fe217e00640 1 --2- 172.21.15.62:0/1027894299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe218104490 0x7fe2181109a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe210009cd0 tx=0x7fe21002c0f0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.405+0000 7fe217e00640 1 -- 172.21.15.62:0/1027894299 shutdown_connections 2024-04-02T14:44:09.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.405+0000 7fe217e00640 1 --2- 172.21.15.62:0/1027894299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe218104490 0x7fe2181109a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.405+0000 7fe217e00640 1 --2- 172.21.15.62:0/1027894299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe218103ad0 0x7fe218103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.405+0000 7fe217e00640 1 --2- 172.21.15.62:0/1027894299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218109b70 0x7fe218109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.405+0000 7fe217e00640 1 -- 172.21.15.62:0/1027894299 >> 172.21.15.62:0/1027894299 conn(0x7fe2180ff5a0 msgr2=0x7fe2181019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:09.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.405+0000 7fe217e00640 1 -- 172.21.15.62:0/1027894299 shutdown_connections 2024-04-02T14:44:09.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.405+0000 7fe217e00640 1 -- 172.21.15.62:0/1027894299 wait complete. 2024-04-02T14:44:09.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.406+0000 7fe217e00640 1 Processor -- start 2024-04-02T14:44:09.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe217e00640 1 -- start start 2024-04-02T14:44:09.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe217e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe218103ad0 0x7fe21819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:09.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe217e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe218104490 0x7fe21819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:09.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe217e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218109b70 0x7fe21819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:09.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe217e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe218114010 con 0x7fe218103ad0 2024-04-02T14:44:09.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe217e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe218113e90 con 0x7fe218109b70 2024-04-02T14:44:09.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe217e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe218114190 con 0x7fe218104490 2024-04-02T14:44:09.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe216a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe218103ad0 0x7fe21819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:09.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe216a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe218103ad0 0x7fe21819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35858/0 (socket says 172.21.15.62:35858) 2024-04-02T14:44:09.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe216a00640 1 -- 172.21.15.62:0/858689191 learned_addr learned my addr 172.21.15.62:0/858689191 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:09.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe216000640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe218104490 0x7fe21819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:09.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.407+0000 7fe217400640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218109b70 0x7fe21819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:09.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.408+0000 7fe216a00640 1 -- 172.21.15.62:0/858689191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe218104490 msgr2=0x7fe21819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:09.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.408+0000 7fe216a00640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe218104490 0x7fe21819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.408+0000 7fe216a00640 1 -- 172.21.15.62:0/858689191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218109b70 msgr2=0x7fe21819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:09.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.408+0000 7fe216a00640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218109b70 0x7fe21819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.408+0000 7fe216a00640 1 -- 172.21.15.62:0/858689191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21819f470 con 0x7fe218103ad0 2024-04-02T14:44:09.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.408+0000 7fe216000640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe218104490 0x7fe21819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:09.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.408+0000 7fe216a00640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe218103ad0 0x7fe21819a4d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe20800e9f0 tx=0x7fe20800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:09.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.408+0000 7fe20f400640 1 -- 172.21.15.62:0/858689191 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20800cd80 con 0x7fe218103ad0 2024-04-02T14:44:09.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.408+0000 7fe20f400640 1 -- 172.21.15.62:0/858689191 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe208005520 con 0x7fe218103ad0 2024-04-02T14:44:09.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.408+0000 7fe217e00640 1 -- 172.21.15.62:0/858689191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21819f760 con 0x7fe218103ad0 2024-04-02T14:44:09.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.409+0000 7fe20f400640 1 -- 172.21.15.62:0/858689191 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208010690 con 0x7fe218103ad0 2024-04-02T14:44:09.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.409+0000 7fe217e00640 1 -- 172.21.15.62:0/858689191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2181a6ff0 con 0x7fe218103ad0 2024-04-02T14:44:09.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.410+0000 7fe217e00640 1 -- 172.21.15.62:0/858689191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe218068c60 con 0x7fe218103ad0 2024-04-02T14:44:09.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.412+0000 7fe20f400640 1 -- 172.21.15.62:0/858689191 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fe2080046c0 con 0x7fe218103ad0 2024-04-02T14:44:09.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.414+0000 7fe20f400640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1fc0a3ec0 0x7fe1fc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:09.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.414+0000 7fe20f400640 1 -- 172.21.15.62:0/858689191 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fe2080d0da0 con 0x7fe218103ad0 2024-04-02T14:44:09.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.414+0000 7fe216000640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1fc0a3ec0 0x7fe1fc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:09.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.415+0000 7fe216000640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1fc0a3ec0 0x7fe1fc0a6380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe2181091b0 tx=0x7fe204005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:09.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.418+0000 7fe20f400640 1 -- 172.21.15.62:0/858689191 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe20809d1e0 con 0x7fe218103ad0 2024-04-02T14:44:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: Updating smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:44:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 880 B/s wr, 26 op/s 2024-04-02T14:44:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2477766454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:44:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1561025303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:44:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:44:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:44:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:44:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:09 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3721668920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:44:09.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: Updating smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:44:09.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 880 B/s wr, 26 op/s 2024-04-02T14:44:09.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2477766454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:44:09.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1561025303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:44:09.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:44:09.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:44:09.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:44:09.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:09 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3721668920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:44:09.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: Updating smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T14:44:09.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 880 B/s wr, 26 op/s 2024-04-02T14:44:09.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2477766454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:44:09.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1561025303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:44:09.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:44:09.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:44:09.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:44:09.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:44:09.618 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:09 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3721668920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:44:09.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.666+0000 7fe217e00640 1 -- 172.21.15.62:0/858689191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe21810c0a0 con 0x7fe218103ad0 2024-04-02T14:44:09.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.668+0000 7fe20f400640 1 -- 172.21.15.62:0/858689191 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fe20809cae0 con 0x7fe218103ad0 2024-04-02T14:44:09.666 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:44:09.666 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:44:00.377074+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584,24535,24533,24553,24565]},"id":1}]} 2024-04-02T14:44:09.667 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-04-02T14:44:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.671+0000 7fe217e00640 1 -- 172.21.15.62:0/858689191 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1fc0a3ec0 msgr2=0x7fe1fc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.671+0000 7fe217e00640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1fc0a3ec0 0x7fe1fc0a6380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe2181091b0 tx=0x7fe204005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.671+0000 7fe217e00640 1 -- 172.21.15.62:0/858689191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe218103ad0 msgr2=0x7fe21819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.671+0000 7fe217e00640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe218103ad0 0x7fe21819a4d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe20800e9f0 tx=0x7fe20800edf0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.671+0000 7fe217e00640 1 -- 172.21.15.62:0/858689191 shutdown_connections 2024-04-02T14:44:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.671+0000 7fe217e00640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1fc0a3ec0 0x7fe1fc0a6380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.671+0000 7fe217e00640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218109b70 0x7fe21819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.671+0000 7fe217e00640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe218104490 0x7fe21819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.671+0000 7fe217e00640 1 --2- 172.21.15.62:0/858689191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe218103ad0 0x7fe21819a4d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.671+0000 7fe217e00640 1 -- 172.21.15.62:0/858689191 >> 172.21.15.62:0/858689191 conn(0x7fe2180ff5a0 msgr2=0x7fe2180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:09.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.672+0000 7fe217e00640 1 -- 172.21.15.62:0/858689191 shutdown_connections 2024-04-02T14:44:09.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.672+0000 7fe217e00640 1 -- 172.21.15.62:0/858689191 wait complete. 2024-04-02T14:44:09.690 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:44:09.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.904+0000 7f48d1600640 1 -- 172.21.15.62:0/3879551950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48cc108130 msgr2=0x7f48cc108510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:09.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.904+0000 7f48d1600640 1 --2- 172.21.15.62:0/3879551950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48cc108130 0x7f48cc108510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009ca0 tx=0x7f48bc02c100 comp rx=0 tx=0).stop 2024-04-02T14:44:09.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.904+0000 7f48d1600640 1 -- 172.21.15.62:0/3879551950 shutdown_connections 2024-04-02T14:44:09.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.904+0000 7f48d1600640 1 --2- 172.21.15.62:0/3879551950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48cc10d200 0x7f48cc114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.904+0000 7f48d1600640 1 --2- 172.21.15.62:0/3879551950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48cc108ae0 0x7f48cc10cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.904+0000 7f48d1600640 1 --2- 172.21.15.62:0/3879551950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48cc108130 0x7f48cc108510 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.904+0000 7f48d1600640 1 -- 172.21.15.62:0/3879551950 >> 172.21.15.62:0/3879551950 conn(0x7f48cc077620 msgr2=0x7f48cc079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:09.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.904+0000 7f48d1600640 1 -- 172.21.15.62:0/3879551950 shutdown_connections 2024-04-02T14:44:09.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.904+0000 7f48d1600640 1 -- 172.21.15.62:0/3879551950 wait complete. 2024-04-02T14:44:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.905+0000 7f48d1600640 1 Processor -- start 2024-04-02T14:44:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.905+0000 7f48d1600640 1 -- start start 2024-04-02T14:44:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.906+0000 7f48d1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48cc108130 0x7f48cc1a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.906+0000 7f48d1600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48cc10d200 0x7f48cc1a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.906+0000 7f48d1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48cc19e690 0x7f48cc19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.906+0000 7f48d1600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1183f0 con 0x7f48cc19e690 2024-04-02T14:44:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.906+0000 7f48d1600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc118270 con 0x7f48cc10d200 2024-04-02T14:44:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.906+0000 7f48d1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc118570 con 0x7f48cc108130 2024-04-02T14:44:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.906+0000 7f48cbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48cc108130 0x7f48cc1a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.906+0000 7f48cb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48cc10d200 0x7f48cc1a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.906+0000 7f48d0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48cc19e690 0x7f48cc19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.906+0000 7f48cbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48cc108130 0x7f48cc1a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39374/0 (socket says 172.21.15.62:39374) 2024-04-02T14:44:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.906+0000 7f48cbe00640 1 -- 172.21.15.62:0/2377858954 learned_addr learned my addr 172.21.15.62:0/2377858954 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.907+0000 7f48cbe00640 1 -- 172.21.15.62:0/2377858954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48cc10d200 msgr2=0x7f48cc1a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.907+0000 7f48cbe00640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48cc10d200 0x7f48cc1a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.907+0000 7f48cbe00640 1 -- 172.21.15.62:0/2377858954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48cc19e690 msgr2=0x7f48cc19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.907+0000 7f48cbe00640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48cc19e690 0x7f48cc19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.907+0000 7f48cbe00640 1 -- 172.21.15.62:0/2377858954 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc19f290 con 0x7f48cc108130 2024-04-02T14:44:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.907+0000 7f48cb400640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48cc10d200 0x7f48cc1a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:09.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.907+0000 7f48cbe00640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48cc108130 0x7f48cc1a45d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48bc005e30 tx=0x7f48bc0344c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:09.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.907+0000 7f48c8c00640 1 -- 172.21.15.62:0/2377858954 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc039040 con 0x7f48cc108130 2024-04-02T14:44:09.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.908+0000 7f48d1600640 1 -- 172.21.15.62:0/2377858954 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc19f520 con 0x7f48cc108130 2024-04-02T14:44:09.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.908+0000 7f48c8c00640 1 -- 172.21.15.62:0/2377858954 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f48bc044070 con 0x7f48cc108130 2024-04-02T14:44:09.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.908+0000 7f48c8c00640 1 -- 172.21.15.62:0/2377858954 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc004420 con 0x7f48cc108130 2024-04-02T14:44:09.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.908+0000 7f48d1600640 1 -- 172.21.15.62:0/2377858954 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc1ab6b0 con 0x7f48cc108130 2024-04-02T14:44:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.909+0000 7f48d1600640 1 -- 172.21.15.62:0/2377858954 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc109600 con 0x7f48cc108130 2024-04-02T14:44:09.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.911+0000 7f48c8c00640 1 -- 172.21.15.62:0/2377858954 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f48bc032840 con 0x7f48cc108130 2024-04-02T14:44:09.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.912+0000 7f48c8c00640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f48b00a3e40 0x7f48b00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:09.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.912+0000 7f48c8c00640 1 -- 172.21.15.62:0/2377858954 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f48bc0f3840 con 0x7f48cc108130 2024-04-02T14:44:09.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.913+0000 7f48cb400640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f48b00a3e40 0x7f48b00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:09.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.913+0000 7f48cb400640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f48b00a3e40 0x7f48b00a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48b40095f0 tx=0x7f48b4005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:09.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:09.914+0000 7f48c8c00640 1 -- 172.21.15.62:0/2377858954 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f48bc0bfd90 con 0x7f48cc108130 2024-04-02T14:44:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.123+0000 7f48d1600640 1 -- 172.21.15.62:0/2377858954 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48cc110430 con 0x7f48cc108130 2024-04-02T14:44:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.125+0000 7f48c8c00640 1 -- 172.21.15.62:0/2377858954 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22957 (secure 0 0 0) 0x7f48bc0bf690 con 0x7f48cc108130 2024-04-02T14:44:10.125 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:44:10.125 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":91,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:44:00.087363+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/129789951":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/79979639":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/566820061":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3400451184":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:44:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.128+0000 7f48d1600640 1 -- 172.21.15.62:0/2377858954 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f48b00a3e40 msgr2=0x7f48b00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.128+0000 7f48d1600640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f48b00a3e40 0x7f48b00a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48b40095f0 tx=0x7f48b4005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.128+0000 7f48d1600640 1 -- 172.21.15.62:0/2377858954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48cc108130 msgr2=0x7f48cc1a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.128+0000 7f48d1600640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48cc108130 0x7f48cc1a45d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48bc005e30 tx=0x7f48bc0344c0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.128+0000 7f48d1600640 1 -- 172.21.15.62:0/2377858954 shutdown_connections 2024-04-02T14:44:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.128+0000 7f48d1600640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f48b00a3e40 0x7f48b00a6300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.128+0000 7f48d1600640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48cc19e690 0x7f48cc19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.128+0000 7f48d1600640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48cc10d200 0x7f48cc1a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.128+0000 7f48d1600640 1 --2- 172.21.15.62:0/2377858954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48cc108130 0x7f48cc1a45d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.128+0000 7f48d1600640 1 -- 172.21.15.62:0/2377858954 >> 172.21.15.62:0/2377858954 conn(0x7f48cc077620 msgr2=0x7f48cc0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.129+0000 7f48d1600640 1 -- 172.21.15.62:0/2377858954 shutdown_connections 2024-04-02T14:44:10.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.129+0000 7f48d1600640 1 -- 172.21.15.62:0/2377858954 wait complete. 2024-04-02T14:44:10.148 DEBUG:tasks.fwd_scrub:Yielding 2024-04-02T14:44:10.148 INFO:teuthology.run_tasks:Running task exec... 2024-04-02T14:44:10.161 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T14:44:10.161 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-04-02T14:44:10.161 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-02T14:44:10.162 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-02T14:44:10.162 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:44:10.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:44:10.162 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:44:10.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.378+0000 7fb8df200640 1 -- 172.21.15.62:0/35629999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d81048d0 msgr2=0x7fb8d8108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.378+0000 7fb8df200640 1 --2- 172.21.15.62:0/35629999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d81048d0 0x7fb8d8108910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009900 tx=0x7fb8c402bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:10.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.378+0000 7fdc2ba00640 1 -- 172.21.15.62:0/3121404403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc24107aa0 msgr2=0x7fdc24107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.378+0000 7fdc2ba00640 1 --2- 172.21.15.62:0/3121404403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc24107aa0 0x7fdc24107e80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009930 tx=0x7fdc1802bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:10.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fb8df200640 1 -- 172.21.15.62:0/35629999 shutdown_connections 2024-04-02T14:44:10.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fb8df200640 1 --2- 172.21.15.62:0/35629999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d8108ff0 0x7fb8d8110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fb8df200640 1 --2- 172.21.15.62:0/35629999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d81048d0 0x7fb8d8108910 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fb8df200640 1 --2- 172.21.15.62:0/35629999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8d8103f20 0x7fb8d8104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fb8df200640 1 -- 172.21.15.62:0/35629999 >> 172.21.15.62:0/35629999 conn(0x7fb8d80ff780 msgr2=0x7fb8d8101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fdc2ba00640 1 -- 172.21.15.62:0/3121404403 shutdown_connections 2024-04-02T14:44:10.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fdc2ba00640 1 --2- 172.21.15.62:0/3121404403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc24102450 0x7fdc2410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fdc2ba00640 1 --2- 172.21.15.62:0/3121404403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc24101a90 0x7fdc24101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fdc2ba00640 1 --2- 172.21.15.62:0/3121404403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc24107aa0 0x7fdc24107e80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fdc2ba00640 1 -- 172.21.15.62:0/3121404403 >> 172.21.15.62:0/3121404403 conn(0x7fdc24078f00 msgr2=0x7fdc24079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fb8df200640 1 -- 172.21.15.62:0/35629999 shutdown_connections 2024-04-02T14:44:10.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fdc2ba00640 1 -- 172.21.15.62:0/3121404403 shutdown_connections 2024-04-02T14:44:10.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fdc2ba00640 1 -- 172.21.15.62:0/3121404403 wait complete. 2024-04-02T14:44:10.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.379+0000 7fb8df200640 1 -- 172.21.15.62:0/35629999 wait complete. 2024-04-02T14:44:10.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fb8df200640 1 Processor -- start 2024-04-02T14:44:10.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fdc2ba00640 1 Processor -- start 2024-04-02T14:44:10.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fb8df200640 1 -- start start 2024-04-02T14:44:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fdc2ba00640 1 -- start start 2024-04-02T14:44:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fdc2ba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc24101a90 0x7fdc2419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fdc2ba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc24102450 0x7fdc2419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fb8df200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d8103f20 0x7fb8d819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fdc2ba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc24107aa0 0x7fdc2419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fdc2ba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24111f80 con 0x7fdc24107aa0 2024-04-02T14:44:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fdc2ba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24111e00 con 0x7fdc24101a90 2024-04-02T14:44:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fdc2ba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24112100 con 0x7fdc24102450 2024-04-02T14:44:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fb8df200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8108ff0 0x7fb8d819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fb8df200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8d819eed0 0x7fb8d819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fb8df200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8114150 con 0x7fb8d819eed0 2024-04-02T14:44:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fb8df200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8113fd0 con 0x7fb8d8103f20 2024-04-02T14:44:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.380+0000 7fb8df200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81142d0 con 0x7fb8d8108ff0 2024-04-02T14:44:10.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc29000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc24101a90 0x7fdc2419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc29a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc24107aa0 0x7fdc2419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc24102450 0x7fdc2419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8d7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d8103f20 0x7fb8d819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8dd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8d819eed0 0x7fb8d819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8d7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8108ff0 0x7fb8d819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc29000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc24101a90 0x7fdc2419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41454/0 (socket says 172.21.15.62:41454) 2024-04-02T14:44:10.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc24102450 0x7fdc2419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39414/0 (socket says 172.21.15.62:39414) 2024-04-02T14:44:10.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc29000640 1 -- 172.21.15.62:0/683225792 learned_addr learned my addr 172.21.15.62:0/683225792 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:10.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc29a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc24107aa0 0x7fdc2419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35894/0 (socket says 172.21.15.62:35894) 2024-04-02T14:44:10.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8d7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d8103f20 0x7fb8d819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41468/0 (socket says 172.21.15.62:41468) 2024-04-02T14:44:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8d7e00640 1 -- 172.21.15.62:0/110240059 learned_addr learned my addr 172.21.15.62:0/110240059 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8d7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8108ff0 0x7fb8d819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39416/0 (socket says 172.21.15.62:39416) 2024-04-02T14:44:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8dd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8d819eed0 0x7fb8d819f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35902/0 (socket says 172.21.15.62:35902) 2024-04-02T14:44:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc29000640 1 -- 172.21.15.62:0/683225792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc24102450 msgr2=0x7fdc2419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc29000640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc24102450 0x7fdc2419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc29000640 1 -- 172.21.15.62:0/683225792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc24107aa0 msgr2=0x7fdc2419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc29000640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc24107aa0 0x7fdc2419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fdc29000640 1 -- 172.21.15.62:0/683225792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2419f660 con 0x7fdc24101a90 2024-04-02T14:44:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8d7e00640 1 -- 172.21.15.62:0/110240059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8108ff0 msgr2=0x7fb8d819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8d7e00640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8108ff0 0x7fb8d819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8d7e00640 1 -- 172.21.15.62:0/110240059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8d819eed0 msgr2=0x7fb8d819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8d7e00640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8d819eed0 0x7fb8d819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.381+0000 7fb8d7e00640 1 -- 172.21.15.62:0/110240059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d819f9c0 con 0x7fb8d8103f20 2024-04-02T14:44:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fb8d7e00640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d8103f20 0x7fb8d819a610 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800ea60 tx=0x7fb8c800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fdc29000640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc24101a90 0x7fdc2419a6c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009900 tx=0x7fdc1802dc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fb8d4c00640 1 -- 172.21.15.62:0/110240059 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c800cde0 con 0x7fb8d8103f20 2024-04-02T14:44:10.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fb8d4c00640 1 -- 172.21.15.62:0/110240059 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb8c80035f0 con 0x7fb8d8103f20 2024-04-02T14:44:10.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fb8df200640 1 -- 172.21.15.62:0/110240059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d819fcb0 con 0x7fb8d8103f20 2024-04-02T14:44:10.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fb8d4c00640 1 -- 172.21.15.62:0/110240059 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8010720 con 0x7fb8d8103f20 2024-04-02T14:44:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fdc21600640 1 -- 172.21.15.62:0/683225792 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18043020 con 0x7fdc24101a90 2024-04-02T14:44:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fdc21600640 1 -- 172.21.15.62:0/683225792 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc18004250 con 0x7fdc24101a90 2024-04-02T14:44:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fdc21600640 1 -- 172.21.15.62:0/683225792 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc180039f0 con 0x7fdc24101a90 2024-04-02T14:44:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fdc2ba00640 1 -- 172.21.15.62:0/683225792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2419f8f0 con 0x7fdc24101a90 2024-04-02T14:44:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fb8df200640 1 -- 172.21.15.62:0/110240059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d81a7470 con 0x7fb8d8103f20 2024-04-02T14:44:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.382+0000 7fdc2ba00640 1 -- 172.21.15.62:0/683225792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc241a71e0 con 0x7fdc24101a90 2024-04-02T14:44:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.383+0000 7fb8df200640 1 -- 172.21.15.62:0/110240059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d804f860 con 0x7fb8d8103f20 2024-04-02T14:44:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.386+0000 7fdc21600640 1 -- 172.21.15.62:0/683225792 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fdc18046020 con 0x7fdc24101a90 2024-04-02T14:44:10.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.387+0000 7fb8d4c00640 1 -- 172.21.15.62:0/110240059 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fb8c80037a0 con 0x7fb8d8103f20 2024-04-02T14:44:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.387+0000 7fdc2e81d740 1 Processor -- start 2024-04-02T14:44:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.387+0000 7fdc2e81d740 1 -- start start 2024-04-02T14:44:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.387+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1014030 0x5610b1012d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.387+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 0x5610b10137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.387+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b101c6c0 0x5610b101eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.387+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5610b0f77310 con 0x5610b101c6c0 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.387+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5610b0f1f690 con 0x5610b1013350 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.387+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5610b0f732c0 con 0x5610b1014030 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc29000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 0x5610b10137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc21600640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fdc0c0a3ec0 0x7fdc0c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc29a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1014030 0x5610b1012d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc29a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1014030 0x5610b1012d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39432/0 (socket says 172.21.15.62:39432) 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc29a00640 1 -- 172.21.15.62:0/2471197746 learned_addr learned my addr 172.21.15.62:0/2471197746 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc23e00640 1 --2- 172.21.15.62:0/2471197746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b101c6c0 0x5610b101eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc21600640 1 -- 172.21.15.62:0/683225792 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fdc180f29c0 con 0x7fdc24101a90 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc29a00640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fdc0c0a3ec0 0x7fdc0c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc29000640 1 -- 172.21.15.62:0/2471197746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1014030 msgr2=0x5610b1012d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc29000640 1 --2- 172.21.15.62:0/2471197746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1014030 0x5610b1012d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc29000640 1 -- 172.21.15.62:0/2471197746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b101c6c0 msgr2=0x5610b101eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc29000640 1 --2- 172.21.15.62:0/2471197746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b101c6c0 0x5610b101eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.388+0000 7fdc29000640 1 -- 172.21.15.62:0/2471197746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b0d537e0 con 0x5610b1013350 2024-04-02T14:44:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.389+0000 7fdc29000640 1 --2- 172.21.15.62:0/2471197746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 0x5610b10137d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdc180f00f0 tx=0x7fdc18009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2b6da013a83603 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.389+0000 7fb8d4c00640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb8bc0a3ee0 0x7fb8bc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.389+0000 7fb8d4c00640 1 -- 172.21.15.62:0/110240059 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fb8c80d0dd0 con 0x7fb8d8103f20 2024-04-02T14:44:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.389+0000 7fb8d7400640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb8bc0a3ee0 0x7fb8bc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.389+0000 7fdc29a00640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fdc0c0a3ec0 0x7fdc0c0a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005f40 tx=0x7fdc1c004ab0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.390+0000 7fb8d7400640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb8bc0a3ee0 0x7fb8bc0a63a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4000c00 tx=0x7fb8c4005fd0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.392+0000 7fdc16000640 1 -- 172.21.15.62:0/2471197746 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc180be1d0 con 0x5610b1013350 2024-04-02T14:44:10.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.392+0000 7fdc16000640 1 -- 172.21.15.62:0/2471197746 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc180bef70 con 0x5610b1013350 2024-04-02T14:44:10.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.392+0000 7fdc16000640 1 -- 172.21.15.62:0/2471197746 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc180c7050 con 0x5610b1013350 2024-04-02T14:44:10.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.393+0000 7fb8d4c00640 1 -- 172.21.15.62:0/110240059 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb8c809d210 con 0x7fb8d8103f20 2024-04-02T14:44:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.393+0000 7fdc2e81d740 1 -- 172.21.15.62:0/2471197746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 msgr2=0x5610b10137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.393+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/2471197746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 0x5610b10137d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdc180f00f0 tx=0x7fdc18009260 comp rx=0 tx=0).stop 2024-04-02T14:44:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.393+0000 7fdc2e81d740 1 -- 172.21.15.62:0/2471197746 shutdown_connections 2024-04-02T14:44:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.393+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/2471197746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b101c6c0 0x5610b101eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.393+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/2471197746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 0x5610b10137d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.393+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/2471197746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1014030 0x5610b1012d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.393+0000 7fdc2e81d740 1 -- 172.21.15.62:0/2471197746 >> 172.21.15.62:0/2471197746 conn(0x5610b0fa7270 msgr2=0x5610b100dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.393+0000 7fdc2e81d740 1 -- 172.21.15.62:0/2471197746 shutdown_connections 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.393+0000 7fdc2e81d740 1 -- 172.21.15.62:0/2471197746 wait complete. 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.394+0000 7fdc2e81d740 1 -- start start 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.394+0000 7fdc2e81d740 1 Processor -- start 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.394+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1013350 0x5610b1019660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1014030 0x5610b1019ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b101c6c0 0x5610b101a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5610b0f1f690 con 0x5610b101c6c0 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5610b0f32a60 con 0x5610b1014030 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5610b101f2f0 con 0x5610b1013350 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc2e81d740 10 client.0 _ll_register_callbacks cb 0x5610b0f38030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc29a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b101c6c0 0x5610b101a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1014030 0x5610b1019ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc29000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1013350 0x5610b1019660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1014030 0x5610b1019ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41484/0 (socket says 172.21.15.62:41484) 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc29a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b101c6c0 0x5610b101a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35914/0 (socket says 172.21.15.62:35914) 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc29000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1013350 0x5610b1019660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39436/0 (socket says 172.21.15.62:39436) 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc23e00640 1 -- 172.21.15.62:0/1087118060 learned_addr learned my addr 172.21.15.62:0/1087118060 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc23e00640 1 -- 172.21.15.62:0/1087118060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1013350 msgr2=0x5610b1019660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc23e00640 1 --2- 172.21.15.62:0/1087118060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1013350 0x5610b1019660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc23e00640 1 -- 172.21.15.62:0/1087118060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b101c6c0 msgr2=0x5610b101a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc23e00640 1 --2- 172.21.15.62:0/1087118060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b101c6c0 0x5610b101a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.395+0000 7fdc23e00640 1 -- 172.21.15.62:0/1087118060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b0db3f70 con 0x5610b1014030 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.396+0000 7fdc23e00640 1 --2- 172.21.15.62:0/1087118060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1014030 0x5610b1019ba0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdc10012740 tx=0x7fdc10012b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ad1cd08fcc7069 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.396+0000 7fdc0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.396+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1087118060 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000f920 con 0x5610b1014030 2024-04-02T14:44:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.396+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1087118060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc180bef70 con 0x5610b1014030 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.396+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1087118060 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fdc1000fac0 con 0x5610b1014030 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.397+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1087118060 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc1000fcc0 con 0x5610b1014030 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.397+0000 7fdc2e81d740 10 client.25496 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.397+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1087118060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610b0c4b170 con 0x5610b1014030 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.397+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1087118060 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc180bef70 con 0x5610b1014030 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.397+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1087118060 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdc1001c440 con 0x5610b1014030 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.397+0000 7fdc2e81d740 10 client.25496 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.397+0000 7fdc2e81d740 10 client.25496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.397+0000 7fdc2e81d740 20 client.25496 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.397+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1087118060 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5610b102c830 0x5610b102ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.397+0000 7fdc2e81d740 4 client.25496 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.397+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1087118060 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610b0f84850 con 0x5610b102c830 2024-04-02T14:44:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.398+0000 7fdc29000640 1 --2- 172.21.15.62:0/1087118060 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5610b102c830 0x5610b102ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.398+0000 7fdc29000640 1 --2- 172.21.15.62:0/1087118060 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5610b102c830 0x5610b102ecb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e10f69c671989e39 server_cookie=3e84e1e092003178 in_seq=0 out_seq=0 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.398+0000 7fdc0aa00640 10 client.25496 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1087118060 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610b0f84850 con 0x5610b102c830 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc0aa00640 10 client.25496 handle_command_reply: tid=0 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 1 client.25496 shutdown 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1087118060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1014030 msgr2=0x5610b1019ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1087118060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1014030 0x5610b1019ba0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdc10012740 tx=0x7fdc10012b40 comp rx=0 tx=0).stop 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1087118060 shutdown_connections 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1087118060 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5610b102c830 0x5610b102ecb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1087118060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b101c6c0 0x5610b101a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1087118060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1014030 0x5610b1019ba0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1087118060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1013350 0x5610b1019660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1087118060 >> 172.21.15.62:0/1087118060 conn(0x5610b0fa7270 msgr2=0x5610b100faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc0aa00640 0 client.25496 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1087118060 shutdown_connections 2024-04-02T14:44:10.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1087118060 wait complete. 2024-04-02T14:44:10.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.399+0000 7fdc2e81d740 20 client.25496 trim_cache size 0 max 16384 2024-04-02T14:44:10.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.420+0000 7fdc2e81d740 1 Processor -- start 2024-04-02T14:44:10.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.420+0000 7fdc2e81d740 1 -- start start 2024-04-02T14:44:10.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.420+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1013350 0x5610b101bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.420+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 0x5610b10280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.420+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b101c6c0 0x5610b1034580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.420+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241af9d0 con 0x5610b1014030 2024-04-02T14:44:10.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.420+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241b1510 con 0x5610b101c6c0 2024-04-02T14:44:10.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.420+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241b00a0 con 0x5610b1013350 2024-04-02T14:44:10.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.420+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 0x5610b10280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.421+0000 7fdc29a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b101c6c0 0x5610b1034580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.421+0000 7fdc29000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1013350 0x5610b101bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.421+0000 7fdc29a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b101c6c0 0x5610b1034580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41488/0 (socket says 172.21.15.62:41488) 2024-04-02T14:44:10.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.421+0000 7fdc29a00640 1 -- 172.21.15.62:0/3031956905 learned_addr learned my addr 172.21.15.62:0/3031956905 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:10.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.421+0000 7fdc29a00640 1 -- 172.21.15.62:0/3031956905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1013350 msgr2=0x5610b101bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.421+0000 7fdc29a00640 1 --2- 172.21.15.62:0/3031956905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1013350 0x5610b101bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.421+0000 7fdc29a00640 1 -- 172.21.15.62:0/3031956905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 msgr2=0x5610b10280a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.421+0000 7fdc29a00640 1 --2- 172.21.15.62:0/3031956905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 0x5610b10280a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.421+0000 7fdc29a00640 1 -- 172.21.15.62:0/3031956905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b102b3c0 con 0x5610b101c6c0 2024-04-02T14:44:10.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.421+0000 7fdc29a00640 1 --2- 172.21.15.62:0/3031956905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b101c6c0 0x5610b1034580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0142f0 tx=0x7fdc1c009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8e1ca8dfbee01e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.422+0000 7fdc15600640 1 -- 172.21.15.62:0/3031956905 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c01a040 con 0x5610b101c6c0 2024-04-02T14:44:10.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.422+0000 7fdc15600640 1 -- 172.21.15.62:0/3031956905 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc1c028020 con 0x5610b101c6c0 2024-04-02T14:44:10.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.422+0000 7fdc15600640 1 -- 172.21.15.62:0/3031956905 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0042b0 con 0x5610b101c6c0 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.422+0000 7fdc2e81d740 1 -- 172.21.15.62:0/3031956905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b101c6c0 msgr2=0x5610b1034580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.422+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/3031956905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b101c6c0 0x5610b1034580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0142f0 tx=0x7fdc1c009b10 comp rx=0 tx=0).stop 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.423+0000 7fdc2e81d740 1 -- 172.21.15.62:0/3031956905 shutdown_connections 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.423+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/3031956905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b101c6c0 0x5610b1034580 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.423+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/3031956905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 0x5610b10280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.423+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/3031956905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b1013350 0x5610b101bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.423+0000 7fdc2e81d740 1 -- 172.21.15.62:0/3031956905 >> 172.21.15.62:0/3031956905 conn(0x5610b0fa7270 msgr2=0x5610b100fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.423+0000 7fdc2e81d740 1 -- 172.21.15.62:0/3031956905 shutdown_connections 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.423+0000 7fdc2e81d740 1 -- 172.21.15.62:0/3031956905 wait complete. 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.423+0000 7fdc2e81d740 1 -- start start 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.423+0000 7fdc2e81d740 1 Processor -- start 2024-04-02T14:44:10.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 0x5610b1021580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 0x5610b1021ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc2e81d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b101c6c0 0x5610b1022000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241b0e40 con 0x5610b1014030 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241b0770 con 0x5610b1013350 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc2e81d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241ad9b0 con 0x5610b101c6c0 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc2e81d740 10 client.0 _ll_register_callbacks cb 0x7fdc10009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 0x5610b1021ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 0x5610b1021ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35926/0 (socket says 172.21.15.62:35926) 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc29000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 0x5610b1021580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc23e00640 1 -- 172.21.15.62:0/1074236054 learned_addr learned my addr 172.21.15.62:0/1074236054 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc29a00640 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b101c6c0 0x5610b1022000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc23e00640 1 -- 172.21.15.62:0/1074236054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b101c6c0 msgr2=0x5610b1022000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.424+0000 7fdc23e00640 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b101c6c0 0x5610b1022000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.425+0000 7fdc23e00640 1 -- 172.21.15.62:0/1074236054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 msgr2=0x5610b1021580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.425+0000 7fdc23e00640 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 0x5610b1021580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.425+0000 7fdc23e00640 1 -- 172.21.15.62:0/1074236054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b1092e00 con 0x5610b1014030 2024-04-02T14:44:10.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.425+0000 7fdc29000640 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 0x5610b1021580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:10.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.425+0000 7fdc23e00640 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 0x5610b1021ac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5610b101ac30 tx=0x7fdc1001cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f950b311dcf9cb13 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.425+0000 7fdc0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:10.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.425+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1074236054 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10029070 con 0x5610b1014030 2024-04-02T14:44:10.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.425+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1074236054 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fdc1001c630 con 0x5610b1014030 2024-04-02T14:44:10.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.425+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1074236054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc1c028020 con 0x5610b1014030 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.425+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1074236054 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc10035050 con 0x5610b1014030 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.425+0000 7fdc2e81d740 10 client.15369 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.426+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1074236054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610b10940d0 con 0x5610b1014030 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.426+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1074236054 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc1c028020 con 0x5610b1014030 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.426+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1074236054 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdc100037b0 con 0x5610b1014030 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.426+0000 7fdc2e81d740 10 client.15369 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.426+0000 7fdc2e81d740 10 client.15369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.426+0000 7fdc2e81d740 20 client.15369 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.426+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5610b1094880 0x5610b1096ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.426+0000 7fdc2e81d740 4 client.15369 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.426+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1074236054 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5610b0f7eba0 con 0x5610b1094880 2024-04-02T14:44:10.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.426+0000 7fdc29000640 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5610b1094880 0x5610b1096ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.427+0000 7fdc29000640 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5610b1094880 0x5610b1096ca0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd817e2dcb0376ff server_cookie=ee32b05bcae9d510 in_seq=0 out_seq=0 2024-04-02T14:44:10.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.427+0000 7fdc0aa00640 10 client.15369 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:10.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc0aa00640 1 -- 172.21.15.62:0/1074236054 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5610b0f7eba0 con 0x5610b1094880 2024-04-02T14:44:10.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc0aa00640 10 client.15369 handle_command_reply: tid=0 2024-04-02T14:44:10.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc2e81d740 1 client.15369 shutdown 2024-04-02T14:44:10.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1074236054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 msgr2=0x5610b1021ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 0x5610b1021ac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5610b101ac30 tx=0x7fdc1001cd50 comp rx=0 tx=0).stop 2024-04-02T14:44:10.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1074236054 shutdown_connections 2024-04-02T14:44:10.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5610b1094880 0x5610b1096ca0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610b101c6c0 0x5610b1022000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610b1014030 0x5610b1021ac0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc2e81d740 1 --2- 172.21.15.62:0/1074236054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5610b1013350 0x5610b1021580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc0aa00640 0 client.15369 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:10.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.428+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1074236054 >> 172.21.15.62:0/1074236054 conn(0x5610b0fa7270 msgr2=0x5610b1019600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.429+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1074236054 shutdown_connections 2024-04-02T14:44:10.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.429+0000 7fdc2e81d740 1 -- 172.21.15.62:0/1074236054 wait complete. 2024-04-02T14:44:10.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.429+0000 7fdc2e81d740 20 client.15369 trim_cache size 0 max 16384 2024-04-02T14:44:10.430 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:44:10.431 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-02T14:44:10.431 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "88de8791-0106-4661-9a52-a177e6c729b9", 2024-04-02T14:44:10.431 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-02T14:44:10.431 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:44:10.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 -- 172.21.15.62:0/683225792 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fdc0c0a3ec0 msgr2=0x7fdc0c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fdc0c0a3ec0 0x7fdc0c0a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005f40 tx=0x7fdc1c004ab0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 -- 172.21.15.62:0/683225792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc24101a90 msgr2=0x7fdc2419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc24101a90 0x7fdc2419a6c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009900 tx=0x7fdc1802dc80 comp rx=0 tx=0).stop 2024-04-02T14:44:10.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 -- 172.21.15.62:0/683225792 shutdown_connections 2024-04-02T14:44:10.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fdc0c0a3ec0 0x7fdc0c0a6380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc24107aa0 0x7fdc2419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc24102450 0x7fdc2419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 --2- 172.21.15.62:0/683225792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc24101a90 0x7fdc2419a6c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 -- 172.21.15.62:0/683225792 >> 172.21.15.62:0/683225792 conn(0x7fdc24078f00 msgr2=0x7fdc241041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 -- 172.21.15.62:0/683225792 shutdown_connections 2024-04-02T14:44:10.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.430+0000 7fdc2ba00640 1 -- 172.21.15.62:0/683225792 wait complete. 2024-04-02T14:44:10.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:10 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/858689191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:44:10.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:10 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2377858954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:10.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:44:10.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:44:10.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:10 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/858689191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:44:10.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:10 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2377858954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:10.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.640+0000 7fb8df200640 1 -- 172.21.15.62:0/110240059 --> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fb8d810c2e0 con 0x7fb8bc0a3ee0 2024-04-02T14:44:10.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.662+0000 7f5f52e00640 1 -- 172.21.15.62:0/2165072183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f4c107980 msgr2=0x7f5f4c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.662+0000 7f5f52e00640 1 --2- 172.21.15.62:0/2165072183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f4c107980 0x7f5f4c107d60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009ca0 tx=0x7f5f4002c100 comp rx=0 tx=0).stop 2024-04-02T14:44:10.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.662+0000 7f5f52e00640 1 -- 172.21.15.62:0/2165072183 shutdown_connections 2024-04-02T14:44:10.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.662+0000 7f5f52e00640 1 --2- 172.21.15.62:0/2165072183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4c102330 0x7f5f4c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.662+0000 7f5f52e00640 1 --2- 172.21.15.62:0/2165072183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f4c101970 0x7f5f4c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.662+0000 7f5f52e00640 1 --2- 172.21.15.62:0/2165072183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f4c107980 0x7f5f4c107d60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.662+0000 7f5f52e00640 1 -- 172.21.15.62:0/2165072183 >> 172.21.15.62:0/2165072183 conn(0x7f5f4c0fd4a0 msgr2=0x7f5f4c0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.662+0000 7f5f52e00640 1 -- 172.21.15.62:0/2165072183 shutdown_connections 2024-04-02T14:44:10.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.662+0000 7f5f52e00640 1 -- 172.21.15.62:0/2165072183 wait complete. 2024-04-02T14:44:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.663+0000 7f5f52e00640 1 Processor -- start 2024-04-02T14:44:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.664+0000 7f5f52e00640 1 -- start start 2024-04-02T14:44:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.664+0000 7f5f52e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f4c101970 0x7f5f4c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.664+0000 7f5f52e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f4c102330 0x7f5f4c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.664+0000 7f5f52e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4c107980 0x7f5f4c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.664+0000 7f5f52e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c111eb0 con 0x7f5f4c107980 2024-04-02T14:44:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.664+0000 7f5f52e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c111d30 con 0x7f5f4c102330 2024-04-02T14:44:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.664+0000 7f5f52e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c112030 con 0x7f5f4c101970 2024-04-02T14:44:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.665+0000 7f5f51a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f4c101970 0x7f5f4c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.665+0000 7f5f51a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f4c101970 0x7f5f4c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39472/0 (socket says 172.21.15.62:39472) 2024-04-02T14:44:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.665+0000 7f5f51a00640 1 -- 172.21.15.62:0/4032087399 learned_addr learned my addr 172.21.15.62:0/4032087399 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.665+0000 7f5f52400640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4c107980 0x7f5f4c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.665+0000 7f5f51000640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f4c102330 0x7f5f4c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.665+0000 7f5f51a00640 1 -- 172.21.15.62:0/4032087399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f4c102330 msgr2=0x7f5f4c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.665+0000 7f5f51a00640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f4c102330 0x7f5f4c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.665+0000 7f5f51a00640 1 -- 172.21.15.62:0/4032087399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4c107980 msgr2=0x7f5f4c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.665+0000 7f5f51a00640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4c107980 0x7f5f4c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.665+0000 7f5f51a00640 1 -- 172.21.15.62:0/4032087399 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4c19d300 con 0x7f5f4c101970 2024-04-02T14:44:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.665+0000 7f5f52400640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4c107980 0x7f5f4c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.666+0000 7f5f51a00640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f4c101970 0x7f5f4c198310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5f40004af0 tx=0x7f5f40034440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.666+0000 7f5f4a000640 1 -- 172.21.15.62:0/4032087399 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40039050 con 0x7f5f4c101970 2024-04-02T14:44:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.666+0000 7f5f4a000640 1 -- 172.21.15.62:0/4032087399 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5f40045070 con 0x7f5f4c101970 2024-04-02T14:44:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.666+0000 7f5f4a000640 1 -- 172.21.15.62:0/4032087399 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40004600 con 0x7f5f4c101970 2024-04-02T14:44:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.666+0000 7f5f52e00640 1 -- 172.21.15.62:0/4032087399 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4c19d590 con 0x7f5f4c101970 2024-04-02T14:44:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.667+0000 7f5f52e00640 1 -- 172.21.15.62:0/4032087399 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f4c073d30 con 0x7f5f4c101970 2024-04-02T14:44:10.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.670+0000 7f5f57185740 1 Processor -- start 2024-04-02T14:44:10.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f57185740 1 -- start start 2024-04-02T14:44:10.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f4a000640 1 -- 172.21.15.62:0/4032087399 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f5f40032990 con 0x7f5f4c101970 2024-04-02T14:44:10.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f57185740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe21960 0x55aedbe21d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f57185740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe22280 0x55aedbe2a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f57185740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe2ae60 0x55aedbe2d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f57185740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55aedbd85770 con 0x55aedbe2ae60 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f57185740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55aedbd2d700 con 0x55aedbe22280 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f57185740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aedbd81720 con 0x55aedbe21960 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f51a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe22280 0x55aedbe2a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f52400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe21960 0x55aedbe21d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f52400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe21960 0x55aedbe21d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39474/0 (socket says 172.21.15.62:39474) 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f52400640 1 -- 172.21.15.62:0/2599170925 learned_addr learned my addr 172.21.15.62:0/2599170925 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.671+0000 7f5f51a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe22280 0x55aedbe2a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41510/0 (socket says 172.21.15.62:41510) 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.672+0000 7f5f51000640 1 --2- 172.21.15.62:0/2599170925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe2ae60 0x55aedbe2d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.672+0000 7f5f52400640 1 -- 172.21.15.62:0/2599170925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe22280 msgr2=0x55aedbe2a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.672+0000 7f5f52400640 1 --2- 172.21.15.62:0/2599170925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe22280 0x55aedbe2a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.672+0000 7f5f52400640 1 -- 172.21.15.62:0/2599170925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe2ae60 msgr2=0x55aedbe2d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.672+0000 7f5f52400640 1 --2- 172.21.15.62:0/2599170925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe2ae60 0x55aedbe2d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.672+0000 7f5f52400640 1 -- 172.21.15.62:0/2599170925 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aedbb637d0 con 0x55aedbe21960 2024-04-02T14:44:10.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.672+0000 7f5f52400640 1 --2- 172.21.15.62:0/2599170925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe21960 0x55aedbe21d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009c20 tx=0x7f5f440117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9517de7ccfc2ab9c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.673+0000 7f5f3ea00640 1 -- 172.21.15.62:0/2599170925 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f44012450 con 0x55aedbe21960 2024-04-02T14:44:10.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.673+0000 7f5f3ea00640 1 -- 172.21.15.62:0/2599170925 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5f44012af0 con 0x55aedbe21960 2024-04-02T14:44:10.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.673+0000 7f5f3ea00640 1 -- 172.21.15.62:0/2599170925 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400fc60 con 0x55aedbe21960 2024-04-02T14:44:10.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.673+0000 7f5f4a000640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5f340a82d0 0x7f5f340aa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.673+0000 7f5f4a000640 1 -- 172.21.15.62:0/4032087399 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f5f400f2d90 con 0x7f5f4c101970 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.673+0000 7f5f51a00640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5f340a82d0 0x7f5f340aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.674+0000 7f5f57185740 1 -- 172.21.15.62:0/2599170925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe21960 msgr2=0x55aedbe21d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.674+0000 7f5f57185740 1 --2- 172.21.15.62:0/2599170925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe21960 0x55aedbe21d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009c20 tx=0x7f5f440117b0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.674+0000 7f5f57185740 1 -- 172.21.15.62:0/2599170925 shutdown_connections 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.674+0000 7f5f57185740 1 --2- 172.21.15.62:0/2599170925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe2ae60 0x55aedbe2d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.674+0000 7f5f57185740 1 --2- 172.21.15.62:0/2599170925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe22280 0x55aedbe2a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.674+0000 7f5f57185740 1 --2- 172.21.15.62:0/2599170925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe21960 0x55aedbe21d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.674+0000 7f5f57185740 1 -- 172.21.15.62:0/2599170925 >> 172.21.15.62:0/2599170925 conn(0x55aedbe1c090 msgr2=0x55aedbe1c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.674+0000 7f5f57185740 1 -- 172.21.15.62:0/2599170925 shutdown_connections 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.674+0000 7f5f57185740 1 -- 172.21.15.62:0/2599170925 wait complete. 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.674+0000 7f5f51a00640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5f340a82d0 0x7f5f340aa790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f400f0520 tx=0x7f5f400df040 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.674+0000 7f5f57185740 1 -- start start 2024-04-02T14:44:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.675+0000 7f5f57185740 1 Processor -- start 2024-04-02T14:44:10.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.675+0000 7f5f57185740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 0x55aedbe27e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.675+0000 7f5f57185740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 0x55aedbe283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.675+0000 7f5f57185740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe2ae60 0x55aedbe28910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.675+0000 7f5f57185740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55aedbe2dc50 con 0x55aedbe22280 2024-04-02T14:44:10.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.676+0000 7f5f57185740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55aedbd40ac0 con 0x55aedbe21960 2024-04-02T14:44:10.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.676+0000 7f5f57185740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aedbe2ddd0 con 0x55aedbe2ae60 2024-04-02T14:44:10.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.676+0000 7f5f57185740 10 client.0 _ll_register_callbacks cb 0x55aedbd43180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:10.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.676+0000 7f5f51000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 0x55aedbe283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.676+0000 7f5f51000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 0x55aedbe283d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35954/0 (socket says 172.21.15.62:35954) 2024-04-02T14:44:10.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.676+0000 7f5f51000640 1 -- 172.21.15.62:0/4156133107 learned_addr learned my addr 172.21.15.62:0/4156133107 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:10.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.676+0000 7f5f52400640 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 0x55aedbe27e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.676+0000 7f5f52400640 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe2ae60 0x55aedbe28910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.677+0000 7f5f51000640 1 -- 172.21.15.62:0/4156133107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe2ae60 msgr2=0x55aedbe28910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.677+0000 7f5f51000640 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe2ae60 0x55aedbe28910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.677+0000 7f5f51000640 1 -- 172.21.15.62:0/4156133107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 msgr2=0x55aedbe27e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.677+0000 7f5f51000640 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 0x55aedbe27e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.677+0000 7f5f51000640 1 -- 172.21.15.62:0/4156133107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aedba993f0 con 0x55aedbe22280 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.677+0000 7f5f52400640 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 0x55aedbe27e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.677+0000 7f5f51000640 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 0x55aedbe283d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5f38012730 tx=0x7f5f38012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808307e0b2ea77c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.677+0000 7f5f33400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.677+0000 7f5f33400640 1 -- 172.21.15.62:0/4156133107 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38004050 con 0x55aedbe22280 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.677+0000 7f5f33400640 1 -- 172.21.15.62:0/4156133107 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f5f380041f0 con 0x55aedbe22280 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.677+0000 7f5f57185740 1 -- 172.21.15.62:0/4156133107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f44012af0 con 0x55aedbe22280 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.678+0000 7f5f33400640 1 -- 172.21.15.62:0/4156133107 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f380043f0 con 0x55aedbe22280 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.678+0000 7f5f57185740 10 client.15381 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.678+0000 7f5f57185740 1 -- 172.21.15.62:0/4156133107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aedba590f0 con 0x55aedbe22280 2024-04-02T14:44:10.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.678+0000 7f5f33400640 1 -- 172.21.15.62:0/4156133107 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5f44012af0 con 0x55aedbe22280 2024-04-02T14:44:10.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.678+0000 7f5f33400640 1 -- 172.21.15.62:0/4156133107 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5f38003ca0 con 0x55aedbe22280 2024-04-02T14:44:10.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.679+0000 7f5f57185740 10 client.15381 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:10.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.679+0000 7f5f57185740 10 client.15381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:10.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.679+0000 7f5f57185740 20 client.15381 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:10.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.679+0000 7f5f57185740 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55aedbe3ade0 0x55aedbe3d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.679+0000 7f5f57185740 4 client.15381 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:10.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.679+0000 7f5f57185740 1 -- 172.21.15.62:0/4156133107 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aedbd26630 con 0x55aedbe3ade0 2024-04-02T14:44:10.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.679+0000 7f5f52400640 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55aedbe3ade0 0x55aedbe3d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.680+0000 7f5f52400640 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55aedbe3ade0 0x55aedbe3d1f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab599ef86443da49 server_cookie=9127570b96ec6c42 in_seq=0 out_seq=0 2024-04-02T14:44:10.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.680+0000 7f5f33400640 10 client.15381 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f33400640 1 -- 172.21.15.62:0/4156133107 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aedbd26630 con 0x55aedbe3ade0 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f33400640 10 client.15381 handle_command_reply: tid=0 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f57185740 1 client.15381 shutdown 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f57185740 1 -- 172.21.15.62:0/4156133107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 msgr2=0x55aedbe283d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f57185740 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 0x55aedbe283d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5f38012730 tx=0x7f5f38012bf0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f57185740 1 -- 172.21.15.62:0/4156133107 shutdown_connections 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f57185740 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55aedbe3ade0 0x55aedbe3d1f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f57185740 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe2ae60 0x55aedbe28910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f57185740 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 0x55aedbe283d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f57185740 1 --2- 172.21.15.62:0/4156133107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 0x55aedbe27e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f57185740 1 -- 172.21.15.62:0/4156133107 >> 172.21.15.62:0/4156133107 conn(0x55aedbe1c090 msgr2=0x55aedbe1b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f33400640 0 client.15381 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:10.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f57185740 1 -- 172.21.15.62:0/4156133107 shutdown_connections 2024-04-02T14:44:10.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.681+0000 7f5f57185740 1 -- 172.21.15.62:0/4156133107 wait complete. 2024-04-02T14:44:10.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.682+0000 7f5f57185740 20 client.15381 trim_cache size 0 max 16384 2024-04-02T14:44:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.706+0000 7f5f57185740 1 Processor -- start 2024-04-02T14:44:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.706+0000 7f5f57185740 1 -- start start 2024-04-02T14:44:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.706+0000 7f5f57185740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe21960 0x55aedbe2a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.706+0000 7f5f57185740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe22280 0x55aedbe36b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.706+0000 7f5f57185740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe2ae60 0x55aedbe42b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.706+0000 7f5f57185740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c1ab430 con 0x55aedbe21960 2024-04-02T14:44:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.706+0000 7f5f57185740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c1acf70 con 0x55aedbe2ae60 2024-04-02T14:44:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.706+0000 7f5f57185740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c1abb00 con 0x55aedbe22280 2024-04-02T14:44:10.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.706+0000 7f5f51000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe22280 0x55aedbe36b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.707+0000 7f5f51000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe22280 0x55aedbe36b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39492/0 (socket says 172.21.15.62:39492) 2024-04-02T14:44:10.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.707+0000 7f5f52400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe21960 0x55aedbe2a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.707+0000 7f5f51000640 1 -- 172.21.15.62:0/714930903 learned_addr learned my addr 172.21.15.62:0/714930903 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.707+0000 7f5f52400640 1 --2- 172.21.15.62:0/714930903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe2ae60 0x55aedbe42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.707+0000 7f5f51000640 1 -- 172.21.15.62:0/714930903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe2ae60 msgr2=0x55aedbe42b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.707+0000 7f5f51000640 1 --2- 172.21.15.62:0/714930903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe2ae60 0x55aedbe42b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.707+0000 7f5f51000640 1 -- 172.21.15.62:0/714930903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe21960 msgr2=0x55aedbe2a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.707+0000 7f5f51000640 1 --2- 172.21.15.62:0/714930903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe21960 0x55aedbe2a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.707+0000 7f5f51000640 1 -- 172.21.15.62:0/714930903 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aedbe23a30 con 0x55aedbe22280 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.707+0000 7f5f52400640 1 --2- 172.21.15.62:0/714930903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe21960 0x55aedbe2a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.707+0000 7f5f51000640 1 --2- 172.21.15.62:0/714930903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe22280 0x55aedbe36b10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5f38025930 tx=0x7f5f38025960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3691dda4ea963bf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.708+0000 7f5f3e000640 1 -- 172.21.15.62:0/714930903 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38025b40 con 0x55aedbe22280 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.708+0000 7f5f3e000640 1 -- 172.21.15.62:0/714930903 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5f3800fe20 con 0x55aedbe22280 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.708+0000 7f5f3e000640 1 -- 172.21.15.62:0/714930903 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38005460 con 0x55aedbe22280 2024-04-02T14:44:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.709+0000 7f5f57185740 1 -- 172.21.15.62:0/714930903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe22280 msgr2=0x55aedbe36b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.709+0000 7f5f57185740 1 --2- 172.21.15.62:0/714930903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe22280 0x55aedbe36b10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5f38025930 tx=0x7f5f38025960 comp rx=0 tx=0).stop 2024-04-02T14:44:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.709+0000 7f5f57185740 1 -- 172.21.15.62:0/714930903 shutdown_connections 2024-04-02T14:44:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.709+0000 7f5f57185740 1 --2- 172.21.15.62:0/714930903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe2ae60 0x55aedbe42b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.709+0000 7f5f57185740 1 --2- 172.21.15.62:0/714930903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe22280 0x55aedbe36b10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.709+0000 7f5f57185740 1 --2- 172.21.15.62:0/714930903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe21960 0x55aedbe2a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.709+0000 7f5f57185740 1 -- 172.21.15.62:0/714930903 >> 172.21.15.62:0/714930903 conn(0x55aedbe1c090 msgr2=0x55aedbe1b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.709+0000 7f5f57185740 1 -- 172.21.15.62:0/714930903 shutdown_connections 2024-04-02T14:44:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.709+0000 7f5f57185740 1 -- 172.21.15.62:0/714930903 wait complete. 2024-04-02T14:44:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.709+0000 7f5f57185740 1 -- start start 2024-04-02T14:44:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f57185740 1 Processor -- start 2024-04-02T14:44:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f57185740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 0x55aedbe2f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f57185740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 0x55aedbe2f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f57185740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe2ae60 0x55aedbe9a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f57185740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c1ac8a0 con 0x55aedbe22280 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f57185740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c1ac1d0 con 0x55aedbe21960 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f57185740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c1a9430 con 0x55aedbe2ae60 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f57185740 10 client.0 _ll_register_callbacks cb 0x7f5f38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f52400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 0x55aedbe2f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f52400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 0x55aedbe2f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:41528/0 (socket says 172.21.15.62:41528) 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f52400640 1 -- 172.21.15.62:0/1615869410 learned_addr learned my addr 172.21.15.62:0/1615869410 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.710+0000 7f5f51000640 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 0x55aedbe2f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.711+0000 7f5f52400640 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe2ae60 0x55aedbe9a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.711+0000 7f5f52400640 1 -- 172.21.15.62:0/1615869410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe2ae60 msgr2=0x55aedbe9a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.711+0000 7f5f52400640 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe2ae60 0x55aedbe9a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.711+0000 7f5f52400640 1 -- 172.21.15.62:0/1615869410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 msgr2=0x55aedbe2f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.711+0000 7f5f52400640 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 0x55aedbe2f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.711+0000 7f5f52400640 1 -- 172.21.15.62:0/1615869410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aedbe9a930 con 0x55aedbe21960 2024-04-02T14:44:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.711+0000 7f5f52400640 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 0x55aedbe2f2a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5f4401c040 tx=0x7f5f44032040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba92e35318cc3d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.711+0000 7f5f33400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.712+0000 7f5f33400640 1 -- 172.21.15.62:0/1615869410 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f44030c90 con 0x55aedbe21960 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.712+0000 7f5f33400640 1 -- 172.21.15.62:0/1615869410 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f5f4400f930 con 0x55aedbe21960 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.712+0000 7f5f57185740 1 -- 172.21.15.62:0/1615869410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3800fe20 con 0x55aedbe21960 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.712+0000 7f5f33400640 1 -- 172.21.15.62:0/1615869410 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f44030e30 con 0x55aedbe21960 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.712+0000 7f5f57185740 10 client.25520 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.712+0000 7f5f57185740 1 -- 172.21.15.62:0/1615869410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aedbea1ce0 con 0x55aedbe21960 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.712+0000 7f5f33400640 1 -- 172.21.15.62:0/1615869410 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5f3800fe20 con 0x55aedbe21960 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.713+0000 7f5f33400640 1 -- 172.21.15.62:0/1615869410 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5f44026040 con 0x55aedbe21960 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.713+0000 7f5f57185740 10 client.25520 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.713+0000 7f5f57185740 10 client.25520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.713+0000 7f5f57185740 20 client.25520 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.713+0000 7f5f57185740 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55aedbea21c0 0x55aedbea45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:10.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.713+0000 7f5f57185740 4 client.25520 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.713+0000 7f5f57185740 1 -- 172.21.15.62:0/1615869410 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aedbd92cb0 con 0x55aedbea21c0 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.713+0000 7f5f51000640 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55aedbea21c0 0x55aedbea45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f51000640 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55aedbea21c0 0x55aedbea45b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8b5a42e37b3102 server_cookie=423dab423477919f in_seq=0 out_seq=0 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f33400640 10 client.25520 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f33400640 1 -- 172.21.15.62:0/1615869410 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55aedbd92cb0 con 0x55aedbea21c0 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f33400640 10 client.25520 handle_command_reply: tid=0 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f57185740 1 client.25520 shutdown 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f57185740 1 -- 172.21.15.62:0/1615869410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 msgr2=0x55aedbe2f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f57185740 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 0x55aedbe2f2a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5f4401c040 tx=0x7f5f44032040 comp rx=0 tx=0).stop 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f57185740 1 -- 172.21.15.62:0/1615869410 shutdown_connections 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f57185740 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55aedbea21c0 0x55aedbea45b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f57185740 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aedbe2ae60 0x55aedbe9a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f57185740 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aedbe22280 0x55aedbe2f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f57185740 1 --2- 172.21.15.62:0/1615869410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aedbe21960 0x55aedbe2f2a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f57185740 1 -- 172.21.15.62:0/1615869410 >> 172.21.15.62:0/1615869410 conn(0x55aedbe1c090 msgr2=0x55aedbe27e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f33400640 0 client.25520 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.714+0000 7f5f57185740 1 -- 172.21.15.62:0/1615869410 shutdown_connections 2024-04-02T14:44:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.715+0000 7f5f57185740 1 -- 172.21.15.62:0/1615869410 wait complete. 2024-04-02T14:44:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.715+0000 7f5f57185740 20 client.25520 trim_cache size 0 max 16384 2024-04-02T14:44:10.716 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:44:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T14:44:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-02T14:44:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "88de8791-0106-4661-9a52-a177e6c729b9": { 2024-04-02T14:44:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-04-02T14:44:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "88de8791-0106-4661-9a52-a177e6c729b9", 2024-04-02T14:44:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-04-02T14:44:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:44:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:44:10.717 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:44:10.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.715+0000 7f5f52e00640 1 -- 172.21.15.62:0/4032087399 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5f340a82d0 msgr2=0x7f5f340aa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.715+0000 7f5f52e00640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5f340a82d0 0x7f5f340aa790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f400f0520 tx=0x7f5f400df040 comp rx=0 tx=0).stop 2024-04-02T14:44:10.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.715+0000 7f5f52e00640 1 -- 172.21.15.62:0/4032087399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f4c101970 msgr2=0x7f5f4c198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.715+0000 7f5f52e00640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f4c101970 0x7f5f4c198310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5f40004af0 tx=0x7f5f40034440 comp rx=0 tx=0).stop 2024-04-02T14:44:10.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.716+0000 7f5f52e00640 1 -- 172.21.15.62:0/4032087399 shutdown_connections 2024-04-02T14:44:10.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.716+0000 7f5f52e00640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5f340a82d0 0x7f5f340aa790 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.716+0000 7f5f52e00640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4c107980 0x7f5f4c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.716+0000 7f5f52e00640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f4c102330 0x7f5f4c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.716+0000 7f5f52e00640 1 --2- 172.21.15.62:0/4032087399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f4c101970 0x7f5f4c198310 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.716+0000 7f5f52e00640 1 -- 172.21.15.62:0/4032087399 >> 172.21.15.62:0/4032087399 conn(0x7f5f4c0fd4a0 msgr2=0x7f5f4c0fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.716+0000 7f5f52e00640 1 -- 172.21.15.62:0/4032087399 shutdown_connections 2024-04-02T14:44:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.716+0000 7f5f52e00640 1 -- 172.21.15.62:0/4032087399 wait complete. 2024-04-02T14:44:10.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.733+0000 7fb8d4c00640 1 -- 172.21.15.62:0/110240059 <== mgr.25273 v2:172.21.15.62:6800/110874546 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb8d810c2e0 con 0x7fb8bc0a3ee0 2024-04-02T14:44:10.731 INFO:teuthology.orchestra.run.smithi062.stdout:Schedule set for path / 2024-04-02T14:44:10.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.737+0000 7fb8df200640 1 -- 172.21.15.62:0/110240059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb8bc0a3ee0 msgr2=0x7fb8bc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.737+0000 7fb8df200640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb8bc0a3ee0 0x7fb8bc0a63a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4000c00 tx=0x7fb8c4005fd0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.737+0000 7fb8df200640 1 -- 172.21.15.62:0/110240059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d8103f20 msgr2=0x7fb8d819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:10.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.737+0000 7fb8df200640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d8103f20 0x7fb8d819a610 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800ea60 tx=0x7fb8c800ef20 comp rx=0 tx=0).stop 2024-04-02T14:44:10.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.737+0000 7fb8df200640 1 -- 172.21.15.62:0/110240059 shutdown_connections 2024-04-02T14:44:10.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.737+0000 7fb8df200640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb8bc0a3ee0 0x7fb8bc0a63a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.737+0000 7fb8df200640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8d819eed0 0x7fb8d819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.737+0000 7fb8df200640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8108ff0 0x7fb8d819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.737+0000 7fb8df200640 1 --2- 172.21.15.62:0/110240059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d8103f20 0x7fb8d819a610 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:10.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.737+0000 7fb8df200640 1 -- 172.21.15.62:0/110240059 >> 172.21.15.62:0/110240059 conn(0x7fb8d80ff780 msgr2=0x7fb8d8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:10.737 INFO:tasks.cephfs.filesystem:scrub status for tag:88de8791-0106-4661-9a52-a177e6c729b9 - {'path': '/', 'tag': '88de8791-0106-4661-9a52-a177e6c729b9', 'options': 'recursive,force'} 2024-04-02T14:44:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.737+0000 7fb8df200640 1 -- 172.21.15.62:0/110240059 shutdown_connections 2024-04-02T14:44:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:10.738+0000 7fb8df200640 1 -- 172.21.15.62:0/110240059 wait complete. 2024-04-02T14:44:10.754 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-02T14:44:10.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:10 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/858689191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:44:10.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:10 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2377858954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:44:11.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.017+0000 7f15ed400640 1 -- 172.21.15.62:0/318903352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8108cf0 msgr2=0x7f15e810cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:11.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.017+0000 7f15ed400640 1 --2- 172.21.15.62:0/318903352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8108cf0 0x7f15e810cd30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009900 tx=0x7f15d802bc40 comp rx=0 tx=0).stop 2024-04-02T14:44:11.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.017+0000 7f15ed400640 1 -- 172.21.15.62:0/318903352 shutdown_connections 2024-04-02T14:44:11.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.017+0000 7f15ed400640 1 --2- 172.21.15.62:0/318903352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e810d410 0x7f15e8114fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.017+0000 7f15ed400640 1 --2- 172.21.15.62:0/318903352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8108cf0 0x7f15e810cd30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.017+0000 7f15ed400640 1 --2- 172.21.15.62:0/318903352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e8108340 0x7f15e8108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.017+0000 7f15ed400640 1 -- 172.21.15.62:0/318903352 >> 172.21.15.62:0/318903352 conn(0x7f15e8076070 msgr2=0x7f15e8078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:11.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.017+0000 7f15ed400640 1 -- 172.21.15.62:0/318903352 shutdown_connections 2024-04-02T14:44:11.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.017+0000 7f15ed400640 1 -- 172.21.15.62:0/318903352 wait complete. 2024-04-02T14:44:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.018+0000 7f15ed400640 1 Processor -- start 2024-04-02T14:44:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.018+0000 7f15ed400640 1 -- start start 2024-04-02T14:44:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15ed400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8108340 0x7f15e81a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15ed400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e810d410 0x7f15e81a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15ed400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e819e8a0 0x7f15e819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15ed400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81185e0 con 0x7f15e810d410 2024-04-02T14:44:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15ed400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8118460 con 0x7f15e819e8a0 2024-04-02T14:44:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15ed400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8118760 con 0x7f15e8108340 2024-04-02T14:44:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e810d410 0x7f15e81a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e810d410 0x7f15e81a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35988/0 (socket says 172.21.15.62:35988) 2024-04-02T14:44:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15e5e00640 1 -- 172.21.15.62:0/1525280214 learned_addr learned my addr 172.21.15.62:0/1525280214 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15e7200640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e819e8a0 0x7f15e819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15e6800640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8108340 0x7f15e81a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15e7200640 1 -- 172.21.15.62:0/1525280214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8108340 msgr2=0x7f15e81a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.019+0000 7f15e7200640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8108340 0x7f15e81a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.020+0000 7f15e7200640 1 -- 172.21.15.62:0/1525280214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e810d410 msgr2=0x7f15e81a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.020+0000 7f15e7200640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e810d410 0x7f15e81a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.020+0000 7f15e7200640 1 -- 172.21.15.62:0/1525280214 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e819f430 con 0x7f15e819e8a0 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.020+0000 7f15e5e00640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e810d410 0x7f15e81a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.020+0000 7f15e7200640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e819e8a0 0x7f15e819ed50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f15e000da10 tx=0x7f15e000ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.020+0000 7f15df400640 1 -- 172.21.15.62:0/1525280214 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e000bd70 con 0x7f15e819e8a0 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.020+0000 7f15df400640 1 -- 172.21.15.62:0/1525280214 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15e0004030 con 0x7f15e819e8a0 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.020+0000 7f15ed400640 1 -- 172.21.15.62:0/1525280214 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e819f720 con 0x7f15e819e8a0 2024-04-02T14:44:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.020+0000 7f15df400640 1 -- 172.21.15.62:0/1525280214 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e00107e0 con 0x7f15e819e8a0 2024-04-02T14:44:11.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.020+0000 7f15ed400640 1 -- 172.21.15.62:0/1525280214 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e81ab830 con 0x7f15e819e8a0 2024-04-02T14:44:11.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.021+0000 7f15ed400640 1 -- 172.21.15.62:0/1525280214 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e8109810 con 0x7f15e819e8a0 2024-04-02T14:44:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.024+0000 7f15df400640 1 -- 172.21.15.62:0/1525280214 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f15e00041e0 con 0x7f15e819e8a0 2024-04-02T14:44:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.025+0000 7f15df400640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f15cc0a3ee0 0x7f15cc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.025+0000 7f15df400640 1 -- 172.21.15.62:0/1525280214 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f15e00d0fc0 con 0x7f15e819e8a0 2024-04-02T14:44:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.025+0000 7f15e6800640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f15cc0a3ee0 0x7f15cc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.026+0000 7f15e6800640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f15cc0a3ee0 0x7f15cc0a63a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009bb0 tx=0x7f15d4005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:11.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.027+0000 7f15df400640 1 -- 172.21.15.62:0/1525280214 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f15e819f720 con 0x7f15e819e8a0 2024-04-02T14:44:11.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.267+0000 7f15ed400640 1 -- 172.21.15.62:0/1525280214 --> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f15e8108cf0 con 0x7f15cc0a3ee0 2024-04-02T14:44:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.431+0000 7f15df400640 1 -- 172.21.15.62:0/1525280214 <== mgr.25273 v2:172.21.15.62:6800/110874546 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f15e8108cf0 con 0x7f15cc0a3ee0 2024-04-02T14:44:11.429 INFO:teuthology.orchestra.run.smithi062.stdout:Retention added to path / 2024-04-02T14:44:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.435+0000 7f15ed400640 1 -- 172.21.15.62:0/1525280214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f15cc0a3ee0 msgr2=0x7f15cc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.435+0000 7f15ed400640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f15cc0a3ee0 0x7f15cc0a63a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009bb0 tx=0x7f15d4005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.436+0000 7f15ed400640 1 -- 172.21.15.62:0/1525280214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e819e8a0 msgr2=0x7f15e819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.436+0000 7f15ed400640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e819e8a0 0x7f15e819ed50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f15e000da10 tx=0x7f15e000ded0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.436+0000 7f15ed400640 1 -- 172.21.15.62:0/1525280214 shutdown_connections 2024-04-02T14:44:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.436+0000 7f15ed400640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f15cc0a3ee0 0x7f15cc0a63a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.436+0000 7f15ed400640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e819e8a0 0x7f15e819ed50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.436+0000 7f15ed400640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e810d410 0x7f15e81a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.436+0000 7f15ed400640 1 --2- 172.21.15.62:0/1525280214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8108340 0x7f15e81a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.436+0000 7f15ed400640 1 -- 172.21.15.62:0/1525280214 >> 172.21.15.62:0/1525280214 conn(0x7f15e8076070 msgr2=0x7f15e8076be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.436+0000 7f15ed400640 1 -- 172.21.15.62:0/1525280214 shutdown_connections 2024-04-02T14:44:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.436+0000 7f15ed400640 1 -- 172.21.15.62:0/1525280214 wait complete. 2024-04-02T14:44:11.455 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-02T14:44:11.537 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:11 smithi062 ceph-mon[32116]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 684 B/s wr, 20 op/s 2024-04-02T14:44:11.537 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:11 smithi062 ceph-mon[32116]: from='client.25490 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:11.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:11 smithi113 ceph-mon[36135]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 684 B/s wr, 20 op/s 2024-04-02T14:44:11.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:11 smithi113 ceph-mon[36135]: from='client.25490 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:11.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.673+0000 7fc025200640 1 -- 172.21.15.62:0/1605257910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc020103b30 msgr2=0x7fc020103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:11.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.673+0000 7fc025200640 1 --2- 172.21.15.62:0/1605257910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc020103b30 0x7fc020103fb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc008009930 tx=0x7fc00802bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:11.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.673+0000 7fc025200640 1 -- 172.21.15.62:0/1605257910 shutdown_connections 2024-04-02T14:44:11.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.673+0000 7fc025200640 1 --2- 172.21.15.62:0/1605257910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0201044f0 0x7fc0201109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.673+0000 7fc025200640 1 --2- 172.21.15.62:0/1605257910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc020103b30 0x7fc020103fb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.673+0000 7fc025200640 1 --2- 172.21.15.62:0/1605257910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc020109bf0 0x7fc020109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.673+0000 7fc025200640 1 -- 172.21.15.62:0/1605257910 >> 172.21.15.62:0/1605257910 conn(0x7fc0200ff5e0 msgr2=0x7fc020101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:11.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.673+0000 7fc025200640 1 -- 172.21.15.62:0/1605257910 shutdown_connections 2024-04-02T14:44:11.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.673+0000 7fc025200640 1 -- 172.21.15.62:0/1605257910 wait complete. 2024-04-02T14:44:11.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.674+0000 7fc025200640 1 Processor -- start 2024-04-02T14:44:11.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.674+0000 7fc025200640 1 -- start start 2024-04-02T14:44:11.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.674+0000 7fc025200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc020103b30 0x7fc02019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:11.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.674+0000 7fc025200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0201044f0 0x7fc02019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:11.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.674+0000 7fc025200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc020109bf0 0x7fc02019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:11.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.674+0000 7fc025200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc020114000 con 0x7fc0201044f0 2024-04-02T14:44:11.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.674+0000 7fc025200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc020113e80 con 0x7fc020109bf0 2024-04-02T14:44:11.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.674+0000 7fc025200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc020114180 con 0x7fc020103b30 2024-04-02T14:44:11.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.675+0000 7fc01ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0201044f0 0x7fc02019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:11.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.675+0000 7fc01ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0201044f0 0x7fc02019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36000/0 (socket says 172.21.15.62:36000) 2024-04-02T14:44:11.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.675+0000 7fc01ea00640 1 -- 172.21.15.62:0/4203905018 learned_addr learned my addr 172.21.15.62:0/4203905018 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:11.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.675+0000 7fc01f400640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc020103b30 0x7fc02019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.675+0000 7fc01fe00640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc020109bf0 0x7fc02019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.675+0000 7fc01ea00640 1 -- 172.21.15.62:0/4203905018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc020103b30 msgr2=0x7fc02019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.675+0000 7fc01ea00640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc020103b30 0x7fc02019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.675+0000 7fc01ea00640 1 -- 172.21.15.62:0/4203905018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc020109bf0 msgr2=0x7fc02019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.675+0000 7fc01ea00640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc020109bf0 0x7fc02019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.675+0000 7fc01ea00640 1 -- 172.21.15.62:0/4203905018 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02019f470 con 0x7fc0201044f0 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.676+0000 7fc01ea00640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0201044f0 0x7fc02019a9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc008009230 tx=0x7fc00802dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.676+0000 7fc017e00640 1 -- 172.21.15.62:0/4203905018 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008043020 con 0x7fc0201044f0 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.676+0000 7fc017e00640 1 -- 172.21.15.62:0/4203905018 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc008004250 con 0x7fc0201044f0 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.676+0000 7fc017e00640 1 -- 172.21.15.62:0/4203905018 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008003970 con 0x7fc0201044f0 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.676+0000 7fc025200640 1 -- 172.21.15.62:0/4203905018 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02019f700 con 0x7fc0201044f0 2024-04-02T14:44:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.676+0000 7fc025200640 1 -- 172.21.15.62:0/4203905018 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0201a6ff0 con 0x7fc0201044f0 2024-04-02T14:44:11.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.677+0000 7fc025200640 1 -- 172.21.15.62:0/4203905018 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0201051e0 con 0x7fc0201044f0 2024-04-02T14:44:11.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.679+0000 7fc017e00640 1 -- 172.21.15.62:0/4203905018 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fc008046020 con 0x7fc0201044f0 2024-04-02T14:44:11.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.682+0000 7fc017e00640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc0040a3eb0 0x7fc0040a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:11.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.682+0000 7fc017e00640 1 -- 172.21.15.62:0/4203905018 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc0080f28c0 con 0x7fc0201044f0 2024-04-02T14:44:11.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.682+0000 7fc01f400640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc0040a3eb0 0x7fc0040a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:11.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.683+0000 7fc01f400640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc0040a3eb0 0x7fc0040a6370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc010005fd0 tx=0x7fc010009500 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.687+0000 7fc017e00640 1 -- 172.21.15.62:0/4203905018 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc00804d020 con 0x7fc0201044f0 2024-04-02T14:44:11.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:11 smithi204 ceph-mon[35948]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 684 B/s wr, 20 op/s 2024-04-02T14:44:11.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:11 smithi204 ceph-mon[35948]: from='client.25490 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:11.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.917+0000 7fc025200640 1 -- 172.21.15.62:0/4203905018 --> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fc020061f40 con 0x7fc0040a3eb0 2024-04-02T14:44:11.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.935+0000 7fc017e00640 1 -- 172.21.15.62:0/4203905018 <== mgr.25273 v2:172.21.15.62:6800/110874546 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fc020061f40 con 0x7fc0040a3eb0 2024-04-02T14:44:11.933 INFO:teuthology.orchestra.run.smithi062.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-02T00:00:00", "created": "2024-04-02T14:44:10", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-02T14:44:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 -- 172.21.15.62:0/4203905018 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc0040a3eb0 msgr2=0x7fc0040a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc0040a3eb0 0x7fc0040a6370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc010005fd0 tx=0x7fc010009500 comp rx=0 tx=0).stop 2024-04-02T14:44:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 -- 172.21.15.62:0/4203905018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0201044f0 msgr2=0x7fc02019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0201044f0 0x7fc02019a9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc008009230 tx=0x7fc00802dc90 comp rx=0 tx=0).stop 2024-04-02T14:44:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 -- 172.21.15.62:0/4203905018 shutdown_connections 2024-04-02T14:44:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc0040a3eb0 0x7fc0040a6370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc020109bf0 0x7fc02019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0201044f0 0x7fc02019a9c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 --2- 172.21.15.62:0/4203905018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc020103b30 0x7fc02019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 -- 172.21.15.62:0/4203905018 >> 172.21.15.62:0/4203905018 conn(0x7fc0200ff5e0 msgr2=0x7fc020100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 -- 172.21.15.62:0/4203905018 shutdown_connections 2024-04-02T14:44:11.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:11.938+0000 7fc025200640 1 -- 172.21.15.62:0/4203905018 wait complete. 2024-04-02T14:44:11.955 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-02T14:44:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.178+0000 7fb132400640 1 -- 172.21.15.62:0/2353363597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c10dfa0 msgr2=0x7fb12c10e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.178+0000 7fb132400640 1 --2- 172.21.15.62:0/2353363597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c10dfa0 0x7fb12c10e380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb120009930 tx=0x7fb12002bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.178+0000 7fb132400640 1 -- 172.21.15.62:0/2353363597 shutdown_connections 2024-04-02T14:44:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.178+0000 7fb132400640 1 --2- 172.21.15.62:0/2353363597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c1089c0 0x7fb12c114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.178+0000 7fb132400640 1 --2- 172.21.15.62:0/2353363597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12c108040 0x7fb12c108480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.178+0000 7fb132400640 1 --2- 172.21.15.62:0/2353363597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c10dfa0 0x7fb12c10e380 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.178+0000 7fb132400640 1 -- 172.21.15.62:0/2353363597 >> 172.21.15.62:0/2353363597 conn(0x7fb12c0778e0 msgr2=0x7fb12c106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.178+0000 7fb132400640 1 -- 172.21.15.62:0/2353363597 shutdown_connections 2024-04-02T14:44:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.178+0000 7fb132400640 1 -- 172.21.15.62:0/2353363597 wait complete. 2024-04-02T14:44:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.179+0000 7fb132400640 1 Processor -- start 2024-04-02T14:44:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.179+0000 7fb132400640 1 -- start start 2024-04-02T14:44:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.180+0000 7fb132400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c108040 0x7fb12c1a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.180+0000 7fb132400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12c1089c0 0x7fb12c1a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.180+0000 7fb132400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c10dfa0 0x7fb12c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.180+0000 7fb132400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1183e0 con 0x7fb12c10dfa0 2024-04-02T14:44:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.180+0000 7fb132400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c118260 con 0x7fb12c1089c0 2024-04-02T14:44:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.180+0000 7fb132400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c118560 con 0x7fb12c108040 2024-04-02T14:44:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.180+0000 7fb131000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c108040 0x7fb12c1a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.180+0000 7fb131a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c10dfa0 0x7fb12c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.180+0000 7fb12be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12c1089c0 0x7fb12c1a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.180+0000 7fb131000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c108040 0x7fb12c1a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39538/0 (socket says 172.21.15.62:39538) 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.180+0000 7fb131000640 1 -- 172.21.15.62:0/2952302153 learned_addr learned my addr 172.21.15.62:0/2952302153 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb12be00640 1 -- 172.21.15.62:0/2952302153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c108040 msgr2=0x7fb12c1a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb12be00640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c108040 0x7fb12c1a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb12be00640 1 -- 172.21.15.62:0/2952302153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c10dfa0 msgr2=0x7fb12c19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb12be00640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c10dfa0 0x7fb12c19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb12be00640 1 -- 172.21.15.62:0/2952302153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12c19f000 con 0x7fb12c1089c0 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb131a00640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c10dfa0 0x7fb12c19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb12be00640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12c1089c0 0x7fb12c1a4bc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9f0 tx=0x7fb11800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb129600640 1 -- 172.21.15.62:0/2952302153 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11800ccc0 con 0x7fb12c1089c0 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb132400640 1 -- 172.21.15.62:0/2952302153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12c19f2f0 con 0x7fb12c1089c0 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb129600640 1 -- 172.21.15.62:0/2952302153 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb118005020 con 0x7fb12c1089c0 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb129600640 1 -- 172.21.15.62:0/2952302153 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118004b40 con 0x7fb12c1089c0 2024-04-02T14:44:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.181+0000 7fb132400640 1 -- 172.21.15.62:0/2952302153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12c1ab600 con 0x7fb12c1089c0 2024-04-02T14:44:12.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.183+0000 7fb132400640 1 -- 172.21.15.62:0/2952302153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb12c109640 con 0x7fb12c1089c0 2024-04-02T14:44:12.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.185+0000 7fb129600640 1 -- 172.21.15.62:0/2952302153 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fb118004ce0 con 0x7fb12c1089c0 2024-04-02T14:44:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.187+0000 7fb129600640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb1140a3e30 0x7fb1140a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.187+0000 7fb129600640 1 -- 172.21.15.62:0/2952302153 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fb1180d09e0 con 0x7fb12c1089c0 2024-04-02T14:44:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.187+0000 7fb131000640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb1140a3e30 0x7fb1140a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:12.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.188+0000 7fb131000640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb1140a3e30 0x7fb1140a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb120000c00 tx=0x7fb1200058d0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:12.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.190+0000 7fb129600640 1 -- 172.21.15.62:0/2952302153 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb11809ce80 con 0x7fb12c1089c0 2024-04-02T14:44:12.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.426+0000 7fb132400640 1 -- 172.21.15.62:0/2952302153 --> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb12c061f40 con 0x7fb1140a3e30 2024-04-02T14:44:12.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.438+0000 7fb129600640 1 -- 172.21.15.62:0/2952302153 <== mgr.25273 v2:172.21.15.62:6800/110874546 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb12c061f40 con 0x7fb1140a3e30 2024-04-02T14:44:12.436 INFO:teuthology.orchestra.run.smithi062.stdout:/ 1m 6m3h 2024-04-02T14:44:12.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.441+0000 7fb132400640 1 -- 172.21.15.62:0/2952302153 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb1140a3e30 msgr2=0x7fb1140a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:12.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.441+0000 7fb132400640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb1140a3e30 0x7fb1140a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb120000c00 tx=0x7fb1200058d0 comp rx=0 tx=0).stop 2024-04-02T14:44:12.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.441+0000 7fb132400640 1 -- 172.21.15.62:0/2952302153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12c1089c0 msgr2=0x7fb12c1a4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:12.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.441+0000 7fb132400640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12c1089c0 0x7fb12c1a4bc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9f0 tx=0x7fb11800eeb0 comp rx=0 tx=0).stop 2024-04-02T14:44:12.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.442+0000 7fb132400640 1 -- 172.21.15.62:0/2952302153 shutdown_connections 2024-04-02T14:44:12.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.442+0000 7fb132400640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb1140a3e30 0x7fb1140a62f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:12.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.442+0000 7fb132400640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c10dfa0 0x7fb12c19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:12.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.442+0000 7fb132400640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12c1089c0 0x7fb12c1a4bc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:12.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.442+0000 7fb132400640 1 --2- 172.21.15.62:0/2952302153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c108040 0x7fb12c1a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:12.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.442+0000 7fb132400640 1 -- 172.21.15.62:0/2952302153 >> 172.21.15.62:0/2952302153 conn(0x7fb12c0778e0 msgr2=0x7fb12c078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:12.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.442+0000 7fb132400640 1 -- 172.21.15.62:0/2952302153 shutdown_connections 2024-04-02T14:44:12.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:12.442+0000 7fb132400640 1 -- 172.21.15.62:0/2952302153 wait complete. 2024-04-02T14:44:12.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-02T14:44:12.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:12 smithi062 ceph-mon[32116]: scrub queued for path: / 2024-04-02T14:44:12.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:12 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:12.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:12 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:44:12.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:12 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:12.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:12 smithi062 ceph-mon[32116]: from='client.25526 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:12.557 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-02T14:44:12.570 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-02T14:44:12.583 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-04-02T14:44:12.583 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-02T14:44:12.591 INFO:teuthology.run_tasks:Running task workunit... 2024-04-02T14:44:12.608 INFO:tasks.workunit:Pulling workunits from ref 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:44:12.609 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-02T14:44:12.609 INFO:tasks.workunit:timeout=3h 2024-04-02T14:44:12.609 INFO:tasks.workunit:cleanup=True 2024-04-02T14:44:12.609 DEBUG:teuthology.orchestra.run.smithi062:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-02T14:44:12.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:12 smithi113 ceph-mon[36135]: scrub queued for path: / 2024-04-02T14:44:12.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:12 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:12.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:12 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:44:12.621 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:12 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:12.622 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:12 smithi113 ceph-mon[36135]: from='client.25526 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:12.660 INFO:teuthology.orchestra.run.smithi062.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-02T14:44:12.660 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-02T14:44:12.660 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 76h/118d Inode: 1099511628283 Links: 2 2024-04-02T14:44:12.660 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T14:44:12.660 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-02T14:44:12.660 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-02 14:41:30.071844605 +0000 2024-04-02T14:44:12.660 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-02 14:41:30.071844605 +0000 2024-04-02T14:44:12.660 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-02 14:43:59.219001168 +0000 2024-04-02T14:44:12.660 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-02 14:41:30.071844605 +0000 2024-04-02T14:44:12.660 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-02T14:44:12.661 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-02T14:44:12.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:12 smithi204 ceph-mon[35948]: scrub queued for path: / 2024-04-02T14:44:12.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:12 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:12.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:12 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:44:12.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:12 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:12.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:12 smithi204 ceph-mon[35948]: from='client.25526 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:12.801 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T14:44:12.860 INFO:tasks.workunit.client.0.smithi062.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-02T14:44:13.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:13 smithi062 ceph-mon[32116]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:13.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:13 smithi062 ceph-mon[32116]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 37 op/s 2024-04-02T14:44:13.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:13 smithi062 ceph-mon[32116]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:13.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:13 smithi113 ceph-mon[36135]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:13.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:13 smithi113 ceph-mon[36135]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 37 op/s 2024-04-02T14:44:13.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:13 smithi113 ceph-mon[36135]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:13.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:13 smithi204 ceph-mon[35948]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:13.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:13 smithi204 ceph-mon[35948]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 37 op/s 2024-04-02T14:44:13.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:13 smithi204 ceph-mon[35948]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:44:15.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:15 smithi062 ceph-mon[32116]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-04-02T14:44:15.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:15 smithi113 ceph-mon[36135]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-04-02T14:44:15.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:15 smithi204 ceph-mon[35948]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-04-02T14:44:17.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:17 smithi062 ceph-mon[32116]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-04-02T14:44:17.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:17 smithi113 ceph-mon[36135]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-04-02T14:44:17.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:17 smithi204 ceph-mon[35948]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-04-02T14:44:19.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:19 smithi062 ceph-mon[32116]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-02T14:44:19.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:19 smithi113 ceph-mon[36135]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-02T14:44:19.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:19 smithi204 ceph-mon[35948]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-02T14:44:21.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:21 smithi062 ceph-mon[32116]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-02T14:44:21.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:21 smithi113 ceph-mon[36135]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-02T14:44:21.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:21 smithi204 ceph-mon[35948]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-02T14:44:23.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:23 smithi062 ceph-mon[32116]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-02T14:44:23.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:23 smithi113 ceph-mon[36135]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-02T14:44:23.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:23 smithi204 ceph-mon[35948]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-02T14:44:25.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:25 smithi062 ceph-mon[32116]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T14:44:25.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:25 smithi113 ceph-mon[36135]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T14:44:25.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:25 smithi204 ceph-mon[35948]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T14:44:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.856+0000 7f4150000640 1 -- 172.21.15.62:0/1560066100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f414810e310 msgr2=0x7f4148110760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.856+0000 7f4150000640 1 --2- 172.21.15.62:0/1560066100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f414810e310 0x7f4148110760 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4140009be0 tx=0x7f414002bf80 comp rx=0 tx=0).stop 2024-04-02T14:44:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.856+0000 7f4150000640 1 -- 172.21.15.62:0/1560066100 shutdown_connections 2024-04-02T14:44:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.856+0000 7f4150000640 1 --2- 172.21.15.62:0/1560066100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f414810e310 0x7f4148110760 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.856+0000 7f4150000640 1 --2- 172.21.15.62:0/1560066100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4148074850 0x7f414810ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.856+0000 7f4150000640 1 --2- 172.21.15.62:0/1560066100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4148073f30 0x7f4148074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.856+0000 7f4150000640 1 -- 172.21.15.62:0/1560066100 >> 172.21.15.62:0/1560066100 conn(0x7f41480fd5c0 msgr2=0x7f41480ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.856+0000 7f4150000640 1 -- 172.21.15.62:0/1560066100 shutdown_connections 2024-04-02T14:44:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.856+0000 7f4150000640 1 -- 172.21.15.62:0/1560066100 wait complete. 2024-04-02T14:44:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.857+0000 7f4150000640 1 Processor -- start 2024-04-02T14:44:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f4150000640 1 -- start start 2024-04-02T14:44:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f4150000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4148073f30 0x7f414819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f4150000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4148074850 0x7f414819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f4150000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414810e310 0x7f414819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f4150000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4148114220 con 0x7f4148074850 2024-04-02T14:44:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f4150000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f41481140a0 con 0x7f4148073f30 2024-04-02T14:44:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f4150000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f41481143a0 con 0x7f414810e310 2024-04-02T14:44:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f414cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4148074850 0x7f414819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f414cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4148074850 0x7f414819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52072/0 (socket says 172.21.15.62:52072) 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f414cc00640 1 -- 172.21.15.62:0/1245531291 learned_addr learned my addr 172.21.15.62:0/1245531291 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f414e000640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414810e310 0x7f414819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.858+0000 7f414d600640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4148073f30 0x7f414819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.859+0000 7f414cc00640 1 -- 172.21.15.62:0/1245531291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414810e310 msgr2=0x7f414819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.859+0000 7f414cc00640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414810e310 0x7f414819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.859+0000 7f414cc00640 1 -- 172.21.15.62:0/1245531291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4148073f30 msgr2=0x7f414819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.859+0000 7f414cc00640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4148073f30 0x7f414819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.859+0000 7f414cc00640 1 -- 172.21.15.62:0/1245531291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414819f4e0 con 0x7f4148074850 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.859+0000 7f414cc00640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4148074850 0x7f414819aa80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f413400ea60 tx=0x7f413400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.859+0000 7f4146000640 1 -- 172.21.15.62:0/1245531291 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413400cde0 con 0x7f4148074850 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.859+0000 7f4146000640 1 -- 172.21.15.62:0/1245531291 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f41340035f0 con 0x7f4148074850 2024-04-02T14:44:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.859+0000 7f4146000640 1 -- 172.21.15.62:0/1245531291 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134010720 con 0x7f4148074850 2024-04-02T14:44:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.859+0000 7f4150000640 1 -- 172.21.15.62:0/1245531291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414819f7d0 con 0x7f4148074850 2024-04-02T14:44:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.860+0000 7f4150000640 1 -- 172.21.15.62:0/1245531291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41481a7060 con 0x7f4148074850 2024-04-02T14:44:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.863+0000 7f4152c80740 1 Processor -- start 2024-04-02T14:44:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.863+0000 7f4152c80740 1 -- start start 2024-04-02T14:44:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.863+0000 7f4152c80740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 0x55c2a724d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.863+0000 7f4146000640 1 -- 172.21.15.62:0/1245531291 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f41340037a0 con 0x7f4148074850 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.863+0000 7f4152c80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724d7d0 0x55c2a7255e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.863+0000 7f4152c80740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a72587a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.863+0000 7f4152c80740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c2a71b1d70 con 0x55c2a724d7d0 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.863+0000 7f4152c80740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c2a715a2a0 con 0x55c2a724ceb0 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.863+0000 7f4152c80740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c2a71add20 con 0x55c2a72563b0 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.864+0000 7f414e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 0x55c2a724d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.864+0000 7f414d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724d7d0 0x55c2a7255e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.864+0000 7f414e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 0x55c2a724d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37570/0 (socket says 172.21.15.62:37570) 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.864+0000 7f414e000640 1 -- 172.21.15.62:0/3552728199 learned_addr learned my addr 172.21.15.62:0/3552728199 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.864+0000 7f414cc00640 1 --2- 172.21.15.62:0/3552728199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a72587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.864+0000 7f414e000640 1 -- 172.21.15.62:0/3552728199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 msgr2=0x55c2a72587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.864+0000 7f414e000640 1 --2- 172.21.15.62:0/3552728199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a72587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.864+0000 7f414e000640 1 -- 172.21.15.62:0/3552728199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724d7d0 msgr2=0x55c2a7255e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.864+0000 7f414e000640 1 --2- 172.21.15.62:0/3552728199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724d7d0 0x55c2a7255e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.864+0000 7f414e000640 1 -- 172.21.15.62:0/3552728199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2a716e840 con 0x55c2a724ceb0 2024-04-02T14:44:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.865+0000 7f414e000640 1 --2- 172.21.15.62:0/3552728199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 0x55c2a724d290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f414002c590 tx=0x7f41400098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ffb0e1936fe145 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.865+0000 7f413a600640 1 -- 172.21.15.62:0/3552728199 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4140043030 con 0x55c2a724ceb0 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.865+0000 7f413a600640 1 -- 172.21.15.62:0/3552728199 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f414002c920 con 0x55c2a724ceb0 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.865+0000 7f413a600640 1 -- 172.21.15.62:0/3552728199 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4140004380 con 0x55c2a724ceb0 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.865+0000 7f4146000640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f41300a3f60 0x7f41300a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.865+0000 7f4146000640 1 -- 172.21.15.62:0/1245531291 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f41340d0d50 con 0x7f4148074850 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f414d600640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f41300a3f60 0x7f41300a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f4152c80740 1 -- 172.21.15.62:0/3552728199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 msgr2=0x55c2a724d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f4152c80740 1 --2- 172.21.15.62:0/3552728199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 0x55c2a724d290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f414002c590 tx=0x7f41400098d0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f4152c80740 1 -- 172.21.15.62:0/3552728199 shutdown_connections 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f4152c80740 1 --2- 172.21.15.62:0/3552728199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a72587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f4152c80740 1 --2- 172.21.15.62:0/3552728199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724d7d0 0x55c2a7255e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f4152c80740 1 --2- 172.21.15.62:0/3552728199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 0x55c2a724d290 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f4152c80740 1 -- 172.21.15.62:0/3552728199 >> 172.21.15.62:0/3552728199 conn(0x55c2a72475e0 msgr2=0x55c2a72479c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f4152c80740 1 -- 172.21.15.62:0/3552728199 shutdown_connections 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f4152c80740 1 -- 172.21.15.62:0/3552728199 wait complete. 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f414d600640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f41300a3f60 0x7f41300a6420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55c2a724c980 tx=0x7f413c009260 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.866+0000 7f4152c80740 1 -- start start 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f4152c80740 1 Processor -- start 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f4152c80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 0x55c2a7253400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f4152c80740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 0x55c2a7253940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f4152c80740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a7253e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f4152c80740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c2a7259140 con 0x55c2a724ceb0 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f4152c80740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c2a716d660 con 0x55c2a724d7d0 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f4152c80740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c2a72592c0 con 0x55c2a72563b0 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f4152c80740 10 client.0 _ll_register_callbacks cb 0x55c2a716d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f414cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 0x55c2a7253940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f414d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 0x55c2a7253400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f414d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 0x55c2a7253400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52082/0 (socket says 172.21.15.62:52082) 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.867+0000 7f414d600640 1 -- 172.21.15.62:0/3075051730 learned_addr learned my addr 172.21.15.62:0/3075051730 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.868+0000 7f414e000640 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a7253e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.868+0000 7f414cc00640 1 -- 172.21.15.62:0/3075051730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 msgr2=0x55c2a7253e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.868+0000 7f414cc00640 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a7253e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.868+0000 7f414cc00640 1 -- 172.21.15.62:0/3075051730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 msgr2=0x55c2a7253400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.868+0000 7f414cc00640 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 0x55c2a7253400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.868+0000 7f414cc00640 1 -- 172.21.15.62:0/3075051730 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2a717ce80 con 0x55c2a724d7d0 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.868+0000 7f414cc00640 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 0x55c2a7253940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f41340ce310 tx=0x7f413409c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abdfd70139bc66c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.868+0000 7f412ea00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.868+0000 7f412ea00640 1 -- 172.21.15.62:0/3075051730 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413409d190 con 0x55c2a724d7d0 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.868+0000 7f4152c80740 1 -- 172.21.15.62:0/3075051730 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f414002c920 con 0x55c2a724d7d0 2024-04-02T14:44:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.868+0000 7f412ea00640 1 -- 172.21.15.62:0/3075051730 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f4134004000 con 0x55c2a724d7d0 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f412ea00640 1 -- 172.21.15.62:0/3075051730 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f413409ca90 con 0x55c2a724d7d0 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f4152c80740 10 client.25550 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f4152c80740 1 -- 172.21.15.62:0/3075051730 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c2a6ec6210 con 0x55c2a724d7d0 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f412ea00640 1 -- 172.21.15.62:0/3075051730 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f414002c920 con 0x55c2a724d7d0 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f412ea00640 1 -- 172.21.15.62:0/3075051730 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f41340a5270 con 0x55c2a724d7d0 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f4152c80740 10 client.25550 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f4152c80740 10 client.25550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f4152c80740 20 client.25550 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f4152c80740 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55c2a7266700 0x55c2a7268af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f4152c80740 4 client.25550 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f4152c80740 1 -- 172.21.15.62:0/3075051730 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2a7161cd0 con 0x55c2a7266700 2024-04-02T14:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.869+0000 7f414d600640 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55c2a7266700 0x55c2a7268af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.870+0000 7f414d600640 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55c2a7266700 0x55c2a7268af0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75b0e8238bf00c6 server_cookie=35012ab3eb312db3 in_seq=0 out_seq=0 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.870+0000 7f412ea00640 10 client.25550 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f412ea00640 1 -- 172.21.15.62:0/3075051730 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2a7161cd0 con 0x55c2a7266700 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f412ea00640 10 client.25550 handle_command_reply: tid=0 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f4152c80740 1 client.25550 shutdown 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f4152c80740 1 -- 172.21.15.62:0/3075051730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 msgr2=0x55c2a7253940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f4152c80740 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 0x55c2a7253940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f41340ce310 tx=0x7f413409c5d0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f4152c80740 1 -- 172.21.15.62:0/3075051730 shutdown_connections 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f4152c80740 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55c2a7266700 0x55c2a7268af0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f4152c80740 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a7253e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f4152c80740 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 0x55c2a7253940 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f4152c80740 1 --2- 172.21.15.62:0/3075051730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 0x55c2a7253400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f4152c80740 1 -- 172.21.15.62:0/3075051730 >> 172.21.15.62:0/3075051730 conn(0x55c2a72475e0 msgr2=0x55c2a7249330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f412ea00640 0 client.25550 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f4152c80740 1 -- 172.21.15.62:0/3075051730 shutdown_connections 2024-04-02T14:44:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.871+0000 7f4152c80740 1 -- 172.21.15.62:0/3075051730 wait complete. 2024-04-02T14:44:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.872+0000 7f4152c80740 20 client.25550 trim_cache size 0 max 16384 2024-04-02T14:44:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.887+0000 7f4152c80740 1 Processor -- start 2024-04-02T14:44:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.887+0000 7f4152c80740 1 -- start start 2024-04-02T14:44:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f4152c80740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 0x55c2a7255b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f4152c80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724d7d0 0x55c2a7262000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f4152c80740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a726e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f4152c80740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f41481af8b0 con 0x55c2a724d7d0 2024-04-02T14:44:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f4152c80740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f41481b13f0 con 0x55c2a724ceb0 2024-04-02T14:44:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f4152c80740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f41481aff80 con 0x55c2a72563b0 2024-04-02T14:44:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f414e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a726e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f414e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a726e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:47722/0 (socket says 172.21.15.62:47722) 2024-04-02T14:44:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f414e000640 1 -- 172.21.15.62:0/4179493584 learned_addr learned my addr 172.21.15.62:0/4179493584 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f414cc00640 1 --2- 172.21.15.62:0/4179493584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724d7d0 0x55c2a7262000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f414d600640 1 --2- 172.21.15.62:0/4179493584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 0x55c2a7255b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f414e000640 1 -- 172.21.15.62:0/4179493584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 msgr2=0x55c2a7255b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.888+0000 7f414e000640 1 --2- 172.21.15.62:0/4179493584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 0x55c2a7255b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.889+0000 7f414e000640 1 -- 172.21.15.62:0/4179493584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724d7d0 msgr2=0x55c2a7262000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.889+0000 7f414e000640 1 --2- 172.21.15.62:0/4179493584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724d7d0 0x55c2a7262000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.889+0000 7f414e000640 1 -- 172.21.15.62:0/4179493584 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2a6f92620 con 0x55c2a72563b0 2024-04-02T14:44:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.889+0000 7f414e000640 1 --2- 172.21.15.62:0/4179493584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a726e400 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4140004150 tx=0x7f4140004220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db8929fe2626d0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.889+0000 7f4139c00640 1 -- 172.21.15.62:0/4179493584 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4140043040 con 0x55c2a72563b0 2024-04-02T14:44:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.890+0000 7f4139c00640 1 -- 172.21.15.62:0/4179493584 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f414002ce20 con 0x55c2a72563b0 2024-04-02T14:44:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.890+0000 7f4139c00640 1 -- 172.21.15.62:0/4179493584 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41400335c0 con 0x55c2a72563b0 2024-04-02T14:44:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.890+0000 7f4152c80740 1 -- 172.21.15.62:0/4179493584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 msgr2=0x55c2a726e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.890+0000 7f4152c80740 1 --2- 172.21.15.62:0/4179493584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a726e400 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4140004150 tx=0x7f4140004220 comp rx=0 tx=0).stop 2024-04-02T14:44:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.890+0000 7f4152c80740 1 -- 172.21.15.62:0/4179493584 shutdown_connections 2024-04-02T14:44:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.890+0000 7f4152c80740 1 --2- 172.21.15.62:0/4179493584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a726e400 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.890+0000 7f4152c80740 1 --2- 172.21.15.62:0/4179493584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724d7d0 0x55c2a7262000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.890+0000 7f4152c80740 1 --2- 172.21.15.62:0/4179493584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724ceb0 0x55c2a7255b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.890+0000 7f4152c80740 1 -- 172.21.15.62:0/4179493584 >> 172.21.15.62:0/4179493584 conn(0x55c2a72475e0 msgr2=0x55c2a7253c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.890+0000 7f4152c80740 1 -- 172.21.15.62:0/4179493584 shutdown_connections 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.890+0000 7f4152c80740 1 -- 172.21.15.62:0/4179493584 wait complete. 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.891+0000 7f4152c80740 1 -- start start 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.891+0000 7f4152c80740 1 Processor -- start 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.891+0000 7f4152c80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 0x55c2a725e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.891+0000 7f4152c80740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 0x55c2a725ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.891+0000 7f4152c80740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a725b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.891+0000 7f4152c80740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f41481b0d20 con 0x55c2a724ceb0 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.891+0000 7f4152c80740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f41481b0650 con 0x55c2a724d7d0 2024-04-02T14:44:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.891+0000 7f4152c80740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f41481ad890 con 0x55c2a72563b0 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.891+0000 7f4152c80740 10 client.0 _ll_register_callbacks cb 0x7f4134098d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 0x55c2a725e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 0x55c2a725e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52112/0 (socket says 172.21.15.62:52112) 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414d600640 1 -- 172.21.15.62:0/130371629 learned_addr learned my addr 172.21.15.62:0/130371629 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414cc00640 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 0x55c2a725ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414e000640 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a725b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414d600640 1 -- 172.21.15.62:0/130371629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 msgr2=0x55c2a725b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414d600640 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a725b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414d600640 1 -- 172.21.15.62:0/130371629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 msgr2=0x55c2a725ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414d600640 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 0x55c2a725ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414d600640 1 -- 172.21.15.62:0/130371629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2a725bd70 con 0x55c2a724ceb0 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414cc00640 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 0x55c2a725ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f414d600640 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 0x55c2a725e510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f413c009fa0 tx=0x7f413c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465d68f9a3b95365 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.892+0000 7f412ea00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.893+0000 7f412ea00640 1 -- 172.21.15.62:0/130371629 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c004020 con 0x55c2a724ceb0 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.893+0000 7f412ea00640 1 -- 172.21.15.62:0/130371629 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f413c0041c0 con 0x55c2a724ceb0 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.893+0000 7f4152c80740 1 -- 172.21.15.62:0/130371629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f414002ce20 con 0x55c2a724ceb0 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.893+0000 7f412ea00640 1 -- 172.21.15.62:0/130371629 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f413c039050 con 0x55c2a724ceb0 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.893+0000 7f4152c80740 10 client.15423 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.893+0000 7f4152c80740 1 -- 172.21.15.62:0/130371629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c2a72cd870 con 0x55c2a724ceb0 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.893+0000 7f412ea00640 1 -- 172.21.15.62:0/130371629 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f414002ce20 con 0x55c2a724ceb0 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.893+0000 7f412ea00640 1 -- 172.21.15.62:0/130371629 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f413c02b3d0 con 0x55c2a724ceb0 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.893+0000 7f4152c80740 10 client.15423 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.893+0000 7f4152c80740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.893+0000 7f4152c80740 20 client.15423 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.894+0000 7f4152c80740 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55c2a72cdf90 0x55c2a72d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.894+0000 7f4152c80740 4 client.15423 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.894+0000 7f4152c80740 1 -- 172.21.15.62:0/130371629 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2a71bf2b0 con 0x55c2a72cdf90 2024-04-02T14:44:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.894+0000 7f414cc00640 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55c2a72cdf90 0x55c2a72d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.894+0000 7f414cc00640 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55c2a72cdf90 0x55c2a72d03d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fba69300fb8a8e8 server_cookie=1d0148098a3c0ea6 in_seq=0 out_seq=0 2024-04-02T14:44:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.894+0000 7f412ea00640 10 client.15423 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f412ea00640 1 -- 172.21.15.62:0/130371629 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2a71bf2b0 con 0x55c2a72cdf90 2024-04-02T14:44:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f412ea00640 10 client.15423 handle_command_reply: tid=0 2024-04-02T14:44:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 1 client.15423 shutdown 2024-04-02T14:44:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 1 -- 172.21.15.62:0/130371629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 msgr2=0x55c2a725e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 0x55c2a725e510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f413c009fa0 tx=0x7f413c0046c0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 1 -- 172.21.15.62:0/130371629 shutdown_connections 2024-04-02T14:44:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55c2a72cdf90 0x55c2a72d03d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a72563b0 0x55c2a725b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f412ea00640 0 client.15423 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2a724d7d0 0x55c2a725ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 1 --2- 172.21.15.62:0/130371629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a724ceb0 0x55c2a725e510 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 1 -- 172.21.15.62:0/130371629 >> 172.21.15.62:0/130371629 conn(0x55c2a72475e0 msgr2=0x55c2a72533c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 1 -- 172.21.15.62:0/130371629 shutdown_connections 2024-04-02T14:44:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 1 -- 172.21.15.62:0/130371629 wait complete. 2024-04-02T14:44:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.907+0000 7f4152c80740 20 client.15423 trim_cache size 0 max 16384 2024-04-02T14:44:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:44:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:44:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:44:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:44:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 -- 172.21.15.62:0/1245531291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f41300a3f60 msgr2=0x7f41300a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f41300a3f60 0x7f41300a6420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55c2a724c980 tx=0x7f413c009260 comp rx=0 tx=0).stop 2024-04-02T14:44:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 -- 172.21.15.62:0/1245531291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4148074850 msgr2=0x7f414819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4148074850 0x7f414819aa80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f413400ea60 tx=0x7f413400ef20 comp rx=0 tx=0).stop 2024-04-02T14:44:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 -- 172.21.15.62:0/1245531291 shutdown_connections 2024-04-02T14:44:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f41300a3f60 0x7f41300a6420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414810e310 0x7f414819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4148074850 0x7f414819aa80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 --2- 172.21.15.62:0/1245531291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4148073f30 0x7f414819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 -- 172.21.15.62:0/1245531291 >> 172.21.15.62:0/1245531291 conn(0x7f41480fd5c0 msgr2=0x7f414810fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 -- 172.21.15.62:0/1245531291 shutdown_connections 2024-04-02T14:44:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:25.908+0000 7f4150000640 1 -- 172.21.15.62:0/1245531291 wait complete. 2024-04-02T14:44:26.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:26 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:44:26.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:26 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:44:26.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:26 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:44:27.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:27 smithi062 ceph-mon[32116]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T14:44:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:27 smithi113 ceph-mon[36135]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T14:44:27.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:27 smithi204 ceph-mon[35948]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T14:44:29.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:29 smithi062 ceph-mon[32116]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 34 op/s 2024-04-02T14:44:29.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:29 smithi113 ceph-mon[36135]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 34 op/s 2024-04-02T14:44:29.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:29 smithi204 ceph-mon[35948]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 34 op/s 2024-04-02T14:44:31.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:31 smithi062 ceph-mon[32116]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T14:44:31.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:31 smithi113 ceph-mon[36135]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T14:44:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:31 smithi204 ceph-mon[35948]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T14:44:33.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:33 smithi062 ceph-mon[32116]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-02T14:44:33.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:33 smithi113 ceph-mon[36135]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-02T14:44:33.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:33 smithi204 ceph-mon[35948]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-02T14:44:35.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:35 smithi062 ceph-mon[32116]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T14:44:35.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:35 smithi113 ceph-mon[36135]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T14:44:35.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:35 smithi204 ceph-mon[35948]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T14:44:37.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:37 smithi113 ceph-mon[36135]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-02T14:44:37.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:37 smithi204 ceph-mon[35948]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-02T14:44:37.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:37 smithi062 ceph-mon[32116]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-02T14:44:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.053+0000 7f597f200640 1 -- 172.21.15.62:0/1496142764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5978104900 msgr2=0x7f5978108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.053+0000 7f597f200640 1 --2- 172.21.15.62:0/1496142764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5978104900 0x7f5978108940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5964009900 tx=0x7f596402bc40 comp rx=0 tx=0).stop 2024-04-02T14:44:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.054+0000 7f597f200640 1 -- 172.21.15.62:0/1496142764 shutdown_connections 2024-04-02T14:44:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.054+0000 7f597f200640 1 --2- 172.21.15.62:0/1496142764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5978109020 0x7f5978110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.054+0000 7f597f200640 1 --2- 172.21.15.62:0/1496142764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5978104900 0x7f5978108940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.054+0000 7f597f200640 1 --2- 172.21.15.62:0/1496142764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5978103f50 0x7f5978104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.054+0000 7f597f200640 1 -- 172.21.15.62:0/1496142764 >> 172.21.15.62:0/1496142764 conn(0x7f59780ff7b0 msgr2=0x7f5978101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.054+0000 7f597f200640 1 -- 172.21.15.62:0/1496142764 shutdown_connections 2024-04-02T14:44:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.054+0000 7f597f200640 1 -- 172.21.15.62:0/1496142764 wait complete. 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.055+0000 7f597f200640 1 Processor -- start 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.055+0000 7f597f200640 1 -- start start 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.055+0000 7f597f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5978103f50 0x7f597819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.055+0000 7f597f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5978109020 0x7f597819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.055+0000 7f597f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597819eef0 0x7f597819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.055+0000 7f597f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f59781141f0 con 0x7f5978109020 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.055+0000 7f597f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5978114070 con 0x7f5978103f50 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.055+0000 7f597f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5978114370 con 0x7f597819eef0 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.055+0000 7f5977400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5978109020 0x7f597819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.056+0000 7f5977e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5978103f50 0x7f597819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.056+0000 7f5977400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5978109020 0x7f597819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50386/0 (socket says 172.21.15.62:50386) 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.056+0000 7f5977400640 1 -- 172.21.15.62:0/3191981555 learned_addr learned my addr 172.21.15.62:0/3191981555 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.056+0000 7f597d200640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597819eef0 0x7f597819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.056+0000 7f5977400640 1 -- 172.21.15.62:0/3191981555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597819eef0 msgr2=0x7f597819f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.056+0000 7f5977400640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597819eef0 0x7f597819f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.056+0000 7f5977400640 1 -- 172.21.15.62:0/3191981555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5978103f50 msgr2=0x7f597819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.056+0000 7f5977400640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5978103f50 0x7f597819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.056+0000 7f5977400640 1 -- 172.21.15.62:0/3191981555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597819f9e0 con 0x7f5978109020 2024-04-02T14:44:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.056+0000 7f5977400640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5978109020 0x7f597819ab70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5964036040 tx=0x7f596402dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.057+0000 7f5974c00640 1 -- 172.21.15.62:0/3191981555 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964043020 con 0x7f5978109020 2024-04-02T14:44:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.057+0000 7f5974c00640 1 -- 172.21.15.62:0/3191981555 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f596402de20 con 0x7f5978109020 2024-04-02T14:44:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.057+0000 7f597f200640 1 -- 172.21.15.62:0/3191981555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597819fc70 con 0x7f5978109020 2024-04-02T14:44:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.057+0000 7f5974c00640 1 -- 172.21.15.62:0/3191981555 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964003a80 con 0x7f5978109020 2024-04-02T14:44:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.057+0000 7f597f200640 1 -- 172.21.15.62:0/3191981555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59781a7490 con 0x7f5978109020 2024-04-02T14:44:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.060+0000 7f5981fc9740 1 Processor -- start 2024-04-02T14:44:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.060+0000 7f5974c00640 1 -- 172.21.15.62:0/3191981555 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f5964046020 con 0x7f5978109020 2024-04-02T14:44:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.060+0000 7f5981fc9740 1 -- start start 2024-04-02T14:44:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.061+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 0x562b7caf1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.061+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf17d0 0x562b7caf9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.061+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 0x562b7cafc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.061+0000 7f5981fc9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562b7ca55d70 con 0x562b7cafa3b0 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.061+0000 7f5981fc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562b7c9fe2a0 con 0x562b7caf0eb0 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.061+0000 7f5981fc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562b7ca51d20 con 0x562b7caf17d0 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.061+0000 7f597d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 0x562b7caf1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.061+0000 7f5977400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 0x562b7cafc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.061+0000 7f597d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 0x562b7caf1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:42518/0 (socket says 172.21.15.62:42518) 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.061+0000 7f597d200640 1 -- 172.21.15.62:0/4262892593 learned_addr learned my addr 172.21.15.62:0/4262892593 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.061+0000 7f5977e00640 1 --2- 172.21.15.62:0/4262892593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf17d0 0x562b7caf9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f597d200640 1 -- 172.21.15.62:0/4262892593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf17d0 msgr2=0x562b7caf9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f597d200640 1 --2- 172.21.15.62:0/4262892593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf17d0 0x562b7caf9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f597d200640 1 -- 172.21.15.62:0/4262892593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 msgr2=0x562b7cafc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f597d200640 1 --2- 172.21.15.62:0/4262892593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 0x562b7cafc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f597d200640 1 -- 172.21.15.62:0/4262892593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b7ca12840 con 0x562b7caf0eb0 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f5977e00640 1 --2- 172.21.15.62:0/4262892593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf17d0 0x562b7caf9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f5974c00640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f595c0a3ee0 0x7f595c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f597d200640 1 --2- 172.21.15.62:0/4262892593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 0x562b7caf1290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5970009c20 tx=0x7f5970011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a48adb5d9e66f77f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f5974c00640 1 -- 172.21.15.62:0/3191981555 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f59640f28e0 con 0x7f5978109020 2024-04-02T14:44:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f596d600640 1 -- 172.21.15.62:0/4262892593 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597001e070 con 0x562b7caf0eb0 2024-04-02T14:44:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f5977e00640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f595c0a3ee0 0x7f595c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f596d600640 1 -- 172.21.15.62:0/4262892593 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5970019b30 con 0x562b7caf0eb0 2024-04-02T14:44:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.062+0000 7f596d600640 1 -- 172.21.15.62:0/4262892593 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597000eae0 con 0x562b7caf0eb0 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.063+0000 7f5981fc9740 1 -- 172.21.15.62:0/4262892593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 msgr2=0x562b7caf1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.063+0000 7f5977e00640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f595c0a3ee0 0x7f595c0a63a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x562b7caf0980 tx=0x7f5968009500 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.063+0000 7f5981fc9740 1 --2- 172.21.15.62:0/4262892593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 0x562b7caf1290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5970009c20 tx=0x7f5970011810 comp rx=0 tx=0).stop 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.064+0000 7f5981fc9740 1 -- 172.21.15.62:0/4262892593 shutdown_connections 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.064+0000 7f5981fc9740 1 --2- 172.21.15.62:0/4262892593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 0x562b7cafc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.064+0000 7f5981fc9740 1 --2- 172.21.15.62:0/4262892593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf17d0 0x562b7caf9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.064+0000 7f5981fc9740 1 --2- 172.21.15.62:0/4262892593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 0x562b7caf1290 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.064+0000 7f5981fc9740 1 -- 172.21.15.62:0/4262892593 >> 172.21.15.62:0/4262892593 conn(0x562b7caeb5e0 msgr2=0x562b7caeb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.064+0000 7f5981fc9740 1 -- 172.21.15.62:0/4262892593 shutdown_connections 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.064+0000 7f5981fc9740 1 -- 172.21.15.62:0/4262892593 wait complete. 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.064+0000 7f5981fc9740 1 -- start start 2024-04-02T14:44:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.064+0000 7f5981fc9740 1 Processor -- start 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 0x562b7caf73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7caf7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7cafa3b0 0x562b7caf7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f5981fc9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562b7cafd1a0 con 0x562b7caf17d0 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f5981fc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562b7ca11660 con 0x562b7caf0eb0 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f5981fc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562b7cafd320 con 0x562b7cafa3b0 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f5981fc9740 10 client.0 _ll_register_callbacks cb 0x562b7ca11050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f5977400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7caf7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f597d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7cafa3b0 0x562b7caf7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f5977e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 0x562b7caf73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f5977400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7caf7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50398/0 (socket says 172.21.15.62:50398) 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.065+0000 7f5977400640 1 -- 172.21.15.62:0/4042650451 learned_addr learned my addr 172.21.15.62:0/4042650451 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5977400640 1 -- 172.21.15.62:0/4042650451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7cafa3b0 msgr2=0x562b7caf7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5977400640 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7cafa3b0 0x562b7caf7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5977400640 1 -- 172.21.15.62:0/4042650451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 msgr2=0x562b7caf73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5977400640 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 0x562b7caf73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5977400640 1 -- 172.21.15.62:0/4042650451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b7ca20e80 con 0x562b7caf17d0 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f597d200640 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7cafa3b0 0x562b7caf7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5977e00640 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 0x562b7caf73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5977400640 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7caf7920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5964005ec0 tx=0x7f59640ba3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68992c0f1dd41cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5962000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5962000640 1 -- 172.21.15.62:0/4042650451 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59640be0f0 con 0x562b7caf17d0 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5962000640 1 -- 172.21.15.62:0/4042650451 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f59640be9f0 con 0x562b7caf17d0 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5981fc9740 1 -- 172.21.15.62:0/4042650451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5970019b30 con 0x562b7caf17d0 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5962000640 1 -- 172.21.15.62:0/4042650451 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f59640bee90 con 0x562b7caf17d0 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5981fc9740 10 client.15435 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.066+0000 7f5981fc9740 1 -- 172.21.15.62:0/4042650451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b7c76a210 con 0x562b7caf17d0 2024-04-02T14:44:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.067+0000 7f5962000640 1 -- 172.21.15.62:0/4042650451 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5970019b30 con 0x562b7caf17d0 2024-04-02T14:44:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.067+0000 7f5962000640 1 -- 172.21.15.62:0/4042650451 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f59640c6f70 con 0x562b7caf17d0 2024-04-02T14:44:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.067+0000 7f5981fc9740 10 client.15435 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.067+0000 7f5981fc9740 10 client.15435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.067+0000 7f5981fc9740 20 client.15435 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.067+0000 7f5981fc9740 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x562b7cb0a6e0 0x562b7cb0cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.067+0000 7f5981fc9740 4 client.15435 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.067+0000 7f5981fc9740 1 -- 172.21.15.62:0/4042650451 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b7ca05cd0 con 0x562b7cb0a6e0 2024-04-02T14:44:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.067+0000 7f5977e00640 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x562b7cb0a6e0 0x562b7cb0cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.068+0000 7f5977e00640 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x562b7cb0a6e0 0x562b7cb0cb60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1ddeaa5596066a server_cookie=6683ad6903ce8cc0 in_seq=0 out_seq=0 2024-04-02T14:44:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.068+0000 7f5962000640 10 client.15435 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5962000640 1 -- 172.21.15.62:0/4042650451 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b7ca05cd0 con 0x562b7cb0a6e0 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5962000640 10 client.15435 handle_command_reply: tid=0 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 1 client.15435 shutdown 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 1 -- 172.21.15.62:0/4042650451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 msgr2=0x562b7caf7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7caf7920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5964005ec0 tx=0x7f59640ba3f0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 1 -- 172.21.15.62:0/4042650451 shutdown_connections 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x562b7cb0a6e0 0x562b7cb0cb60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7cafa3b0 0x562b7caf7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7caf7920 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 1 --2- 172.21.15.62:0/4042650451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf0eb0 0x562b7caf73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5962000640 0 client.15435 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 1 -- 172.21.15.62:0/4042650451 >> 172.21.15.62:0/4042650451 conn(0x562b7caeb5e0 msgr2=0x562b7caed330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 1 -- 172.21.15.62:0/4042650451 shutdown_connections 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 1 -- 172.21.15.62:0/4042650451 wait complete. 2024-04-02T14:44:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.069+0000 7f5981fc9740 20 client.15435 trim_cache size 0 max 16384 2024-04-02T14:44:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.086+0000 7f5981fc9740 1 Processor -- start 2024-04-02T14:44:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.086+0000 7f5981fc9740 1 -- start start 2024-04-02T14:44:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.086+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf0eb0 0x562b7caf9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.086+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7cb06090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.086+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7cafa3b0 0x562b7cb12470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.086+0000 7f5981fc9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f59781af980 con 0x562b7caf17d0 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f5981fc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f59781b14c0 con 0x562b7cafa3b0 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f5981fc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59781b0050 con 0x562b7caf0eb0 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f5977400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7cb06090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f5977400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7cb06090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50402/0 (socket says 172.21.15.62:50402) 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f5977400640 1 -- 172.21.15.62:0/607599195 learned_addr learned my addr 172.21.15.62:0/607599195 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f5977e00640 1 --2- 172.21.15.62:0/607599195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf0eb0 0x562b7caf9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f597d200640 1 --2- 172.21.15.62:0/607599195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7cafa3b0 0x562b7cb12470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f5977400640 1 -- 172.21.15.62:0/607599195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf0eb0 msgr2=0x562b7caf9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f5977400640 1 --2- 172.21.15.62:0/607599195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf0eb0 0x562b7caf9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f5977400640 1 -- 172.21.15.62:0/607599195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7cafa3b0 msgr2=0x562b7cb12470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f5977400640 1 --2- 172.21.15.62:0/607599195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7cafa3b0 0x562b7cb12470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f5977400640 1 -- 172.21.15.62:0/607599195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b7c836620 con 0x562b7caf17d0 2024-04-02T14:44:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.087+0000 7f597d200640 1 --2- 172.21.15.62:0/607599195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7cafa3b0 0x562b7cb12470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.088+0000 7f5977400640 1 --2- 172.21.15.62:0/607599195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7cb06090 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f59640bb160 tx=0x7f59640056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f081703ec1de780c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.088+0000 7f596cc00640 1 -- 172.21.15.62:0/607599195 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59640db070 con 0x562b7caf17d0 2024-04-02T14:44:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.088+0000 7f596cc00640 1 -- 172.21.15.62:0/607599195 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f59640eff50 con 0x562b7caf17d0 2024-04-02T14:44:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.088+0000 7f596cc00640 1 -- 172.21.15.62:0/607599195 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964003c20 con 0x562b7caf17d0 2024-04-02T14:44:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.088+0000 7f5981fc9740 1 -- 172.21.15.62:0/607599195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 msgr2=0x562b7cb06090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.088+0000 7f5981fc9740 1 --2- 172.21.15.62:0/607599195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7cb06090 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f59640bb160 tx=0x7f59640056f0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.089+0000 7f5981fc9740 1 -- 172.21.15.62:0/607599195 shutdown_connections 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.089+0000 7f5981fc9740 1 --2- 172.21.15.62:0/607599195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7cafa3b0 0x562b7cb12470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.089+0000 7f5981fc9740 1 --2- 172.21.15.62:0/607599195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7caf17d0 0x562b7cb06090 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.089+0000 7f5981fc9740 1 --2- 172.21.15.62:0/607599195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf0eb0 0x562b7caf9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.089+0000 7f5981fc9740 1 -- 172.21.15.62:0/607599195 >> 172.21.15.62:0/607599195 conn(0x562b7caeb5e0 msgr2=0x562b7caf1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.089+0000 7f5981fc9740 1 -- 172.21.15.62:0/607599195 shutdown_connections 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.089+0000 7f5981fc9740 1 -- 172.21.15.62:0/607599195 wait complete. 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.089+0000 7f5981fc9740 1 -- start start 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.089+0000 7f5981fc9740 1 Processor -- start 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf0eb0 0x562b7cb02950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf17d0 0x562b7caf3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f5981fc9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 0x562b7caf42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f5981fc9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f59781b0df0 con 0x562b7cafa3b0 2024-04-02T14:44:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f5981fc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f59781b0720 con 0x562b7caf17d0 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f5981fc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59781ad960 con 0x562b7caf0eb0 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f5981fc9740 10 client.0 _ll_register_callbacks cb 0x7f5964005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f597d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 0x562b7caf42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f597d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 0x562b7caf42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50412/0 (socket says 172.21.15.62:50412) 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f597d200640 1 -- 172.21.15.62:0/1509616413 learned_addr learned my addr 172.21.15.62:0/1509616413 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f5977e00640 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf0eb0 0x562b7cb02950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f5977400640 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf17d0 0x562b7caf3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f597d200640 1 -- 172.21.15.62:0/1509616413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf0eb0 msgr2=0x562b7cb02950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f597d200640 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf0eb0 0x562b7cb02950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f597d200640 1 -- 172.21.15.62:0/1509616413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf17d0 msgr2=0x562b7caf3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f597d200640 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf17d0 0x562b7caf3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.090+0000 7f597d200640 1 -- 172.21.15.62:0/1509616413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b7caf4ae0 con 0x562b7cafa3b0 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.091+0000 7f597d200640 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 0x562b7caf42e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5970009bc0 tx=0x7f5970005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf0b8c4cc240059 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.091+0000 7f5962000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.091+0000 7f5962000640 1 -- 172.21.15.62:0/1509616413 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970028030 con 0x562b7cafa3b0 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.091+0000 7f5962000640 1 -- 172.21.15.62:0/1509616413 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f5970004020 con 0x562b7cafa3b0 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.091+0000 7f5981fc9740 1 -- 172.21.15.62:0/1509616413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59640eff50 con 0x562b7cafa3b0 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.091+0000 7f5962000640 1 -- 172.21.15.62:0/1509616413 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f597001e040 con 0x562b7cafa3b0 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.091+0000 7f5981fc9740 10 client.15441 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.091+0000 7f5981fc9740 1 -- 172.21.15.62:0/1509616413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b7cb70cc0 con 0x562b7cafa3b0 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.092+0000 7f5962000640 1 -- 172.21.15.62:0/1509616413 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f59640eff50 con 0x562b7cafa3b0 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.092+0000 7f5962000640 1 -- 172.21.15.62:0/1509616413 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5970019c80 con 0x562b7cafa3b0 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.092+0000 7f5981fc9740 10 client.15441 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.092+0000 7f5981fc9740 10 client.15441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.092+0000 7f5981fc9740 20 client.15441 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.092+0000 7f5981fc9740 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x562b7cb71220 0x562b7cb73640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.092+0000 7f5981fc9740 4 client.15441 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:44:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.092+0000 7f5981fc9740 1 -- 172.21.15.62:0/1509616413 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b7ca632b0 con 0x562b7cb71220 2024-04-02T14:44:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.092+0000 7f5977e00640 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x562b7cb71220 0x562b7cb73640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.093+0000 7f5977e00640 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x562b7cb71220 0x562b7cb73640 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea022b64c0d8bab8 server_cookie=737096ddfbc0f4d4 in_seq=0 out_seq=0 2024-04-02T14:44:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.093+0000 7f5962000640 10 client.15441 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.099+0000 7f5962000640 1 -- 172.21.15.62:0/1509616413 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b7ca632b0 con 0x562b7cb71220 2024-04-02T14:44:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.099+0000 7f5962000640 10 client.15441 handle_command_reply: tid=0 2024-04-02T14:44:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 1 client.15441 shutdown 2024-04-02T14:44:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 1 -- 172.21.15.62:0/1509616413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 msgr2=0x562b7caf42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 0x562b7caf42e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5970009bc0 tx=0x7f5970005540 comp rx=0 tx=0).stop 2024-04-02T14:44:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 1 -- 172.21.15.62:0/1509616413 shutdown_connections 2024-04-02T14:44:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x562b7cb71220 0x562b7cb73640 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b7cafa3b0 0x562b7caf42e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562b7caf17d0 0x562b7caf3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5962000640 0 client.15441 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 1 --2- 172.21.15.62:0/1509616413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b7caf0eb0 0x562b7cb02950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 1 -- 172.21.15.62:0/1509616413 >> 172.21.15.62:0/1509616413 conn(0x562b7caeb5e0 msgr2=0x562b7caf73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 1 -- 172.21.15.62:0/1509616413 shutdown_connections 2024-04-02T14:44:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 1 -- 172.21.15.62:0/1509616413 wait complete. 2024-04-02T14:44:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.100+0000 7f5981fc9740 20 client.15441 trim_cache size 0 max 16384 2024-04-02T14:44:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:44:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:44:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:44:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:44:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 -- 172.21.15.62:0/3191981555 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f595c0a3ee0 msgr2=0x7f595c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f595c0a3ee0 0x7f595c0a63a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x562b7caf0980 tx=0x7f5968009500 comp rx=0 tx=0).stop 2024-04-02T14:44:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 -- 172.21.15.62:0/3191981555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5978109020 msgr2=0x7f597819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5978109020 0x7f597819ab70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5964036040 tx=0x7f596402dc00 comp rx=0 tx=0).stop 2024-04-02T14:44:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 -- 172.21.15.62:0/3191981555 shutdown_connections 2024-04-02T14:44:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f595c0a3ee0 0x7f595c0a63a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597819eef0 0x7f597819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5978109020 0x7f597819ab70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 --2- 172.21.15.62:0/3191981555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5978103f50 0x7f597819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 -- 172.21.15.62:0/3191981555 >> 172.21.15.62:0/3191981555 conn(0x7f59780ff7b0 msgr2=0x7f5978100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 -- 172.21.15.62:0/3191981555 shutdown_connections 2024-04-02T14:44:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:39.101+0000 7f597f200640 1 -- 172.21.15.62:0/3191981555 wait complete. 2024-04-02T14:44:39.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:39 smithi113 ceph-mon[36135]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-02T14:44:39.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:39 smithi204 ceph-mon[35948]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-02T14:44:39.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:39 smithi062 ceph-mon[32116]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-02T14:44:40.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:44:40.738 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:44:40.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.984+0000 7f72d5600640 1 -- 172.21.15.62:0/2808912634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d0108c60 msgr2=0x7f72d010cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:40.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.984+0000 7f72d5600640 1 --2- 172.21.15.62:0/2808912634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d0108c60 0x7f72d010cc20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009900 tx=0x7f72b802bc40 comp rx=0 tx=0).stop 2024-04-02T14:44:40.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.984+0000 7f72d5600640 1 -- 172.21.15.62:0/2808912634 shutdown_connections 2024-04-02T14:44:40.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.984+0000 7f72d5600640 1 --2- 172.21.15.62:0/2808912634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d010d300 0x7f72d0114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.984+0000 7f72d5600640 1 --2- 172.21.15.62:0/2808912634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d0108c60 0x7f72d010cc20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.984+0000 7f72d5600640 1 --2- 172.21.15.62:0/2808912634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d01082b0 0x7f72d0108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.984+0000 7f72d5600640 1 -- 172.21.15.62:0/2808912634 >> 172.21.15.62:0/2808912634 conn(0x7f72d0077920 msgr2=0x7f72d0106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:40.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.984+0000 7f72d5600640 1 -- 172.21.15.62:0/2808912634 shutdown_connections 2024-04-02T14:44:40.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.984+0000 7f72d5600640 1 -- 172.21.15.62:0/2808912634 wait complete. 2024-04-02T14:44:40.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.985+0000 7f72d5600640 1 Processor -- start 2024-04-02T14:44:40.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.985+0000 7f72d5600640 1 -- start start 2024-04-02T14:44:40.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.985+0000 7f72d5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d01082b0 0x7f72d01a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:40.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.985+0000 7f72d5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d010d300 0x7f72d01a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:40.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.985+0000 7f72d5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d019e790 0x7f72d019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:40.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.985+0000 7f72d5600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01184d0 con 0x7f72d01082b0 2024-04-02T14:44:40.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.985+0000 7f72d5600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0118350 con 0x7f72d010d300 2024-04-02T14:44:40.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.985+0000 7f72d5600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0118650 con 0x7f72d019e790 2024-04-02T14:44:40.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d01082b0 0x7f72d01a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72c7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d010d300 0x7f72d01a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d01082b0 0x7f72d01a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50436/0 (socket says 172.21.15.62:50436) 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72cfe00640 1 -- 172.21.15.62:0/1212154618 learned_addr learned my addr 172.21.15.62:0/1212154618 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72d4c00640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d019e790 0x7f72d019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72c7e00640 1 -- 172.21.15.62:0/1212154618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d019e790 msgr2=0x7f72d019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72c7e00640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d019e790 0x7f72d019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72c7e00640 1 -- 172.21.15.62:0/1212154618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d01082b0 msgr2=0x7f72d01a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72c7e00640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d01082b0 0x7f72d01a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72c7e00640 1 -- 172.21.15.62:0/1212154618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d019f320 con 0x7f72d010d300 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72d4c00640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d019e790 0x7f72d019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.986+0000 7f72cfe00640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d01082b0 0x7f72d01a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.987+0000 7f72c7e00640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d010d300 0x7f72d01a4b60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f72b802c180 tx=0x7f72b802dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.987+0000 7f72cd600640 1 -- 172.21.15.62:0/1212154618 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8043020 con 0x7f72d010d300 2024-04-02T14:44:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.987+0000 7f72cd600640 1 -- 172.21.15.62:0/1212154618 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72b8004e50 con 0x7f72d010d300 2024-04-02T14:44:40.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.987+0000 7f72d5600640 1 -- 172.21.15.62:0/1212154618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d019f5b0 con 0x7f72d010d300 2024-04-02T14:44:40.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.987+0000 7f72cd600640 1 -- 172.21.15.62:0/1212154618 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8003a10 con 0x7f72d010d300 2024-04-02T14:44:40.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.987+0000 7f72d5600640 1 -- 172.21.15.62:0/1212154618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d01ab760 con 0x7f72d010d300 2024-04-02T14:44:40.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.990+0000 7f72cd600640 1 -- 172.21.15.62:0/1212154618 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f72b8046020 con 0x7f72d010d300 2024-04-02T14:44:40.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.990+0000 7f72d99d6740 1 Processor -- start 2024-04-02T14:44:40.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.990+0000 7f72d99d6740 1 -- start start 2024-04-02T14:44:40.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3cdffcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3ce008250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d99d6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f3cdf60770 con 0x55f3cdffc960 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d99d6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f3cdf08700 con 0x55f3ce005e60 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d99d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f3cdf5c720 con 0x55f3cdffd280 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3cdffcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3cdffcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50442/0 (socket says 172.21.15.62:50442) 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d4c00640 1 -- 172.21.15.62:0/3001317434 learned_addr learned my addr 172.21.15.62:0/3001317434 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72cfe00640 1 --2- 172.21.15.62:0/3001317434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72c7e00640 1 --2- 172.21.15.62:0/3001317434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3ce008250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d4c00640 1 -- 172.21.15.62:0/3001317434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 msgr2=0x55f3ce005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d4c00640 1 --2- 172.21.15.62:0/3001317434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d4c00640 1 -- 172.21.15.62:0/3001317434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 msgr2=0x55f3ce008250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d4c00640 1 --2- 172.21.15.62:0/3001317434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3ce008250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72d4c00640 1 -- 172.21.15.62:0/3001317434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3cdd3e7d0 con 0x55f3cdffc960 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.991+0000 7f72c7e00640 1 --2- 172.21.15.62:0/3001317434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3ce008250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.992+0000 7f72d4c00640 1 --2- 172.21.15.62:0/3001317434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3cdffcd40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009f80 tx=0x7f72c8011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09827cb685be90 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.992+0000 7f72c5200640 1 -- 172.21.15.62:0/3001317434 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8012710 con 0x55f3cdffc960 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.992+0000 7f72cd600640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f72b40a3e40 0x7f72b40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.992+0000 7f72c5200640 1 -- 172.21.15.62:0/3001317434 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72c8012db0 con 0x55f3cdffc960 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.992+0000 7f72c5200640 1 -- 172.21.15.62:0/3001317434 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c801a3f0 con 0x55f3cdffc960 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.992+0000 7f72cd600640 1 -- 172.21.15.62:0/1212154618 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f72b80f2920 con 0x7f72d010d300 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.992+0000 7f72cfe00640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f72b40a3e40 0x7f72b40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.993+0000 7f72d99d6740 1 -- 172.21.15.62:0/3001317434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 msgr2=0x55f3cdffcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.993+0000 7f72d99d6740 1 --2- 172.21.15.62:0/3001317434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3cdffcd40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009f80 tx=0x7f72c8011870 comp rx=0 tx=0).stop 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.993+0000 7f72d99d6740 1 -- 172.21.15.62:0/3001317434 shutdown_connections 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.993+0000 7f72d99d6740 1 --2- 172.21.15.62:0/3001317434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3ce008250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.993+0000 7f72d99d6740 1 --2- 172.21.15.62:0/3001317434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce005920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.993+0000 7f72d99d6740 1 --2- 172.21.15.62:0/3001317434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3cdffcd40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.993+0000 7f72d99d6740 1 -- 172.21.15.62:0/3001317434 >> 172.21.15.62:0/3001317434 conn(0x55f3cdff7090 msgr2=0x55f3cdff7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:40.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.993+0000 7f72d99d6740 1 -- 172.21.15.62:0/3001317434 shutdown_connections 2024-04-02T14:44:40.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.993+0000 7f72cfe00640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f72b40a3e40 0x7f72b40a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009620 tx=0x7f72c0005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:40.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.993+0000 7f72d99d6740 1 -- 172.21.15.62:0/3001317434 wait complete. 2024-04-02T14:44:40.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.994+0000 7f72d99d6740 1 -- start start 2024-04-02T14:44:40.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.994+0000 7f72d99d6740 1 Processor -- start 2024-04-02T14:44:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.994+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce002f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.994+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce003470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.994+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3ce0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.994+0000 7f72d99d6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f3ce008c50 con 0x55f3cdffc960 2024-04-02T14:44:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.994+0000 7f72d99d6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f3cdf1bac0 con 0x55f3ce005e60 2024-04-02T14:44:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.994+0000 7f72d99d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f3ce008dd0 con 0x55f3cdffd280 2024-04-02T14:44:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72d99d6740 10 client.0 _ll_register_callbacks cb 0x55f3cdf1e180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce002f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce002f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50458/0 (socket says 172.21.15.62:50458) 2024-04-02T14:44:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72cfe00640 1 -- 172.21.15.62:0/3612707007 learned_addr learned my addr 172.21.15.62:0/3612707007 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72d4c00640 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3ce0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72c7e00640 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce003470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72cfe00640 1 -- 172.21.15.62:0/3612707007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 msgr2=0x55f3ce003470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72cfe00640 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce003470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72cfe00640 1 -- 172.21.15.62:0/3612707007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 msgr2=0x55f3ce0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72cfe00640 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3ce0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72cfe00640 1 -- 172.21.15.62:0/3612707007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3cdc743f0 con 0x55f3cdffc960 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.995+0000 7f72cfe00640 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce002f30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f72c000f830 tx=0x7f72c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3ddcbb71d777f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.996+0000 7f72bd600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.996+0000 7f72bd600640 1 -- 172.21.15.62:0/3612707007 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0011bd0 con 0x55f3cdffc960 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.996+0000 7f72bd600640 1 -- 172.21.15.62:0/3612707007 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f72c0011d70 con 0x55f3cdffc960 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.996+0000 7f72d99d6740 1 -- 172.21.15.62:0/3612707007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72c8012db0 con 0x55f3cdffc960 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.996+0000 7f72bd600640 1 -- 172.21.15.62:0/3612707007 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72c0029440 con 0x55f3cdffc960 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.996+0000 7f72d99d6740 10 client.15453 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.996+0000 7f72d99d6740 1 -- 172.21.15.62:0/3612707007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3cdc340f0 con 0x55f3cdffc960 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.996+0000 7f72bd600640 1 -- 172.21.15.62:0/3612707007 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72c8012db0 con 0x55f3cdffc960 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.997+0000 7f72bd600640 1 -- 172.21.15.62:0/3612707007 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f72c0027070 con 0x55f3cdffc960 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.997+0000 7f72d99d6740 10 client.15453 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.997+0000 7f72d99d6740 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.997+0000 7f72d99d6740 20 client.15453 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.997+0000 7f72d99d6740 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f3ce015f90 0x55f3ce0183c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.997+0000 7f72d99d6740 4 client.15453 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.997+0000 7f72d99d6740 1 -- 172.21.15.62:0/3612707007 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3cdf01630 con 0x55f3ce015f90 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.997+0000 7f72c7e00640 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f3ce015f90 0x55f3ce0183c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.998+0000 7f72c7e00640 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f3ce015f90 0x55f3ce0183c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e0127d1e3b7b1c9 server_cookie=a184524b665edd8b in_seq=0 out_seq=0 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.998+0000 7f72bd600640 10 client.15453 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.998+0000 7f72bd600640 1 -- 172.21.15.62:0/3612707007 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3cdf01630 con 0x55f3ce015f90 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.998+0000 7f72bd600640 10 client.15453 handle_command_reply: tid=0 2024-04-02T14:44:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.998+0000 7f72d99d6740 1 client.15453 shutdown 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72d99d6740 1 -- 172.21.15.62:0/3612707007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 msgr2=0x55f3ce002f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72d99d6740 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce002f30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f72c000f830 tx=0x7f72c0004730 comp rx=0 tx=0).stop 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72d99d6740 1 -- 172.21.15.62:0/3612707007 shutdown_connections 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72d99d6740 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f3ce015f90 0x55f3ce0183c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72d99d6740 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3ce0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72d99d6740 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce003470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72d99d6740 1 --2- 172.21.15.62:0/3612707007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce002f30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72d99d6740 1 -- 172.21.15.62:0/3612707007 >> 172.21.15.62:0/3612707007 conn(0x55f3cdff7090 msgr2=0x55f3cdff6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72bd600640 0 client.15453 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72d99d6740 1 -- 172.21.15.62:0/3612707007 shutdown_connections 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72d99d6740 1 -- 172.21.15.62:0/3612707007 wait complete. 2024-04-02T14:44:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:40.999+0000 7f72d99d6740 20 client.15453 trim_cache size 0 max 16384 2024-04-02T14:44:41.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.018+0000 7f72d99d6740 1 Processor -- start 2024-04-02T14:44:41.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.018+0000 7f72d99d6740 1 -- start start 2024-04-02T14:44:41.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce005660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce011b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3cdff6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72d99d6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01b3bc0 con 0x55f3cdffc960 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72d99d6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01b5700 con 0x55f3ce005e60 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72d99d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01b4290 con 0x55f3cdffd280 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce005660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce005660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50468/0 (socket says 172.21.15.62:50468) 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72cfe00640 1 -- 172.21.15.62:0/1360305955 learned_addr learned my addr 172.21.15.62:0/1360305955 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72d4c00640 1 --2- 172.21.15.62:0/1360305955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3cdff6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72c7e00640 1 --2- 172.21.15.62:0/1360305955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce011b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72cfe00640 1 -- 172.21.15.62:0/1360305955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 msgr2=0x55f3ce011b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72cfe00640 1 --2- 172.21.15.62:0/1360305955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce011b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72cfe00640 1 -- 172.21.15.62:0/1360305955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 msgr2=0x55f3cdff6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72cfe00640 1 --2- 172.21.15.62:0/1360305955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3cdff6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.019+0000 7f72cfe00640 1 -- 172.21.15.62:0/1360305955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ce008250 con 0x55f3cdffc960 2024-04-02T14:44:41.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.020+0000 7f72cfe00640 1 --2- 172.21.15.62:0/1360305955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce005660 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72c00097f0 tx=0x7f72c000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a75949e310c293d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:41.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.020+0000 7f72bfe00640 1 -- 172.21.15.62:0/1360305955 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0036030 con 0x55f3cdffc960 2024-04-02T14:44:41.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.020+0000 7f72bfe00640 1 -- 172.21.15.62:0/1360305955 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72c000f640 con 0x55f3cdffc960 2024-04-02T14:44:41.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.020+0000 7f72bfe00640 1 -- 172.21.15.62:0/1360305955 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0004310 con 0x55f3cdffc960 2024-04-02T14:44:41.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.021+0000 7f72d99d6740 1 -- 172.21.15.62:0/1360305955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 msgr2=0x55f3ce005660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.021+0000 7f72d99d6740 1 --2- 172.21.15.62:0/1360305955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce005660 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72c00097f0 tx=0x7f72c000f3c0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.021+0000 7f72d99d6740 1 -- 172.21.15.62:0/1360305955 shutdown_connections 2024-04-02T14:44:41.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.021+0000 7f72d99d6740 1 --2- 172.21.15.62:0/1360305955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3ce005e60 0x55f3cdff6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.021+0000 7f72d99d6740 1 --2- 172.21.15.62:0/1360305955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3cdffd280 0x55f3ce011b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.021+0000 7f72d99d6740 1 --2- 172.21.15.62:0/1360305955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffc960 0x55f3ce005660 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.021+0000 7f72d99d6740 1 -- 172.21.15.62:0/1360305955 >> 172.21.15.62:0/1360305955 conn(0x55f3cdff7090 msgr2=0x55f3cdff5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:41.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.021+0000 7f72d99d6740 1 -- 172.21.15.62:0/1360305955 shutdown_connections 2024-04-02T14:44:41.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.021+0000 7f72d99d6740 1 -- 172.21.15.62:0/1360305955 wait complete. 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.021+0000 7f72d99d6740 1 -- start start 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.021+0000 7f72d99d6740 1 Processor -- start 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3cdffc960 0x55f3ce0099e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffd280 0x55f3ce009f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72d99d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ce005e60 0x55f3ce00a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72d99d6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01b5030 con 0x55f3cdffd280 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72d99d6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01b4960 con 0x55f3cdffc960 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72d99d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01b1bc0 con 0x55f3ce005e60 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72d99d6740 10 client.0 _ll_register_callbacks cb 0x7f72c0012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffd280 0x55f3ce009f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffd280 0x55f3ce009f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50480/0 (socket says 172.21.15.62:50480) 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72c7e00640 1 -- 172.21.15.62:0/166467649 learned_addr learned my addr 172.21.15.62:0/166467649 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72d4c00640 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ce005e60 0x55f3ce00a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.022+0000 7f72cfe00640 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3cdffc960 0x55f3ce0099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72c7e00640 1 -- 172.21.15.62:0/166467649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ce005e60 msgr2=0x55f3ce00a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72c7e00640 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ce005e60 0x55f3ce00a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72c7e00640 1 -- 172.21.15.62:0/166467649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3cdffc960 msgr2=0x55f3ce0099e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72c7e00640 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3cdffc960 0x55f3ce0099e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72c7e00640 1 -- 172.21.15.62:0/166467649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ce07b500 con 0x55f3cdffd280 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72c7e00640 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffd280 0x55f3ce009f20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f72b80f0050 tx=0x7f72b802bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c9650fb8eec1b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72bd600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72bd600640 1 -- 172.21.15.62:0/166467649 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80bee70 con 0x55f3cdffd280 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72bd600640 1 -- 172.21.15.62:0/166467649 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f72b80bade0 con 0x55f3cdffd280 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72d99d6740 1 -- 172.21.15.62:0/166467649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72c000f640 con 0x55f3cdffd280 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72bd600640 1 -- 172.21.15.62:0/166467649 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72b80dc050 con 0x55f3cdffd280 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72d99d6740 10 client.15459 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.023+0000 7f72d99d6740 1 -- 172.21.15.62:0/166467649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3ce07c840 con 0x55f3cdffd280 2024-04-02T14:44:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.024+0000 7f72bd600640 1 -- 172.21.15.62:0/166467649 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72c000f640 con 0x55f3cdffd280 2024-04-02T14:44:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.024+0000 7f72bd600640 1 -- 172.21.15.62:0/166467649 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f72b80c6f50 con 0x55f3cdffd280 2024-04-02T14:44:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.024+0000 7f72d99d6740 10 client.15459 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.024+0000 7f72d99d6740 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.024+0000 7f72d99d6740 20 client.15459 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.024+0000 7f72d99d6740 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f3ce07cf20 0x55f3ce07f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.024+0000 7f72d99d6740 4 client.15459 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:44:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.024+0000 7f72d99d6740 1 -- 172.21.15.62:0/166467649 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3cdf6dcb0 con 0x55f3ce07cf20 2024-04-02T14:44:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.024+0000 7f72cfe00640 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f3ce07cf20 0x55f3ce07f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72cfe00640 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f3ce07cf20 0x55f3ce07f340 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=165966c4356e9ae1 server_cookie=80fadbf82715cd88 in_seq=0 out_seq=0 2024-04-02T14:44:41.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72bd600640 10 client.15459 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:41.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72bd600640 1 -- 172.21.15.62:0/166467649 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f3cdf6dcb0 con 0x55f3ce07cf20 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72bd600640 10 client.15459 handle_command_reply: tid=0 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72d99d6740 1 client.15459 shutdown 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72d99d6740 1 -- 172.21.15.62:0/166467649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffd280 msgr2=0x55f3ce009f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72d99d6740 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffd280 0x55f3ce009f20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f72b80f0050 tx=0x7f72b802bc40 comp rx=0 tx=0).stop 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72d99d6740 1 -- 172.21.15.62:0/166467649 shutdown_connections 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72d99d6740 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f3ce07cf20 0x55f3ce07f340 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72d99d6740 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ce005e60 0x55f3ce00a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72d99d6740 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3cdffd280 0x55f3ce009f20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72d99d6740 1 --2- 172.21.15.62:0/166467649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3cdffc960 0x55f3ce0099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72d99d6740 1 -- 172.21.15.62:0/166467649 >> 172.21.15.62:0/166467649 conn(0x55f3cdff7090 msgr2=0x55f3cdff6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72bd600640 0 client.15459 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.025+0000 7f72d99d6740 1 -- 172.21.15.62:0/166467649 shutdown_connections 2024-04-02T14:44:41.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d99d6740 1 -- 172.21.15.62:0/166467649 wait complete. 2024-04-02T14:44:41.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d99d6740 20 client.15459 trim_cache size 0 max 16384 2024-04-02T14:44:41.027 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:44:41.027 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-02T14:44:41.027 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-02T14:44:41.027 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:44:41.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d5600640 1 -- 172.21.15.62:0/1212154618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f72b40a3e40 msgr2=0x7f72b40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d5600640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f72b40a3e40 0x7f72b40a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009620 tx=0x7f72c0005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:41.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d5600640 1 -- 172.21.15.62:0/1212154618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d010d300 msgr2=0x7f72d01a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d5600640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d010d300 0x7f72d01a4b60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f72b802c180 tx=0x7f72b802dc00 comp rx=0 tx=0).stop 2024-04-02T14:44:41.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d5600640 1 -- 172.21.15.62:0/1212154618 shutdown_connections 2024-04-02T14:44:41.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d5600640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f72b40a3e40 0x7f72b40a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d5600640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d019e790 0x7f72d019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d5600640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d010d300 0x7f72d01a4b60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d5600640 1 --2- 172.21.15.62:0/1212154618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d01082b0 0x7f72d01a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.026+0000 7f72d5600640 1 -- 172.21.15.62:0/1212154618 >> 172.21.15.62:0/1212154618 conn(0x7f72d0077920 msgr2=0x7f72d0077f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:41.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.027+0000 7f72d5600640 1 -- 172.21.15.62:0/1212154618 shutdown_connections 2024-04-02T14:44:41.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.027+0000 7f72d5600640 1 -- 172.21.15.62:0/1212154618 wait complete. 2024-04-02T14:44:41.040 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:44:41.041 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:44:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.256+0000 7fc611600640 1 -- 172.21.15.62:0/2786061018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc60c108cd0 msgr2=0x7fc60c10cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.256+0000 7fc611600640 1 --2- 172.21.15.62:0/2786061018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc60c108cd0 0x7fc60c10cd10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009900 tx=0x7fc5f802bc40 comp rx=0 tx=0).stop 2024-04-02T14:44:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.256+0000 7fc611600640 1 -- 172.21.15.62:0/2786061018 shutdown_connections 2024-04-02T14:44:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.256+0000 7fc611600640 1 --2- 172.21.15.62:0/2786061018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc60c10d3f0 0x7fc60c114fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.256+0000 7fc611600640 1 --2- 172.21.15.62:0/2786061018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc60c108cd0 0x7fc60c10cd10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.256+0000 7fc611600640 1 --2- 172.21.15.62:0/2786061018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c108320 0x7fc60c108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.256+0000 7fc611600640 1 -- 172.21.15.62:0/2786061018 >> 172.21.15.62:0/2786061018 conn(0x7fc60c076050 msgr2=0x7fc60c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.256+0000 7fc611600640 1 -- 172.21.15.62:0/2786061018 shutdown_connections 2024-04-02T14:44:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.256+0000 7fc611600640 1 -- 172.21.15.62:0/2786061018 wait complete. 2024-04-02T14:44:41.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.257+0000 7fc611600640 1 Processor -- start 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.257+0000 7fc611600640 1 -- start start 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.258+0000 7fc611600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c108320 0x7fc60c1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.258+0000 7fc611600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc60c10d3f0 0x7fc60c1a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.258+0000 7fc611600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc60c19e8e0 0x7fc60c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.258+0000 7fc611600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1185c0 con 0x7fc60c19e8e0 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.258+0000 7fc611600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c118440 con 0x7fc60c108320 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.258+0000 7fc611600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c118740 con 0x7fc60c10d3f0 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.258+0000 7fc60b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc60c19e8e0 0x7fc60c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.258+0000 7fc60b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc60c19e8e0 0x7fc60c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50506/0 (socket says 172.21.15.62:50506) 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.258+0000 7fc60b400640 1 -- 172.21.15.62:0/2174624009 learned_addr learned my addr 172.21.15.62:0/2174624009 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.258+0000 7fc60aa00640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c108320 0x7fc60c1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.258+0000 7fc60a000640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc60c10d3f0 0x7fc60c1a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.259+0000 7fc60b400640 1 -- 172.21.15.62:0/2174624009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc60c10d3f0 msgr2=0x7fc60c1a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.259+0000 7fc60b400640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc60c10d3f0 0x7fc60c1a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.259+0000 7fc60b400640 1 -- 172.21.15.62:0/2174624009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c108320 msgr2=0x7fc60c1a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.259+0000 7fc60b400640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c108320 0x7fc60c1a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.259+0000 7fc60b400640 1 -- 172.21.15.62:0/2174624009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c19f400 con 0x7fc60c19e8e0 2024-04-02T14:44:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.259+0000 7fc60b400640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc60c19e8e0 0x7fc60c19ed90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc60400efd0 tx=0x7fc60400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.259+0000 7fc603400640 1 -- 172.21.15.62:0/2174624009 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604003b70 con 0x7fc60c19e8e0 2024-04-02T14:44:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.259+0000 7fc603400640 1 -- 172.21.15.62:0/2174624009 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc604003d10 con 0x7fc60c19e8e0 2024-04-02T14:44:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.259+0000 7fc611600640 1 -- 172.21.15.62:0/2174624009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c19f6f0 con 0x7fc60c19e8e0 2024-04-02T14:44:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.259+0000 7fc603400640 1 -- 172.21.15.62:0/2174624009 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604010a90 con 0x7fc60c19e8e0 2024-04-02T14:44:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.259+0000 7fc611600640 1 -- 172.21.15.62:0/2174624009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60c1ab940 con 0x7fc60c19e8e0 2024-04-02T14:44:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.261+0000 7fc611600640 1 -- 172.21.15.62:0/2174624009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60c1097f0 con 0x7fc60c19e8e0 2024-04-02T14:44:41.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.263+0000 7fc603400640 1 -- 172.21.15.62:0/2174624009 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fc604004020 con 0x7fc60c19e8e0 2024-04-02T14:44:41.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.264+0000 7fc603400640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5f00a3ec0 0x7fc5f00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.264+0000 7fc603400640 1 -- 172.21.15.62:0/2174624009 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc6040d0dd0 con 0x7fc60c19e8e0 2024-04-02T14:44:41.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.264+0000 7fc60aa00640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5f00a3ec0 0x7fc5f00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.265+0000 7fc60aa00640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5f00a3ec0 0x7fc5f00a6380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc60c19fa20 tx=0x7fc5fc005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:41.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.269+0000 7fc603400640 1 -- 172.21.15.62:0/2174624009 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc60409d210 con 0x7fc60c19e8e0 2024-04-02T14:44:41.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.524+0000 7fc611600640 1 -- 172.21.15.62:0/2174624009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc60c068d40 con 0x7fc60c19e8e0 2024-04-02T14:44:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.526+0000 7fc603400640 1 -- 172.21.15.62:0/2174624009 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fc60c068d40 con 0x7fc60c19e8e0 2024-04-02T14:44:41.522 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:44:41.522 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:44:00.377074+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584,24535,24533,24553,24565]},"id":1}]} 2024-04-02T14:44:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-04-02T14:44:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.528+0000 7fc611600640 1 -- 172.21.15.62:0/2174624009 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5f00a3ec0 msgr2=0x7fc5f00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.528+0000 7fc611600640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5f00a3ec0 0x7fc5f00a6380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc60c19fa20 tx=0x7fc5fc005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.528+0000 7fc611600640 1 -- 172.21.15.62:0/2174624009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc60c19e8e0 msgr2=0x7fc60c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.528+0000 7fc611600640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc60c19e8e0 0x7fc60c19ed90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc60400efd0 tx=0x7fc60400c6a0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.529+0000 7fc611600640 1 -- 172.21.15.62:0/2174624009 shutdown_connections 2024-04-02T14:44:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.529+0000 7fc611600640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5f00a3ec0 0x7fc5f00a6380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.529+0000 7fc611600640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc60c19e8e0 0x7fc60c19ed90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.529+0000 7fc611600640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc60c10d3f0 0x7fc60c1a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.529+0000 7fc611600640 1 --2- 172.21.15.62:0/2174624009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c108320 0x7fc60c1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.529+0000 7fc611600640 1 -- 172.21.15.62:0/2174624009 >> 172.21.15.62:0/2174624009 conn(0x7fc60c076050 msgr2=0x7fc60c078400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.529+0000 7fc611600640 1 -- 172.21.15.62:0/2174624009 shutdown_connections 2024-04-02T14:44:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.529+0000 7fc611600640 1 -- 172.21.15.62:0/2174624009 wait complete. 2024-04-02T14:44:41.541 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:41 smithi062 ceph-mon[32116]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T14:44:41.543 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:44:41.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:41 smithi113 ceph-mon[36135]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T14:44:41.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:41 smithi204 ceph-mon[35948]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T14:44:41.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.762+0000 7f2572600640 1 -- 172.21.15.62:0/1331086839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f256c101330 msgr2=0x7f256c10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.762+0000 7f2572600640 1 --2- 172.21.15.62:0/1331086839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f256c101330 0x7f256c10d840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2564009890 tx=0x7f256402bb70 comp rx=0 tx=0).stop 2024-04-02T14:44:41.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.762+0000 7f2572600640 1 -- 172.21.15.62:0/1331086839 shutdown_connections 2024-04-02T14:44:41.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.762+0000 7f2572600640 1 --2- 172.21.15.62:0/1331086839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f256c101330 0x7f256c10d840 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.762+0000 7f2572600640 1 --2- 172.21.15.62:0/1331086839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256c100970 0x7f256c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.762+0000 7f2572600640 1 --2- 172.21.15.62:0/1331086839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256c106980 0x7f256c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.762+0000 7f2572600640 1 -- 172.21.15.62:0/1331086839 >> 172.21.15.62:0/1331086839 conn(0x7f256c077b00 msgr2=0x7f256c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:41.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.763+0000 7f2572600640 1 -- 172.21.15.62:0/1331086839 shutdown_connections 2024-04-02T14:44:41.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.763+0000 7f2572600640 1 -- 172.21.15.62:0/1331086839 wait complete. 2024-04-02T14:44:41.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.763+0000 7f2572600640 1 Processor -- start 2024-04-02T14:44:41.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f2572600640 1 -- start start 2024-04-02T14:44:41.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f2572600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256c100970 0x7f256c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f2572600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f256c101330 0x7f256c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f2572600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256c106980 0x7f256c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f2572600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f256c110eb0 con 0x7f256c101330 2024-04-02T14:44:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f2572600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f256c110d30 con 0x7f256c100970 2024-04-02T14:44:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f2572600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f256c111030 con 0x7f256c106980 2024-04-02T14:44:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f256aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f256c101330 0x7f256c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f256be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256c106980 0x7f256c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f256b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256c100970 0x7f256c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f256aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f256c101330 0x7f256c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50518/0 (socket says 172.21.15.62:50518) 2024-04-02T14:44:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.764+0000 7f256aa00640 1 -- 172.21.15.62:0/684040949 learned_addr learned my addr 172.21.15.62:0/684040949 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:41.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.765+0000 7f256aa00640 1 -- 172.21.15.62:0/684040949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256c106980 msgr2=0x7f256c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.765+0000 7f256aa00640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256c106980 0x7f256c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.765+0000 7f256aa00640 1 -- 172.21.15.62:0/684040949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256c100970 msgr2=0x7f256c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.765+0000 7f256aa00640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256c100970 0x7f256c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.765+0000 7f256aa00640 1 -- 172.21.15.62:0/684040949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c19f660 con 0x7f256c101330 2024-04-02T14:44:41.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.765+0000 7f256be00640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256c106980 0x7f256c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:41.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.765+0000 7f256aa00640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f256c101330 0x7f256c19abb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f255400c420 tx=0x7f255400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:41.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.765+0000 7f2563e00640 1 -- 172.21.15.62:0/684040949 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554004b20 con 0x7f256c101330 2024-04-02T14:44:41.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.765+0000 7f2563e00640 1 -- 172.21.15.62:0/684040949 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2554004cc0 con 0x7f256c101330 2024-04-02T14:44:41.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.765+0000 7f2563e00640 1 -- 172.21.15.62:0/684040949 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255400b630 con 0x7f256c101330 2024-04-02T14:44:41.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.765+0000 7f2572600640 1 -- 172.21.15.62:0/684040949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256c19f950 con 0x7f256c101330 2024-04-02T14:44:41.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.766+0000 7f2572600640 1 -- 172.21.15.62:0/684040949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f256c1a71e0 con 0x7f256c101330 2024-04-02T14:44:41.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.769+0000 7f2563e00640 1 -- 172.21.15.62:0/684040949 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f255401d020 con 0x7f256c101330 2024-04-02T14:44:41.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.769+0000 7f25753fe740 1 Processor -- start 2024-04-02T14:44:41.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.769+0000 7f25753fe740 1 -- start start 2024-04-02T14:44:41.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f25753fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 0x56478a95d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f25753fe740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 0x56478a95e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f25753fe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 0x56478a969470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f25753fe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56478a89e400 con 0x56478a967080 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f25753fe740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56478a882f10 con 0x56478a95dd10 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f25753fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56478a8c26f0 con 0x56478a95e9f0 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f256be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 0x56478a95d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f256b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 0x56478a95e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f256be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 0x56478a95d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:57546/0 (socket says 172.21.15.62:57546) 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f256aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 0x56478a969470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f256be00640 1 -- 172.21.15.62:0/3325176174 learned_addr learned my addr 172.21.15.62:0/3325176174 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f256be00640 1 -- 172.21.15.62:0/3325176174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 msgr2=0x56478a95e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f256be00640 1 --2- 172.21.15.62:0/3325176174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 0x56478a95e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f256be00640 1 -- 172.21.15.62:0/3325176174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 msgr2=0x56478a969470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.770+0000 7f256be00640 1 --2- 172.21.15.62:0/3325176174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 0x56478a969470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.771+0000 7f256be00640 1 -- 172.21.15.62:0/3325176174 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56478a5d63f0 con 0x56478a95e9f0 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.771+0000 7f256b400640 1 --2- 172.21.15.62:0/3325176174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 0x56478a95e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.771+0000 7f256be00640 1 --2- 172.21.15.62:0/3325176174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 0x56478a95d740 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2564009890 tx=0x7f25640058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2f6c3800161c7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.771+0000 7f2563e00640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f254c0a3e40 0x7f254c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.771+0000 7f2563e00640 1 -- 172.21.15.62:0/684040949 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f25540d0a00 con 0x7f256c101330 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.771+0000 7f2560c00640 1 -- 172.21.15.62:0/3325176174 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564039040 con 0x56478a95e9f0 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.771+0000 7f256b400640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f254c0a3e40 0x7f254c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.771+0000 7f2560c00640 1 -- 172.21.15.62:0/3325176174 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2564034440 con 0x56478a95e9f0 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.771+0000 7f2560c00640 1 -- 172.21.15.62:0/3325176174 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256402ca30 con 0x56478a95e9f0 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.772+0000 7f25753fe740 1 -- 172.21.15.62:0/3325176174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 msgr2=0x56478a95d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.772+0000 7f25753fe740 1 --2- 172.21.15.62:0/3325176174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 0x56478a95d740 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2564009890 tx=0x7f25640058d0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.772+0000 7f256b400640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f254c0a3e40 0x7f254c0a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x56478a95ef40 tx=0x7f255c005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:41.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.772+0000 7f25753fe740 1 -- 172.21.15.62:0/3325176174 shutdown_connections 2024-04-02T14:44:41.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.772+0000 7f25753fe740 1 --2- 172.21.15.62:0/3325176174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 0x56478a969470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.772+0000 7f25753fe740 1 --2- 172.21.15.62:0/3325176174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 0x56478a95e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.772+0000 7f25753fe740 1 --2- 172.21.15.62:0/3325176174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 0x56478a95d740 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.772+0000 7f25753fe740 1 -- 172.21.15.62:0/3325176174 >> 172.21.15.62:0/3325176174 conn(0x56478a924a50 msgr2=0x56478a958760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:41.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.772+0000 7f25753fe740 1 -- 172.21.15.62:0/3325176174 shutdown_connections 2024-04-02T14:44:41.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.772+0000 7f25753fe740 1 -- 172.21.15.62:0/3325176174 wait complete. 2024-04-02T14:44:41.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.773+0000 7f25753fe740 1 -- start start 2024-04-02T14:44:41.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.773+0000 7f25753fe740 1 Processor -- start 2024-04-02T14:44:41.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.773+0000 7f25753fe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95dd10 0x56478a964040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.773+0000 7f25753fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 0x56478a964580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.773+0000 7f25753fe740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a967080 0x56478a964ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.773+0000 7f25753fe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56478a8be6a0 con 0x56478a95dd10 2024-04-02T14:44:41.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.773+0000 7f25753fe740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56478a86a980 con 0x56478a967080 2024-04-02T14:44:41.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.773+0000 7f25753fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56478a87dcb0 con 0x56478a95e9f0 2024-04-02T14:44:41.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f25753fe740 10 client.0 _ll_register_callbacks cb 0x56478a87d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:41.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95dd10 0x56478a964040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95dd10 0x56478a964040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50532/0 (socket says 172.21.15.62:50532) 2024-04-02T14:44:41.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 0x56478a964580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256b400640 1 -- 172.21.15.62:0/45699782 learned_addr learned my addr 172.21.15.62:0/45699782 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a967080 0x56478a964ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256be00640 1 -- 172.21.15.62:0/45699782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 msgr2=0x56478a964580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256be00640 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 0x56478a964580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256be00640 1 -- 172.21.15.62:0/45699782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95dd10 msgr2=0x56478a964040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256be00640 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95dd10 0x56478a964040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256be00640 1 -- 172.21.15.62:0/45699782 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56478a6a07d0 con 0x56478a967080 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256aa00640 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 0x56478a964580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.774+0000 7f256be00640 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a967080 0x56478a964ac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2564005940 tx=0x7f2564034a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f8eb75527142e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.775+0000 7f2559600640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.775+0000 7f2559600640 1 -- 172.21.15.62:0/45699782 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256404c070 con 0x56478a967080 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.775+0000 7f25753fe740 1 -- 172.21.15.62:0/45699782 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2564034440 con 0x56478a967080 2024-04-02T14:44:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.775+0000 7f2559600640 1 -- 172.21.15.62:0/45699782 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f2564004450 con 0x56478a967080 2024-04-02T14:44:41.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.775+0000 7f2559600640 1 -- 172.21.15.62:0/45699782 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2564039040 con 0x56478a967080 2024-04-02T14:44:41.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.775+0000 7f25753fe740 10 client.25610 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:41.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.775+0000 7f25753fe740 1 -- 172.21.15.62:0/45699782 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56478a975e90 con 0x56478a967080 2024-04-02T14:44:41.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.776+0000 7f2559600640 1 -- 172.21.15.62:0/45699782 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2564034440 con 0x56478a967080 2024-04-02T14:44:41.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.776+0000 7f2559600640 1 -- 172.21.15.62:0/45699782 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2564032cf0 con 0x56478a967080 2024-04-02T14:44:41.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.776+0000 7f25753fe740 10 client.25610 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:41.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.776+0000 7f25753fe740 10 client.25610 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:44:41.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.776+0000 7f25753fe740 20 client.25610 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:41.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.776+0000 7f25753fe740 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x56478a9771e0 0x56478a9795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.776+0000 7f25753fe740 4 client.25610 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:41.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.776+0000 7f25753fe740 1 -- 172.21.15.62:0/45699782 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56478a882040 con 0x56478a9771e0 2024-04-02T14:44:41.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.776+0000 7f256b400640 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x56478a9771e0 0x56478a9795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.777+0000 7f256b400640 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x56478a9771e0 0x56478a9795d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ec36dfb5dc5a910b server_cookie=630df1d2c4c84c32 in_seq=0 out_seq=0 2024-04-02T14:44:41.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.777+0000 7f2559600640 10 client.25610 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:44:41.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f2559600640 1 -- 172.21.15.62:0/45699782 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56478a882040 con 0x56478a9771e0 2024-04-02T14:44:41.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f2559600640 10 client.25610 handle_command_reply: tid=0 2024-04-02T14:44:41.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 1 client.25610 shutdown 2024-04-02T14:44:41.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 1 -- 172.21.15.62:0/45699782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a967080 msgr2=0x56478a964ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a967080 0x56478a964ac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2564005940 tx=0x7f2564034a00 comp rx=0 tx=0).stop 2024-04-02T14:44:41.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 1 -- 172.21.15.62:0/45699782 shutdown_connections 2024-04-02T14:44:41.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x56478a9771e0 0x56478a9795d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a967080 0x56478a964ac0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95e9f0 0x56478a964580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 1 --2- 172.21.15.62:0/45699782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95dd10 0x56478a964040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f2559600640 0 client.25610 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:44:41.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 1 -- 172.21.15.62:0/45699782 >> 172.21.15.62:0/45699782 conn(0x56478a924a50 msgr2=0x56478a95a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:41.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 1 -- 172.21.15.62:0/45699782 shutdown_connections 2024-04-02T14:44:41.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 1 -- 172.21.15.62:0/45699782 wait complete. 2024-04-02T14:44:41.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.778+0000 7f25753fe740 20 client.25610 trim_cache size 0 max 16384 2024-04-02T14:44:41.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.793+0000 7f25753fe740 1 Processor -- start 2024-04-02T14:44:41.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.793+0000 7f25753fe740 1 -- start start 2024-04-02T14:44:41.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.793+0000 7f25753fe740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 0x56478a966770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.793+0000 7f25753fe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95e9f0 0x56478a95a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.793+0000 7f25753fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a967080 0x56478a9648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.793+0000 7f25753fe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1af920 con 0x56478a95e9f0 2024-04-02T14:44:41.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.793+0000 7f25753fe740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1b1460 con 0x56478a95dd10 2024-04-02T14:44:41.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.793+0000 7f25753fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1afff0 con 0x56478a967080 2024-04-02T14:44:41.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.794+0000 7f256aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95e9f0 0x56478a95a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.794+0000 7f256be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a967080 0x56478a9648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.794+0000 7f256aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95e9f0 0x56478a95a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50540/0 (socket says 172.21.15.62:50540) 2024-04-02T14:44:41.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.794+0000 7f256aa00640 1 -- 172.21.15.62:0/4131497126 learned_addr learned my addr 172.21.15.62:0/4131497126 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:41.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.794+0000 7f256b400640 1 --2- 172.21.15.62:0/4131497126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 0x56478a966770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.794+0000 7f256b400640 1 -- 172.21.15.62:0/4131497126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a967080 msgr2=0x56478a9648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.794+0000 7f256b400640 1 --2- 172.21.15.62:0/4131497126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a967080 0x56478a9648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.794+0000 7f256b400640 1 -- 172.21.15.62:0/4131497126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95e9f0 msgr2=0x56478a95a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.794+0000 7f256b400640 1 --2- 172.21.15.62:0/4131497126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95e9f0 0x56478a95a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.794+0000 7f256b400640 1 -- 172.21.15.62:0/4131497126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56478a95cd40 con 0x56478a95dd10 2024-04-02T14:44:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.794+0000 7f256b400640 1 --2- 172.21.15.62:0/4131497126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 0x56478a966770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x56478a87dce0 tx=0x7f255c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39821cbcb451c3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.795+0000 7f255be00640 1 -- 172.21.15.62:0/4131497126 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c004420 con 0x56478a95dd10 2024-04-02T14:44:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.795+0000 7f255be00640 1 -- 172.21.15.62:0/4131497126 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f255c02b920 con 0x56478a95dd10 2024-04-02T14:44:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.795+0000 7f255be00640 1 -- 172.21.15.62:0/4131497126 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c0048d0 con 0x56478a95dd10 2024-04-02T14:44:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.795+0000 7f25753fe740 1 -- 172.21.15.62:0/4131497126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 msgr2=0x56478a966770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.795+0000 7f25753fe740 1 --2- 172.21.15.62:0/4131497126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 0x56478a966770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x56478a87dce0 tx=0x7f255c01b000 comp rx=0 tx=0).stop 2024-04-02T14:44:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.795+0000 7f25753fe740 1 -- 172.21.15.62:0/4131497126 shutdown_connections 2024-04-02T14:44:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.795+0000 7f25753fe740 1 --2- 172.21.15.62:0/4131497126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a967080 0x56478a9648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.795+0000 7f25753fe740 1 --2- 172.21.15.62:0/4131497126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a95e9f0 0x56478a95a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.795+0000 7f25753fe740 1 --2- 172.21.15.62:0/4131497126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95dd10 0x56478a966770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.795+0000 7f25753fe740 1 -- 172.21.15.62:0/4131497126 >> 172.21.15.62:0/4131497126 conn(0x56478a924a50 msgr2=0x56478a972830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.796+0000 7f25753fe740 1 -- 172.21.15.62:0/4131497126 shutdown_connections 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.796+0000 7f25753fe740 1 -- 172.21.15.62:0/4131497126 wait complete. 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.796+0000 7f25753fe740 1 -- start start 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.796+0000 7f25753fe740 1 Processor -- start 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.796+0000 7f25753fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95dd10 0x56478a97a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f25753fe740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95e9f0 0x56478a97a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f25753fe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 0x56478a97aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f25753fe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1b0d90 con 0x56478a967080 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f25753fe740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1b06c0 con 0x56478a95e9f0 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f25753fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1ad900 con 0x56478a95dd10 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f25753fe740 10 client.0 _ll_register_callbacks cb 0x7f2554009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f256be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 0x56478a97aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f256be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 0x56478a97aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50544/0 (socket says 172.21.15.62:50544) 2024-04-02T14:44:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f256be00640 1 -- 172.21.15.62:0/138014190 learned_addr learned my addr 172.21.15.62:0/138014190 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f256b400640 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95dd10 0x56478a97a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f256aa00640 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95e9f0 0x56478a97a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f256be00640 1 -- 172.21.15.62:0/138014190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95dd10 msgr2=0x56478a97a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f256be00640 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95dd10 0x56478a97a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f256be00640 1 -- 172.21.15.62:0/138014190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95e9f0 msgr2=0x56478a97a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f256be00640 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95e9f0 0x56478a97a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f256be00640 1 -- 172.21.15.62:0/138014190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56478a9dc3f0 con 0x56478a967080 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.797+0000 7f256be00640 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 0x56478a97aaf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2564009780 tx=0x7f2564005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbbb52585f6256 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.798+0000 7f2559600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.798+0000 7f2559600640 1 -- 172.21.15.62:0/138014190 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256404c040 con 0x56478a967080 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.798+0000 7f2559600640 1 -- 172.21.15.62:0/138014190 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f2564004460 con 0x56478a967080 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.798+0000 7f25753fe740 1 -- 172.21.15.62:0/138014190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56478a8cfc30 con 0x56478a967080 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.798+0000 7f2559600640 1 -- 172.21.15.62:0/138014190 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2564009450 con 0x56478a967080 2024-04-02T14:44:41.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.798+0000 7f25753fe740 10 client.15483 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:41.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.798+0000 7f25753fe740 1 -- 172.21.15.62:0/138014190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56478a9dd6d0 con 0x56478a967080 2024-04-02T14:44:41.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.798+0000 7f2559600640 1 -- 172.21.15.62:0/138014190 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x56478a8cfc30 con 0x56478a967080 2024-04-02T14:44:41.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.798+0000 7f2559600640 1 -- 172.21.15.62:0/138014190 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2564032640 con 0x56478a967080 2024-04-02T14:44:41.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.799+0000 7f25753fe740 10 client.15483 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:41.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.799+0000 7f25753fe740 10 client.15483 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:44:41.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.799+0000 7f25753fe740 20 client.15483 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:41.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.799+0000 7f25753fe740 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x56478a9dddc0 0x56478a9e01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:41.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.799+0000 7f25753fe740 4 client.15483 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:44:41.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.799+0000 7f25753fe740 1 -- 172.21.15.62:0/138014190 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56478a8c9f80 con 0x56478a9dddc0 2024-04-02T14:44:41.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.799+0000 7f256b400640 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x56478a9dddc0 0x56478a9e01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:41.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f256b400640 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x56478a9dddc0 0x56478a9e01e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7b2bcb1b03994d64 server_cookie=df68b298a7ab60e1 in_seq=0 out_seq=0 2024-04-02T14:44:41.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f2559600640 10 client.15483 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:44:41.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f2559600640 1 -- 172.21.15.62:0/138014190 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56478a8c9f80 con 0x56478a9dddc0 2024-04-02T14:44:41.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f2559600640 10 client.15483 handle_command_reply: tid=0 2024-04-02T14:44:41.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 1 client.15483 shutdown 2024-04-02T14:44:41.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 1 -- 172.21.15.62:0/138014190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 msgr2=0x56478a97aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 0x56478a97aaf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2564009780 tx=0x7f2564005c90 comp rx=0 tx=0).stop 2024-04-02T14:44:41.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 1 -- 172.21.15.62:0/138014190 shutdown_connections 2024-04-02T14:44:41.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x56478a9dddc0 0x56478a9e01e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56478a967080 0x56478a97aaf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56478a95e9f0 0x56478a97a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 1 --2- 172.21.15.62:0/138014190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56478a95dd10 0x56478a97a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f2559600640 0 client.15483 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:44:41.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 1 -- 172.21.15.62:0/138014190 >> 172.21.15.62:0/138014190 conn(0x56478a924a50 msgr2=0x56478a9627e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:41.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 1 -- 172.21.15.62:0/138014190 shutdown_connections 2024-04-02T14:44:41.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 1 -- 172.21.15.62:0/138014190 wait complete. 2024-04-02T14:44:41.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.800+0000 7f25753fe740 20 client.15483 trim_cache size 0 max 16384 2024-04-02T14:44:41.804 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:44:41.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 -- 172.21.15.62:0/684040949 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f254c0a3e40 msgr2=0x7f254c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f254c0a3e40 0x7f254c0a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x56478a95ef40 tx=0x7f255c005f90 comp rx=0 tx=0).stop 2024-04-02T14:44:41.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 -- 172.21.15.62:0/684040949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f256c101330 msgr2=0x7f256c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:41.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f256c101330 0x7f256c19abb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f255400c420 tx=0x7f255400c8e0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 -- 172.21.15.62:0/684040949 shutdown_connections 2024-04-02T14:44:41.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f254c0a3e40 0x7f254c0a6300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256c106980 0x7f256c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f256c101330 0x7f256c19abb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 --2- 172.21.15.62:0/684040949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256c100970 0x7f256c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:41.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 -- 172.21.15.62:0/684040949 >> 172.21.15.62:0/684040949 conn(0x7f256c077b00 msgr2=0x7f256c10c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:41.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 -- 172.21.15.62:0/684040949 shutdown_connections 2024-04-02T14:44:41.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:41.801+0000 7f2572600640 1 -- 172.21.15.62:0/684040949 wait complete. 2024-04-02T14:44:41.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:44:42.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.041+0000 7f8d61200640 1 -- 172.21.15.62:0/1001493139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c108c60 msgr2=0x7f8d5c10cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.041+0000 7f8d61200640 1 --2- 172.21.15.62:0/1001493139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c108c60 0x7f8d5c10cc20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009900 tx=0x7f8d4802bc40 comp rx=0 tx=0).stop 2024-04-02T14:44:42.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.041+0000 7f8d61200640 1 -- 172.21.15.62:0/1001493139 shutdown_connections 2024-04-02T14:44:42.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.041+0000 7f8d61200640 1 --2- 172.21.15.62:0/1001493139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5c10d300 0x7f8d5c114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.041+0000 7f8d61200640 1 --2- 172.21.15.62:0/1001493139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c108c60 0x7f8d5c10cc20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.041+0000 7f8d61200640 1 --2- 172.21.15.62:0/1001493139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c1082b0 0x7f8d5c108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.041+0000 7f8d61200640 1 -- 172.21.15.62:0/1001493139 >> 172.21.15.62:0/1001493139 conn(0x7f8d5c077920 msgr2=0x7f8d5c106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.041+0000 7f8d61200640 1 -- 172.21.15.62:0/1001493139 shutdown_connections 2024-04-02T14:44:42.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.041+0000 7f8d61200640 1 -- 172.21.15.62:0/1001493139 wait complete. 2024-04-02T14:44:42.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.042+0000 7f8d61200640 1 Processor -- start 2024-04-02T14:44:42.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.042+0000 7f8d61200640 1 -- start start 2024-04-02T14:44:42.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.042+0000 7f8d61200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c1082b0 0x7f8d5c1a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.042+0000 7f8d61200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c10d300 0x7f8d5c1a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.042+0000 7f8d61200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5c19e7d0 0x7f8d5c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.042+0000 7f8d61200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c1184b0 con 0x7f8d5c19e7d0 2024-04-02T14:44:42.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.042+0000 7f8d61200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c118330 con 0x7f8d5c1082b0 2024-04-02T14:44:42.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.042+0000 7f8d61200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c118630 con 0x7f8d5c10d300 2024-04-02T14:44:42.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.043+0000 7f8d5b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c1082b0 0x7f8d5c1a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.043+0000 7f8d5b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c1082b0 0x7f8d5c1a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:42720/0 (socket says 172.21.15.62:42720) 2024-04-02T14:44:42.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.043+0000 7f8d5b400640 1 -- 172.21.15.62:0/937544697 learned_addr learned my addr 172.21.15.62:0/937544697 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.043+0000 7f8d5be00640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5c19e7d0 0x7f8d5c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.043+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c10d300 0x7f8d5c1a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.043+0000 7f8d5b400640 1 -- 172.21.15.62:0/937544697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c10d300 msgr2=0x7f8d5c1a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.043+0000 7f8d5b400640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c10d300 0x7f8d5c1a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.043+0000 7f8d5b400640 1 -- 172.21.15.62:0/937544697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5c19e7d0 msgr2=0x7f8d5c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.043+0000 7f8d5b400640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5c19e7d0 0x7f8d5c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.043+0000 7f8d5b400640 1 -- 172.21.15.62:0/937544697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c19f360 con 0x7f8d5c1082b0 2024-04-02T14:44:42.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.043+0000 7f8d5b400640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c1082b0 0x7f8d5c1a4610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00e9f0 tx=0x7f8d4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.044+0000 7f8d53e00640 1 -- 172.21.15.62:0/937544697 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c00cd80 con 0x7f8d5c1082b0 2024-04-02T14:44:42.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.044+0000 7f8d61200640 1 -- 172.21.15.62:0/937544697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5c19f650 con 0x7f8d5c1082b0 2024-04-02T14:44:42.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.044+0000 7f8d53e00640 1 -- 172.21.15.62:0/937544697 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8d4c010930 con 0x7f8d5c1082b0 2024-04-02T14:44:42.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.044+0000 7f8d53e00640 1 -- 172.21.15.62:0/937544697 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c018b00 con 0x7f8d5c1082b0 2024-04-02T14:44:42.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.044+0000 7f8d61200640 1 -- 172.21.15.62:0/937544697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5c1ab800 con 0x7f8d5c1082b0 2024-04-02T14:44:42.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.047+0000 7f8d65409740 1 Processor -- start 2024-04-02T14:44:42.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.047+0000 7f8d53e00640 1 -- 172.21.15.62:0/937544697 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f8d4c018ca0 con 0x7f8d5c1082b0 2024-04-02T14:44:42.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.047+0000 7f8d65409740 1 -- start start 2024-04-02T14:44:42.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.047+0000 7f8d65409740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae6418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.047+0000 7f8d65409740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae641ec0 0x55dfae642340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.047+0000 7f8d65409740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae64b230 0x55dfae64d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d65409740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dfae582400 con 0x55dfae64b230 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d65409740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dfae566f10 con 0x55dfae641ec0 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d65409740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dfae5a66f0 con 0x55dfae642ba0 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d5be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae6418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d5be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae6418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:57610/0 (socket says 172.21.15.62:57610) 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d5be00640 1 -- 172.21.15.62:0/4252355246 learned_addr learned my addr 172.21.15.62:0/4252355246 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/4252355246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae64b230 0x55dfae64d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d5b400640 1 --2- 172.21.15.62:0/4252355246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae641ec0 0x55dfae642340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d5be00640 1 -- 172.21.15.62:0/4252355246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae641ec0 msgr2=0x55dfae642340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d5be00640 1 --2- 172.21.15.62:0/4252355246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae641ec0 0x55dfae642340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d5be00640 1 -- 172.21.15.62:0/4252355246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae64b230 msgr2=0x55dfae64d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d5be00640 1 --2- 172.21.15.62:0/4252355246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae64b230 0x55dfae64d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.048+0000 7f8d5be00640 1 -- 172.21.15.62:0/4252355246 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfae2ba3f0 con 0x55dfae642ba0 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.049+0000 7f8d53e00640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8d400a3e40 0x7f8d400a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.049+0000 7f8d5be00640 1 --2- 172.21.15.62:0/4252355246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae6418f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009f40 tx=0x7f8d540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=887ace694b21bce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.049+0000 7f8d53e00640 1 -- 172.21.15.62:0/937544697 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8d4c0d0de0 con 0x7f8d5c1082b0 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.049+0000 7f8d5b400640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8d400a3e40 0x7f8d400a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.049+0000 7f8d50c00640 1 -- 172.21.15.62:0/4252355246 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54004d20 con 0x55dfae642ba0 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.049+0000 7f8d50c00640 1 -- 172.21.15.62:0/4252355246 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8d54004050 con 0x55dfae642ba0 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.049+0000 7f8d50c00640 1 -- 172.21.15.62:0/4252355246 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54005220 con 0x55dfae642ba0 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.050+0000 7f8d65409740 1 -- 172.21.15.62:0/4252355246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 msgr2=0x55dfae6418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.050+0000 7f8d65409740 1 --2- 172.21.15.62:0/4252355246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae6418f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009f40 tx=0x7f8d540118d0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.050+0000 7f8d5b400640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8d400a3e40 0x7f8d400a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0ce460 tx=0x7f8d4c000bc0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.050+0000 7f8d65409740 1 -- 172.21.15.62:0/4252355246 shutdown_connections 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.050+0000 7f8d65409740 1 --2- 172.21.15.62:0/4252355246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae64b230 0x55dfae64d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.050+0000 7f8d65409740 1 --2- 172.21.15.62:0/4252355246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae641ec0 0x55dfae642340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.050+0000 7f8d65409740 1 --2- 172.21.15.62:0/4252355246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae6418f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.050+0000 7f8d65409740 1 -- 172.21.15.62:0/4252355246 >> 172.21.15.62:0/4252355246 conn(0x55dfae608a50 msgr2=0x55dfae63c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.050+0000 7f8d65409740 1 -- 172.21.15.62:0/4252355246 shutdown_connections 2024-04-02T14:44:42.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.050+0000 7f8d65409740 1 -- 172.21.15.62:0/4252355246 wait complete. 2024-04-02T14:44:42.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.050+0000 7f8d65409740 1 -- start start 2024-04-02T14:44:42.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d65409740 1 Processor -- start 2024-04-02T14:44:42.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d65409740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae641ec0 0x55dfae6481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d65409740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae642ba0 0x55dfae6486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d65409740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae64b230 0x55dfae648c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d65409740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dfae5a26a0 con 0x55dfae642ba0 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d65409740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dfae54e980 con 0x55dfae641ec0 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d65409740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dfae561cb0 con 0x55dfae64b230 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d65409740 10 client.0 _ll_register_callbacks cb 0x55dfae5616a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d5aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae642ba0 0x55dfae6486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d5aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae642ba0 0x55dfae6486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50578/0 (socket says 172.21.15.62:50578) 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d5aa00640 1 -- 172.21.15.62:0/3167009703 learned_addr learned my addr 172.21.15.62:0/3167009703 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.051+0000 7f8d5be00640 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae641ec0 0x55dfae6481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.052+0000 7f8d5be00640 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae64b230 0x55dfae648c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.052+0000 7f8d5aa00640 1 -- 172.21.15.62:0/3167009703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae64b230 msgr2=0x55dfae648c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.052+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae64b230 0x55dfae648c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.052+0000 7f8d5aa00640 1 -- 172.21.15.62:0/3167009703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae641ec0 msgr2=0x55dfae6481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.052+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae641ec0 0x55dfae6481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.052+0000 7f8d5aa00640 1 -- 172.21.15.62:0/3167009703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfae3847d0 con 0x55dfae642ba0 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.052+0000 7f8d5be00640 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae64b230 0x55dfae648c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.052+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae642ba0 0x55dfae6486f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c109220 tx=0x7f8d4802dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3fc6e3ff366086f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.052+0000 7f8d45600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.053+0000 7f8d45600640 1 -- 172.21.15.62:0/3167009703 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48044070 con 0x55dfae642ba0 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.053+0000 7f8d45600640 1 -- 172.21.15.62:0/3167009703 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8d48004310 con 0x55dfae642ba0 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.053+0000 7f8d65409740 1 -- 172.21.15.62:0/3167009703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d54004050 con 0x55dfae642ba0 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.053+0000 7f8d45600640 1 -- 172.21.15.62:0/3167009703 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d48050050 con 0x55dfae642ba0 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.053+0000 7f8d65409740 10 client.15495 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.053+0000 7f8d65409740 1 -- 172.21.15.62:0/3167009703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dfae65a060 con 0x55dfae642ba0 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.053+0000 7f8d45600640 1 -- 172.21.15.62:0/3167009703 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8d54004050 con 0x55dfae642ba0 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.053+0000 7f8d45600640 1 -- 172.21.15.62:0/3167009703 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8d480035f0 con 0x55dfae642ba0 2024-04-02T14:44:42.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.053+0000 7f8d65409740 10 client.15495 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:42.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.053+0000 7f8d65409740 10 client.15495 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:44:42.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.054+0000 7f8d65409740 20 client.15495 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:42.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.054+0000 7f8d65409740 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55dfae65b470 0x55dfae65d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.054+0000 7f8d65409740 4 client.15495 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:42.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.054+0000 7f8d65409740 1 -- 172.21.15.62:0/3167009703 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfae566040 con 0x55dfae65b470 2024-04-02T14:44:42.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.054+0000 7f8d5be00640 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55dfae65b470 0x55dfae65d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.055+0000 7f8d5be00640 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55dfae65b470 0x55dfae65d880 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1cfd92fc23b1b75 server_cookie=93717cd5b1b135c in_seq=0 out_seq=0 2024-04-02T14:44:42.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.055+0000 7f8d45600640 10 client.15495 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:44:42.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.055+0000 7f8d45600640 1 -- 172.21.15.62:0/3167009703 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfae566040 con 0x55dfae65b470 2024-04-02T14:44:42.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.055+0000 7f8d45600640 10 client.15495 handle_command_reply: tid=0 2024-04-02T14:44:42.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.055+0000 7f8d65409740 1 client.15495 shutdown 2024-04-02T14:44:42.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.055+0000 7f8d65409740 1 -- 172.21.15.62:0/3167009703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae642ba0 msgr2=0x55dfae6486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.056+0000 7f8d65409740 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae642ba0 0x55dfae6486f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c109220 tx=0x7f8d4802dd60 comp rx=0 tx=0).stop 2024-04-02T14:44:42.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.056+0000 7f8d65409740 1 -- 172.21.15.62:0/3167009703 shutdown_connections 2024-04-02T14:44:42.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.056+0000 7f8d65409740 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55dfae65b470 0x55dfae65d880 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.056+0000 7f8d65409740 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae64b230 0x55dfae648c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.056+0000 7f8d65409740 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae642ba0 0x55dfae6486f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.056+0000 7f8d65409740 1 --2- 172.21.15.62:0/3167009703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae641ec0 0x55dfae6481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.056+0000 7f8d65409740 1 -- 172.21.15.62:0/3167009703 >> 172.21.15.62:0/3167009703 conn(0x55dfae608a50 msgr2=0x55dfae63e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.056+0000 7f8d45600640 0 client.15495 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:44:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.056+0000 7f8d65409740 1 -- 172.21.15.62:0/3167009703 shutdown_connections 2024-04-02T14:44:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.056+0000 7f8d65409740 1 -- 172.21.15.62:0/3167009703 wait complete. 2024-04-02T14:44:42.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.056+0000 7f8d65409740 20 client.15495 trim_cache size 0 max 16384 2024-04-02T14:44:42.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.071+0000 7f8d65409740 1 Processor -- start 2024-04-02T14:44:42.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.071+0000 7f8d65409740 1 -- start start 2024-04-02T14:44:42.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.071+0000 7f8d65409740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 0x55dfae64a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.071+0000 7f8d65409740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae656ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.071+0000 7f8d65409740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae64b230 0x55dfae6631e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.071+0000 7f8d65409740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c1b3c30 con 0x55dfae641ec0 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.071+0000 7f8d65409740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c1b5770 con 0x55dfae64b230 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.071+0000 7f8d65409740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c1b4300 con 0x55dfae642ba0 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.071+0000 7f8d5be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 0x55dfae64a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.071+0000 7f8d5be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 0x55dfae64a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50594/0 (socket says 172.21.15.62:50594) 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.071+0000 7f8d5be00640 1 -- 172.21.15.62:0/2115848882 learned_addr learned my addr 172.21.15.62:0/2115848882 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/2115848882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae656ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d5be00640 1 --2- 172.21.15.62:0/2115848882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae64b230 0x55dfae6631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d5aa00640 1 -- 172.21.15.62:0/2115848882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae64b230 msgr2=0x55dfae6631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/2115848882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae64b230 0x55dfae6631e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d5aa00640 1 -- 172.21.15.62:0/2115848882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 msgr2=0x55dfae64a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/2115848882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 0x55dfae64a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d5aa00640 1 -- 172.21.15.62:0/2115848882 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfae640ef0 con 0x55dfae642ba0 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d5be00640 1 --2- 172.21.15.62:0/2115848882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae64b230 0x55dfae6631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/2115848882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae656ca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8d4802c180 tx=0x7f8d4802bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5f7c1b0a1beba7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d47e00640 1 -- 172.21.15.62:0/2115848882 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4804d070 con 0x55dfae642ba0 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d47e00640 1 -- 172.21.15.62:0/2115848882 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8d48004310 con 0x55dfae642ba0 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.072+0000 7f8d47e00640 1 -- 172.21.15.62:0/2115848882 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48044040 con 0x55dfae642ba0 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.073+0000 7f8d65409740 1 -- 172.21.15.62:0/2115848882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 msgr2=0x55dfae656ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.073+0000 7f8d65409740 1 --2- 172.21.15.62:0/2115848882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae656ca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8d4802c180 tx=0x7f8d4802bc40 comp rx=0 tx=0).stop 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.073+0000 7f8d65409740 1 -- 172.21.15.62:0/2115848882 shutdown_connections 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.073+0000 7f8d65409740 1 --2- 172.21.15.62:0/2115848882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae64b230 0x55dfae6631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.073+0000 7f8d65409740 1 --2- 172.21.15.62:0/2115848882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae656ca0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.073+0000 7f8d65409740 1 --2- 172.21.15.62:0/2115848882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 0x55dfae64a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.073+0000 7f8d65409740 1 -- 172.21.15.62:0/2115848882 >> 172.21.15.62:0/2115848882 conn(0x55dfae608a50 msgr2=0x55dfae643540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.073+0000 7f8d65409740 1 -- 172.21.15.62:0/2115848882 shutdown_connections 2024-04-02T14:44:42.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.073+0000 7f8d65409740 1 -- 172.21.15.62:0/2115848882 wait complete. 2024-04-02T14:44:42.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d65409740 1 -- start start 2024-04-02T14:44:42.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d65409740 1 Processor -- start 2024-04-02T14:44:42.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d65409740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 0x55dfae653600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d65409740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae645450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d65409740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae64b230 0x55dfae6459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d65409740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c1b50a0 con 0x55dfae641ec0 2024-04-02T14:44:42.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d65409740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c1b49d0 con 0x55dfae64b230 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d65409740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c1b1c30 con 0x55dfae642ba0 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d65409740 10 client.0 _ll_register_callbacks cb 0x7f8d4802d9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d5be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 0x55dfae653600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d5be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 0x55dfae653600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50602/0 (socket says 172.21.15.62:50602) 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.074+0000 7f8d5be00640 1 -- 172.21.15.62:0/4102496043 learned_addr learned my addr 172.21.15.62:0/4102496043 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.075+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae645450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.075+0000 7f8d5be00640 1 -- 172.21.15.62:0/4102496043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 msgr2=0x55dfae645450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.075+0000 7f8d5be00640 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae645450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.075+0000 7f8d5be00640 1 -- 172.21.15.62:0/4102496043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae64b230 msgr2=0x55dfae6459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.075+0000 7f8d5be00640 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae64b230 0x55dfae6459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.075+0000 7f8d5be00640 1 -- 172.21.15.62:0/4102496043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfae646220 con 0x55dfae641ec0 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.075+0000 7f8d5be00640 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 0x55dfae653600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d5401b040 tx=0x7f8d54009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df08cbb2f8caf002 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.075+0000 7f8d45600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:42.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.075+0000 7f8d45600640 1 -- 172.21.15.62:0/4102496043 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54037060 con 0x55dfae641ec0 2024-04-02T14:44:42.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.075+0000 7f8d45600640 1 -- 172.21.15.62:0/4102496043 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8d54004050 con 0x55dfae641ec0 2024-04-02T14:44:42.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.075+0000 7f8d65409740 1 -- 172.21.15.62:0/4102496043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d48004310 con 0x55dfae641ec0 2024-04-02T14:44:42.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.076+0000 7f8d45600640 1 -- 172.21.15.62:0/4102496043 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d54004250 con 0x55dfae641ec0 2024-04-02T14:44:42.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.076+0000 7f8d65409740 10 client.15501 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:42.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.076+0000 7f8d65409740 1 -- 172.21.15.62:0/4102496043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dfae6c1b10 con 0x55dfae641ec0 2024-04-02T14:44:42.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.076+0000 7f8d45600640 1 -- 172.21.15.62:0/4102496043 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8d48004310 con 0x55dfae641ec0 2024-04-02T14:44:42.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.076+0000 7f8d45600640 1 -- 172.21.15.62:0/4102496043 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8d5400f480 con 0x55dfae641ec0 2024-04-02T14:44:42.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.076+0000 7f8d65409740 10 client.15501 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:42.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.076+0000 7f8d65409740 10 client.15501 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:44:42.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.076+0000 7f8d65409740 20 client.15501 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:42.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.076+0000 7f8d65409740 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55dfae6c2020 0x55dfae6c4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.076+0000 7f8d65409740 4 client.15501 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:44:42.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.076+0000 7f8d65409740 1 -- 172.21.15.62:0/4102496043 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfae5b3c30 con 0x55dfae6c2020 2024-04-02T14:44:42.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.077+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55dfae6c2020 0x55dfae6c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.077+0000 7f8d5aa00640 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55dfae6c2020 0x55dfae6c4440 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=133e67afff195e44 server_cookie=ee8e4a3a4520e999 in_seq=0 out_seq=0 2024-04-02T14:44:42.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.077+0000 7f8d45600640 10 client.15501 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:44:42.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.077+0000 7f8d45600640 1 -- 172.21.15.62:0/4102496043 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfae5b3c30 con 0x55dfae6c2020 2024-04-02T14:44:42.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.077+0000 7f8d45600640 10 client.15501 handle_command_reply: tid=0 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.077+0000 7f8d65409740 1 client.15501 shutdown 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.077+0000 7f8d65409740 1 -- 172.21.15.62:0/4102496043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 msgr2=0x55dfae653600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.077+0000 7f8d65409740 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 0x55dfae653600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d5401b040 tx=0x7f8d54009b90 comp rx=0 tx=0).stop 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d65409740 1 -- 172.21.15.62:0/4102496043 shutdown_connections 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d65409740 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55dfae6c2020 0x55dfae6c4440 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d65409740 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dfae64b230 0x55dfae6459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d65409740 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfae642ba0 0x55dfae645450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d65409740 1 --2- 172.21.15.62:0/4102496043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfae641ec0 0x55dfae653600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d45600640 0 client.15501 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d65409740 1 -- 172.21.15.62:0/4102496043 >> 172.21.15.62:0/4102496043 conn(0x55dfae608a50 msgr2=0x55dfae648150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d65409740 1 -- 172.21.15.62:0/4102496043 shutdown_connections 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d65409740 1 -- 172.21.15.62:0/4102496043 wait complete. 2024-04-02T14:44:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d65409740 20 client.15501 trim_cache size 0 max 16384 2024-04-02T14:44:42.080 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:44:42.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d61200640 1 -- 172.21.15.62:0/937544697 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8d400a3e40 msgr2=0x7f8d400a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d61200640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8d400a3e40 0x7f8d400a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0ce460 tx=0x7f8d4c000bc0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d61200640 1 -- 172.21.15.62:0/937544697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c1082b0 msgr2=0x7f8d5c1a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.078+0000 7f8d61200640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c1082b0 0x7f8d5c1a4610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00e9f0 tx=0x7f8d4c00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.079+0000 7f8d61200640 1 -- 172.21.15.62:0/937544697 shutdown_connections 2024-04-02T14:44:42.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.079+0000 7f8d61200640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8d400a3e40 0x7f8d400a6300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.079+0000 7f8d61200640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5c19e7d0 0x7f8d5c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.079+0000 7f8d61200640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c10d300 0x7f8d5c1a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.079+0000 7f8d61200640 1 --2- 172.21.15.62:0/937544697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c1082b0 0x7f8d5c1a4610 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.079+0000 7f8d61200640 1 -- 172.21.15.62:0/937544697 >> 172.21.15.62:0/937544697 conn(0x7f8d5c077920 msgr2=0x7f8d5c077f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.079+0000 7f8d61200640 1 -- 172.21.15.62:0/937544697 shutdown_connections 2024-04-02T14:44:42.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.079+0000 7f8d61200640 1 -- 172.21.15.62:0/937544697 wait complete. 2024-04-02T14:44:42.094 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:44:42.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.307+0000 7fc507200640 1 -- 172.21.15.62:0/808955057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc508101970 msgr2=0x7fc508101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.307+0000 7fc507200640 1 --2- 172.21.15.62:0/808955057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc508101970 0x7fc508101df0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009930 tx=0x7fc4f002bc20 comp rx=0 tx=0).stop 2024-04-02T14:44:42.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.307+0000 7fc507200640 1 -- 172.21.15.62:0/808955057 shutdown_connections 2024-04-02T14:44:42.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.307+0000 7fc507200640 1 --2- 172.21.15.62:0/808955057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc508102330 0x7fc50810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.307+0000 7fc507200640 1 --2- 172.21.15.62:0/808955057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc508101970 0x7fc508101df0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.307+0000 7fc507200640 1 --2- 172.21.15.62:0/808955057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc508107980 0x7fc508107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.307+0000 7fc507200640 1 -- 172.21.15.62:0/808955057 >> 172.21.15.62:0/808955057 conn(0x7fc5080fd460 msgr2=0x7fc5080ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.307+0000 7fc507200640 1 -- 172.21.15.62:0/808955057 shutdown_connections 2024-04-02T14:44:42.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.307+0000 7fc507200640 1 -- 172.21.15.62:0/808955057 wait complete. 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.308+0000 7fc507200640 1 Processor -- start 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.308+0000 7fc507200640 1 -- start start 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.308+0000 7fc507200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc508101970 0x7fc50819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.308+0000 7fc507200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc508102330 0x7fc50819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.308+0000 7fc507200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc508107980 0x7fc50819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.308+0000 7fc507200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc508111eb0 con 0x7fc508102330 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.308+0000 7fc507200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc508111d30 con 0x7fc508101970 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.308+0000 7fc507200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc508112030 con 0x7fc508107980 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.309+0000 7fc505400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc508102330 0x7fc50819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.309+0000 7fc505e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc508101970 0x7fc50819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.309+0000 7fc505e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc508101970 0x7fc50819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:52062/0 (socket says 172.21.15.62:52062) 2024-04-02T14:44:42.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.309+0000 7fc505e00640 1 -- 172.21.15.62:0/3715535727 learned_addr learned my addr 172.21.15.62:0/3715535727 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.309+0000 7fc506800640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc508107980 0x7fc50819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.309+0000 7fc505e00640 1 -- 172.21.15.62:0/3715535727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc508107980 msgr2=0x7fc50819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.309+0000 7fc505e00640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc508107980 0x7fc50819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.309+0000 7fc505e00640 1 -- 172.21.15.62:0/3715535727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc508102330 msgr2=0x7fc50819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.309+0000 7fc505e00640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc508102330 0x7fc50819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.309+0000 7fc505e00640 1 -- 172.21.15.62:0/3715535727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50819f580 con 0x7fc508101970 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.309+0000 7fc505e00640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc508101970 0x7fc50819a5e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800ea60 tx=0x7fc4f800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.310+0000 7fc4fea00640 1 -- 172.21.15.62:0/3715535727 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f800cde0 con 0x7fc508101970 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.310+0000 7fc4fea00640 1 -- 172.21.15.62:0/3715535727 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc4f80035f0 con 0x7fc508101970 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.310+0000 7fc507200640 1 -- 172.21.15.62:0/3715535727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50819f870 con 0x7fc508101970 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.310+0000 7fc4fea00640 1 -- 172.21.15.62:0/3715535727 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8010720 con 0x7fc508101970 2024-04-02T14:44:42.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.310+0000 7fc507200640 1 -- 172.21.15.62:0/3715535727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5081a7100 con 0x7fc508101970 2024-04-02T14:44:42.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.313+0000 7fc4fea00640 1 -- 172.21.15.62:0/3715535727 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fc4f80037a0 con 0x7fc508101970 2024-04-02T14:44:42.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.313+0000 7fc50fdcf740 1 Processor -- start 2024-04-02T14:44:42.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.313+0000 7fc50fdcf740 1 -- start start 2024-04-02T14:44:42.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620bba0 0x56515620a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620aec0 0x56515620b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 0x565156216620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56515614b400 con 0x56515620aec0 2024-04-02T14:44:42.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56515612ff10 con 0x565156214230 2024-04-02T14:44:42.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56515616f6f0 con 0x56515620bba0 2024-04-02T14:44:42.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc505e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620aec0 0x56515620b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc505400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 0x565156216620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc506800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620bba0 0x56515620a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc505400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 0x565156216620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:52064/0 (socket says 172.21.15.62:52064) 2024-04-02T14:44:42.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc505400640 1 -- 172.21.15.62:0/1764548665 learned_addr learned my addr 172.21.15.62:0/1764548665 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc505400640 1 -- 172.21.15.62:0/1764548665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620bba0 msgr2=0x56515620a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc505400640 1 --2- 172.21.15.62:0/1764548665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620bba0 0x56515620a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc505400640 1 -- 172.21.15.62:0/1764548665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620aec0 msgr2=0x56515620b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.314+0000 7fc505400640 1 --2- 172.21.15.62:0/1764548665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620aec0 0x56515620b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.315+0000 7fc505400640 1 -- 172.21.15.62:0/1764548665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565155e833f0 con 0x565156214230 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.315+0000 7fc4fea00640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc4ec0a3ec0 0x7fc4ec0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.315+0000 7fc505400640 1 --2- 172.21.15.62:0/1764548665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 0x565156216620 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009930 tx=0x7fc4f00095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9133129c9b4bd738 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.315+0000 7fc4fea00640 1 -- 172.21.15.62:0/3715535727 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc4f80d0da0 con 0x7fc508101970 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.315+0000 7fc506800640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc4ec0a3ec0 0x7fc4ec0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.315+0000 7fc4f7400640 1 -- 172.21.15.62:0/1764548665 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f002c740 con 0x565156214230 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.315+0000 7fc4f7400640 1 -- 172.21.15.62:0/1764548665 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc4f0004040 con 0x565156214230 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.315+0000 7fc4f7400640 1 -- 172.21.15.62:0/1764548665 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00046c0 con 0x565156214230 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.316+0000 7fc50fdcf740 1 -- 172.21.15.62:0/1764548665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 msgr2=0x565156216620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.316+0000 7fc506800640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc4ec0a3ec0 0x7fc4ec0a6380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc500005fd0 tx=0x7fc500009500 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.316+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/1764548665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 0x565156216620 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009930 tx=0x7fc4f00095f0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.316+0000 7fc50fdcf740 1 -- 172.21.15.62:0/1764548665 shutdown_connections 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.316+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/1764548665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 0x565156216620 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.316+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/1764548665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620aec0 0x56515620b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.316+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/1764548665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620bba0 0x56515620a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.316+0000 7fc50fdcf740 1 -- 172.21.15.62:0/1764548665 >> 172.21.15.62:0/1764548665 conn(0x5651561d1a50 msgr2=0x565156205910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.316+0000 7fc50fdcf740 1 -- 172.21.15.62:0/1764548665 shutdown_connections 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.316+0000 7fc50fdcf740 1 -- 172.21.15.62:0/1764548665 wait complete. 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.317+0000 7fc50fdcf740 1 -- start start 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.317+0000 7fc50fdcf740 1 Processor -- start 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.317+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620aec0 0x565156211180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.317+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620bba0 0x5651562116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.317+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 0x565156211c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.317+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56515616b6a0 con 0x56515620bba0 2024-04-02T14:44:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x565156117980 con 0x565156214230 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56515612acb0 con 0x56515620aec0 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc50fdcf740 10 client.0 _ll_register_callbacks cb 0x56515612a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc505400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620bba0 0x5651562116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc505400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620bba0 0x5651562116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55000/0 (socket says 172.21.15.62:55000) 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc505400640 1 -- 172.21.15.62:0/75011833 learned_addr learned my addr 172.21.15.62:0/75011833 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc505e00640 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620aec0 0x565156211180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc506800640 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 0x565156211c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc505400640 1 -- 172.21.15.62:0/75011833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620aec0 msgr2=0x565156211180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc505400640 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620aec0 0x565156211180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc505400640 1 -- 172.21.15.62:0/75011833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 msgr2=0x565156211c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc505400640 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 0x565156211c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.318+0000 7fc505400640 1 -- 172.21.15.62:0/75011833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565155f4d7d0 con 0x56515620bba0 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.319+0000 7fc505400640 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620bba0 0x5651562116c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0036040 tx=0x7fc4f0005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4dbfa0818b5543 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.319+0000 7fc4ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:42.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.319+0000 7fc4ebe00640 1 -- 172.21.15.62:0/75011833 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f002cbc0 con 0x56515620bba0 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.319+0000 7fc4ebe00640 1 -- 172.21.15.62:0/75011833 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc4f002cd60 con 0x56515620bba0 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.319+0000 7fc50fdcf740 1 -- 172.21.15.62:0/75011833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f0004040 con 0x56515620bba0 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.319+0000 7fc4ebe00640 1 -- 172.21.15.62:0/75011833 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4f0059050 con 0x56515620bba0 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.319+0000 7fc50fdcf740 10 client.15513 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.319+0000 7fc50fdcf740 1 -- 172.21.15.62:0/75011833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565156222fd0 con 0x56515620bba0 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.320+0000 7fc4ebe00640 1 -- 172.21.15.62:0/75011833 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc4f0004040 con 0x56515620bba0 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.320+0000 7fc4ebe00640 1 -- 172.21.15.62:0/75011833 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc4f00047c0 con 0x56515620bba0 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.320+0000 7fc50fdcf740 10 client.15513 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.320+0000 7fc50fdcf740 10 client.15513 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.320+0000 7fc50fdcf740 20 client.15513 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.320+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x565156224320 0x565156226710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.320+0000 7fc50fdcf740 4 client.15513 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.320+0000 7fc50fdcf740 1 -- 172.21.15.62:0/75011833 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56515612f040 con 0x565156224320 2024-04-02T14:44:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.320+0000 7fc505e00640 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x565156224320 0x565156226710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.321+0000 7fc505e00640 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x565156224320 0x565156226710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=381441469342aeb3 server_cookie=7a6afbc421c41a1e in_seq=0 out_seq=0 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.321+0000 7fc4ebe00640 10 client.15513 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.321+0000 7fc4ebe00640 1 -- 172.21.15.62:0/75011833 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56515612f040 con 0x565156224320 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.321+0000 7fc4ebe00640 10 client.15513 handle_command_reply: tid=0 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 1 client.15513 shutdown 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 1 -- 172.21.15.62:0/75011833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620bba0 msgr2=0x5651562116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620bba0 0x5651562116c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0036040 tx=0x7fc4f0005870 comp rx=0 tx=0).stop 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 1 -- 172.21.15.62:0/75011833 shutdown_connections 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x565156224320 0x565156226710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565156214230 0x565156211c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620bba0 0x5651562116c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/75011833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620aec0 0x565156211180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 1 -- 172.21.15.62:0/75011833 >> 172.21.15.62:0/75011833 conn(0x5651561d1a50 msgr2=0x565156207660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc4ebe00640 0 client.15513 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:44:42.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 1 -- 172.21.15.62:0/75011833 shutdown_connections 2024-04-02T14:44:42.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 1 -- 172.21.15.62:0/75011833 wait complete. 2024-04-02T14:44:42.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.322+0000 7fc50fdcf740 20 client.15513 trim_cache size 0 max 16384 2024-04-02T14:44:42.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.337+0000 7fc50fdcf740 1 Processor -- start 2024-04-02T14:44:42.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.337+0000 7fc50fdcf740 1 -- start start 2024-04-02T14:44:42.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.337+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620aec0 0x5651562138b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.337+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56515620bba0 0x56515621fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.337+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565156214230 0x565156204970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.337+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081af820 con 0x56515620aec0 2024-04-02T14:44:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.337+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b1360 con 0x56515620bba0 2024-04-02T14:44:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.337+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081afef0 con 0x565156214230 2024-04-02T14:44:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc506800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565156214230 0x565156204970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc505400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56515620bba0 0x56515621fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc505e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620aec0 0x5651562138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc506800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565156214230 0x565156204970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:60308/0 (socket says 172.21.15.62:60308) 2024-04-02T14:44:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc506800640 1 -- 172.21.15.62:0/1045394405 learned_addr learned my addr 172.21.15.62:0/1045394405 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc505400640 1 -- 172.21.15.62:0/1045394405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565156214230 msgr2=0x565156204970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc505400640 1 --2- 172.21.15.62:0/1045394405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565156214230 0x565156204970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc505400640 1 -- 172.21.15.62:0/1045394405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620aec0 msgr2=0x5651562138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc505400640 1 --2- 172.21.15.62:0/1045394405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620aec0 0x5651562138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc505400640 1 -- 172.21.15.62:0/1045394405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565156205b20 con 0x56515620bba0 2024-04-02T14:44:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc506800640 1 --2- 172.21.15.62:0/1045394405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565156214230 0x565156204970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.338+0000 7fc505e00640 1 --2- 172.21.15.62:0/1045394405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620aec0 0x5651562138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.339+0000 7fc505400640 1 --2- 172.21.15.62:0/1045394405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56515620bba0 0x56515621fc40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0004c10 tx=0x7fc4f002bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72bd02f86329c345 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.339+0000 7fc4f6a00640 1 -- 172.21.15.62:0/1045394405 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0005d50 con 0x56515620bba0 2024-04-02T14:44:42.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.339+0000 7fc4f6a00640 1 -- 172.21.15.62:0/1045394405 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc4f00044c0 con 0x56515620bba0 2024-04-02T14:44:42.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.339+0000 7fc4f6a00640 1 -- 172.21.15.62:0/1045394405 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00347a0 con 0x56515620bba0 2024-04-02T14:44:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.339+0000 7fc50fdcf740 1 -- 172.21.15.62:0/1045394405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56515620bba0 msgr2=0x56515621fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.340+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/1045394405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56515620bba0 0x56515621fc40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0004c10 tx=0x7fc4f002bc20 comp rx=0 tx=0).stop 2024-04-02T14:44:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.340+0000 7fc50fdcf740 1 -- 172.21.15.62:0/1045394405 shutdown_connections 2024-04-02T14:44:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.340+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/1045394405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565156214230 0x565156204970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.340+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/1045394405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56515620bba0 0x56515621fc40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.340+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/1045394405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56515620aec0 0x5651562138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.340+0000 7fc50fdcf740 1 -- 172.21.15.62:0/1045394405 >> 172.21.15.62:0/1045394405 conn(0x5651561d1a50 msgr2=0x565156204260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.340+0000 7fc50fdcf740 1 -- 172.21.15.62:0/1045394405 shutdown_connections 2024-04-02T14:44:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.340+0000 7fc50fdcf740 1 -- 172.21.15.62:0/1045394405 wait complete. 2024-04-02T14:44:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.340+0000 7fc50fdcf740 1 -- start start 2024-04-02T14:44:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.340+0000 7fc50fdcf740 1 Processor -- start 2024-04-02T14:44:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56515620aec0 0x56515621c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620bba0 0x56515620d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc50fdcf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565156214230 0x56515620dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b0c90 con 0x565156214230 2024-04-02T14:44:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b05c0 con 0x56515620aec0 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc50fdcf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081ad800 con 0x56515620bba0 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc50fdcf740 10 client.0 _ll_register_callbacks cb 0x7fc4f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc506800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565156214230 0x56515620dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc505400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620bba0 0x56515620d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc506800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565156214230 0x56515620dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55004/0 (socket says 172.21.15.62:55004) 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc506800640 1 -- 172.21.15.62:0/3650076692 learned_addr learned my addr 172.21.15.62:0/3650076692 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc505e00640 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56515620aec0 0x56515621c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc505400640 1 -- 172.21.15.62:0/3650076692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56515620aec0 msgr2=0x56515621c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc505400640 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56515620aec0 0x56515621c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc505400640 1 -- 172.21.15.62:0/3650076692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565156214230 msgr2=0x56515620dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc505400640 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565156214230 0x56515620dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.341+0000 7fc505400640 1 -- 172.21.15.62:0/3650076692 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56515620e2c0 con 0x56515620bba0 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.342+0000 7fc505400640 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620bba0 0x56515620d840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0004b90 tx=0x7fc4f0005030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7358fb2de06ebdc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.342+0000 7fc4ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:44:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.342+0000 7fc4ebe00640 1 -- 172.21.15.62:0/3650076692 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0005d50 con 0x56515620bba0 2024-04-02T14:44:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.342+0000 7fc50fdcf740 1 -- 172.21.15.62:0/3650076692 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56515617cc30 con 0x56515620bba0 2024-04-02T14:44:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.342+0000 7fc4ebe00640 1 -- 172.21.15.62:0/3650076692 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc4f002cbc0 con 0x56515620bba0 2024-04-02T14:44:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.342+0000 7fc4ebe00640 1 -- 172.21.15.62:0/3650076692 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4f00048d0 con 0x56515620bba0 2024-04-02T14:44:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.342+0000 7fc50fdcf740 10 client.25627 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.342+0000 7fc50fdcf740 1 -- 172.21.15.62:0/3650076692 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56515628a260 con 0x56515620bba0 2024-04-02T14:44:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.342+0000 7fc4ebe00640 1 -- 172.21.15.62:0/3650076692 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x56515617cc30 con 0x56515620bba0 2024-04-02T14:44:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.343+0000 7fc4ebe00640 1 -- 172.21.15.62:0/3650076692 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc4f003f760 con 0x56515620bba0 2024-04-02T14:44:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.343+0000 7fc50fdcf740 10 client.25627 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.343+0000 7fc50fdcf740 10 client.25627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.343+0000 7fc50fdcf740 20 client.25627 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.343+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56515628a640 0x56515628ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.343+0000 7fc50fdcf740 4 client.25627 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.343+0000 7fc50fdcf740 1 -- 172.21.15.62:0/3650076692 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565156176f80 con 0x56515628a640 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.343+0000 7fc505e00640 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56515628a640 0x56515628ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc505e00640 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56515628a640 0x56515628ca30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c73fdd3546ab6d30 server_cookie=f29e7e3e3c6e4492 in_seq=0 out_seq=0 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc4ebe00640 10 client.25627 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc4ebe00640 1 -- 172.21.15.62:0/3650076692 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565156176f80 con 0x56515628a640 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc4ebe00640 10 client.25627 handle_command_reply: tid=0 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc50fdcf740 1 client.25627 shutdown 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc50fdcf740 1 -- 172.21.15.62:0/3650076692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620bba0 msgr2=0x56515620d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620bba0 0x56515620d840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0004b90 tx=0x7fc4f0005030 comp rx=0 tx=0).stop 2024-04-02T14:44:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc50fdcf740 1 -- 172.21.15.62:0/3650076692 shutdown_connections 2024-04-02T14:44:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x56515628a640 0x56515628ca30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565156214230 0x56515620dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56515620bba0 0x56515620d840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc50fdcf740 1 --2- 172.21.15.62:0/3650076692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56515620aec0 0x56515621c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc50fdcf740 1 -- 172.21.15.62:0/3650076692 >> 172.21.15.62:0/3650076692 conn(0x5651561d1a50 msgr2=0x565156204af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc4ebe00640 0 client.25627 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:44:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc50fdcf740 1 -- 172.21.15.62:0/3650076692 shutdown_connections 2024-04-02T14:44:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.344+0000 7fc50fdcf740 1 -- 172.21.15.62:0/3650076692 wait complete. 2024-04-02T14:44:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.345+0000 7fc50fdcf740 20 client.25627 trim_cache size 0 max 16384 2024-04-02T14:44:42.352 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:44:42.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.345+0000 7fc507200640 1 -- 172.21.15.62:0/3715535727 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc4ec0a3ec0 msgr2=0x7fc4ec0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.345+0000 7fc507200640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc4ec0a3ec0 0x7fc4ec0a6380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc500005fd0 tx=0x7fc500009500 comp rx=0 tx=0).stop 2024-04-02T14:44:42.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.345+0000 7fc507200640 1 -- 172.21.15.62:0/3715535727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc508101970 msgr2=0x7fc50819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.345+0000 7fc507200640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc508101970 0x7fc50819a5e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800ea60 tx=0x7fc4f800ef20 comp rx=0 tx=0).stop 2024-04-02T14:44:42.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.345+0000 7fc507200640 1 -- 172.21.15.62:0/3715535727 shutdown_connections 2024-04-02T14:44:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.345+0000 7fc507200640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc4ec0a3ec0 0x7fc4ec0a6380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.345+0000 7fc507200640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc508107980 0x7fc50819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.345+0000 7fc507200640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc508102330 0x7fc50819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.345+0000 7fc507200640 1 --2- 172.21.15.62:0/3715535727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc508101970 0x7fc50819a5e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.345+0000 7fc507200640 1 -- 172.21.15.62:0/3715535727 >> 172.21.15.62:0/3715535727 conn(0x7fc5080fd460 msgr2=0x7fc5080fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.346+0000 7fc507200640 1 -- 172.21.15.62:0/3715535727 shutdown_connections 2024-04-02T14:44:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.346+0000 7fc507200640 1 -- 172.21.15.62:0/3715535727 wait complete. 2024-04-02T14:44:42.361 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:44:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.595+0000 7f3922800640 1 -- 172.21.15.62:0/74646491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c103f10 msgr2=0x7f391c1042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.595+0000 7f3922800640 1 --2- 172.21.15.62:0/74646491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c103f10 0x7f391c1042f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f390c009900 tx=0x7f390c02bc50 comp rx=0 tx=0).stop 2024-04-02T14:44:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.595+0000 7f3922800640 1 -- 172.21.15.62:0/74646491 shutdown_connections 2024-04-02T14:44:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.595+0000 7f3922800640 1 --2- 172.21.15.62:0/74646491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c108fe0 0x7f391c110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.595+0000 7f3922800640 1 --2- 172.21.15.62:0/74646491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c1048c0 0x7f391c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.595+0000 7f3922800640 1 --2- 172.21.15.62:0/74646491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c103f10 0x7f391c1042f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.595+0000 7f3922800640 1 -- 172.21.15.62:0/74646491 >> 172.21.15.62:0/74646491 conn(0x7f391c0ff770 msgr2=0x7f391c101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.595+0000 7f3922800640 1 -- 172.21.15.62:0/74646491 shutdown_connections 2024-04-02T14:44:42.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.595+0000 7f3922800640 1 -- 172.21.15.62:0/74646491 wait complete. 2024-04-02T14:44:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.596+0000 7f3922800640 1 Processor -- start 2024-04-02T14:44:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.596+0000 7f3922800640 1 -- start start 2024-04-02T14:44:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f3922800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c103f10 0x7f391c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f3922800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c108fe0 0x7f391c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f3922800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c19ef70 0x7f391c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f3922800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1141b0 con 0x7f391c19ef70 2024-04-02T14:44:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f3922800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f391c114030 con 0x7f391c108fe0 2024-04-02T14:44:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f3922800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f391c114330 con 0x7f391c103f10 2024-04-02T14:44:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f391be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c19ef70 0x7f391c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f391b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c103f10 0x7f391c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f391aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c108fe0 0x7f391c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f391be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c19ef70 0x7f391c19f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55016/0 (socket says 172.21.15.62:55016) 2024-04-02T14:44:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f391be00640 1 -- 172.21.15.62:0/224964034 learned_addr learned my addr 172.21.15.62:0/224964034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.597+0000 7f391b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c103f10 0x7f391c19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:60314/0 (socket says 172.21.15.62:60314) 2024-04-02T14:44:42.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.598+0000 7f391b400640 1 -- 172.21.15.62:0/224964034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c108fe0 msgr2=0x7f391c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.598+0000 7f391b400640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c108fe0 0x7f391c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.598+0000 7f391b400640 1 -- 172.21.15.62:0/224964034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c19ef70 msgr2=0x7f391c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.598+0000 7f391b400640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c19ef70 0x7f391c19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.598+0000 7f391b400640 1 -- 172.21.15.62:0/224964034 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c19fa60 con 0x7f391c103f10 2024-04-02T14:44:42.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.598+0000 7f391be00640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c19ef70 0x7f391c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:42.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.598+0000 7f391b400640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c103f10 0x7f391c19a6b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f390c009230 tx=0x7f390c02dc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.598+0000 7f3913e00640 1 -- 172.21.15.62:0/224964034 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c043020 con 0x7f391c103f10 2024-04-02T14:44:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.598+0000 7f3913e00640 1 -- 172.21.15.62:0/224964034 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f390c004250 con 0x7f391c103f10 2024-04-02T14:44:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.598+0000 7f3922800640 1 -- 172.21.15.62:0/224964034 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c19fcf0 con 0x7f391c103f10 2024-04-02T14:44:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.598+0000 7f3913e00640 1 -- 172.21.15.62:0/224964034 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c0039f0 con 0x7f391c103f10 2024-04-02T14:44:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.599+0000 7f3922800640 1 -- 172.21.15.62:0/224964034 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c1a7510 con 0x7f391c103f10 2024-04-02T14:44:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.602+0000 7f3925479740 1 Processor -- start 2024-04-02T14:44:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.602+0000 7f3925479740 1 -- start start 2024-04-02T14:44:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.602+0000 7f3913e00640 1 -- 172.21.15.62:0/224964034 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f390c046020 con 0x7f391c103f10 2024-04-02T14:44:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.602+0000 7f3925479740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4f200 0x55c535b4df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.602+0000 7f3925479740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4e520 0x55c535b4e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.602+0000 7f3925479740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b57890 0x55c535b59c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.602+0000 7f3925479740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c535a86400 con 0x55c535b4e520 2024-04-02T14:44:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.602+0000 7f3925479740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c535a6af10 con 0x55c535b4f200 2024-04-02T14:44:42.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.602+0000 7f3925479740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c535aaa6f0 con 0x55c535b57890 2024-04-02T14:44:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.603+0000 7f391b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4e520 0x55c535b4e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.603+0000 7f391b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4e520 0x55c535b4e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55030/0 (socket says 172.21.15.62:55030) 2024-04-02T14:44:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.603+0000 7f391b400640 1 -- 172.21.15.62:0/768744766 learned_addr learned my addr 172.21.15.62:0/768744766 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.603+0000 7f391be00640 1 --2- 172.21.15.62:0/768744766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4f200 0x55c535b4df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.603+0000 7f391aa00640 1 --2- 172.21.15.62:0/768744766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b57890 0x55c535b59c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.603+0000 7f391be00640 1 -- 172.21.15.62:0/768744766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b57890 msgr2=0x55c535b59c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.603+0000 7f391be00640 1 --2- 172.21.15.62:0/768744766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b57890 0x55c535b59c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.603+0000 7f391be00640 1 -- 172.21.15.62:0/768744766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4e520 msgr2=0x55c535b4e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.603+0000 7f391be00640 1 --2- 172.21.15.62:0/768744766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4e520 0x55c535b4e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.603+0000 7f391be00640 1 -- 172.21.15.62:0/768744766 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5357be3f0 con 0x55c535b4f200 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.603+0000 7f391be00640 1 --2- 172.21.15.62:0/768744766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4f200 0x55c535b4df50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3914011760 tx=0x7f3914011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108daba0c7c7522c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.604+0000 7f3910c00640 1 -- 172.21.15.62:0/768744766 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914004d20 con 0x55c535b4f200 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.604+0000 7f3910c00640 1 -- 172.21.15.62:0/768744766 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3914005020 con 0x55c535b4f200 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.604+0000 7f3910c00640 1 -- 172.21.15.62:0/768744766 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914019690 con 0x55c535b4f200 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.604+0000 7f3913e00640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f39000a3e40 0x7f39000a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.604+0000 7f3913e00640 1 -- 172.21.15.62:0/224964034 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f390c0f28b0 con 0x7f391c103f10 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.604+0000 7f391b400640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f39000a3e40 0x7f39000a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.605+0000 7f3925479740 1 -- 172.21.15.62:0/768744766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4f200 msgr2=0x55c535b4df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.605+0000 7f3925479740 1 --2- 172.21.15.62:0/768744766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4f200 0x55c535b4df50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3914011760 tx=0x7f3914011c20 comp rx=0 tx=0).stop 2024-04-02T14:44:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.605+0000 7f3925479740 1 -- 172.21.15.62:0/768744766 shutdown_connections 2024-04-02T14:44:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.605+0000 7f3925479740 1 --2- 172.21.15.62:0/768744766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b57890 0x55c535b59c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.605+0000 7f3925479740 1 --2- 172.21.15.62:0/768744766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4e520 0x55c535b4e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.605+0000 7f3925479740 1 --2- 172.21.15.62:0/768744766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4f200 0x55c535b4df50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.605+0000 7f3925479740 1 -- 172.21.15.62:0/768744766 >> 172.21.15.62:0/768744766 conn(0x55c535b0ca50 msgr2=0x55c535b48f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.605+0000 7f3925479740 1 -- 172.21.15.62:0/768744766 shutdown_connections 2024-04-02T14:44:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.605+0000 7f3925479740 1 -- 172.21.15.62:0/768744766 wait complete. 2024-04-02T14:44:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.605+0000 7f391b400640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f39000a3e40 0x7f39000a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f390c0f0650 tx=0x7f390c0f8510 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.605+0000 7f3925479740 1 -- start start 2024-04-02T14:44:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.606+0000 7f3925479740 1 Processor -- start 2024-04-02T14:44:42.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.606+0000 7f3925479740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 0x55c535b54770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.606+0000 7f3925479740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4f200 0x55c535b54cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.606+0000 7f3925479740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b57890 0x55c535b551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.606+0000 7f3925479740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c535aa66a0 con 0x55c535b57890 2024-04-02T14:44:42.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.606+0000 7f3925479740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c535a52980 con 0x55c535b4e520 2024-04-02T14:44:42.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.606+0000 7f3925479740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c535a65cb0 con 0x55c535b4f200 2024-04-02T14:44:42.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.606+0000 7f3925479740 10 client.0 _ll_register_callbacks cb 0x55c535a656a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:42.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.606+0000 7f391be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b57890 0x55c535b551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.606+0000 7f391aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4f200 0x55c535b54cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f391be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b57890 0x55c535b551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55044/0 (socket says 172.21.15.62:55044) 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f391aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4f200 0x55c535b54cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:60338/0 (socket says 172.21.15.62:60338) 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f391be00640 1 -- 172.21.15.62:0/140753590 learned_addr learned my addr 172.21.15.62:0/140753590 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f391be00640 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 0x55c535b54770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f391aa00640 1 -- 172.21.15.62:0/140753590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 msgr2=0x55c535b54770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f391aa00640 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 0x55c535b54770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f391aa00640 1 -- 172.21.15.62:0/140753590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b57890 msgr2=0x55c535b551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f391aa00640 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b57890 0x55c535b551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f391aa00640 1 -- 172.21.15.62:0/140753590 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5358887d0 con 0x55c535b4f200 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f391be00640 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 0x55c535b54770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f391aa00640 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4f200 0x55c535b54cb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f391c06b240 tx=0x7f3908012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=405c319dc80b38a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f3905600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.607+0000 7f3905600640 1 -- 172.21.15.62:0/140753590 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390800f920 con 0x55c535b4f200 2024-04-02T14:44:42.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.608+0000 7f3905600640 1 -- 172.21.15.62:0/140753590 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f390800fac0 con 0x55c535b4f200 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.608+0000 7f3925479740 1 -- 172.21.15.62:0/140753590 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3914005020 con 0x55c535b4f200 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.608+0000 7f3905600640 1 -- 172.21.15.62:0/140753590 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f390800fcc0 con 0x55c535b4f200 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.608+0000 7f3925479740 10 client.25639 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.608+0000 7f3925479740 1 -- 172.21.15.62:0/140753590 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c535b66680 con 0x55c535b4f200 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.608+0000 7f3905600640 1 -- 172.21.15.62:0/140753590 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3914005020 con 0x55c535b4f200 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.608+0000 7f3905600640 1 -- 172.21.15.62:0/140753590 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f390801c420 con 0x55c535b4f200 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.609+0000 7f3925479740 10 client.25639 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.609+0000 7f3925479740 10 client.25639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.609+0000 7f3925479740 20 client.25639 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.609+0000 7f3925479740 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55c535b67be0 0x55c535b6a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.609+0000 7f3925479740 4 client.25639 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.609+0000 7f3925479740 1 -- 172.21.15.62:0/140753590 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c535a6a040 con 0x55c535b67be0 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.609+0000 7f391be00640 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55c535b67be0 0x55c535b6a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.610+0000 7f391be00640 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55c535b67be0 0x55c535b6a020 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb4d45c5af0e5125 server_cookie=4104fb2a57f8616e in_seq=0 out_seq=0 2024-04-02T14:44:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.610+0000 7f3905600640 10 client.25639 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.610+0000 7f3905600640 1 -- 172.21.15.62:0/140753590 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c535a6a040 con 0x55c535b67be0 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.610+0000 7f3905600640 10 client.25639 handle_command_reply: tid=0 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.610+0000 7f3925479740 1 client.25639 shutdown 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3925479740 1 -- 172.21.15.62:0/140753590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4f200 msgr2=0x55c535b54cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3925479740 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4f200 0x55c535b54cb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f391c06b240 tx=0x7f3908012b60 comp rx=0 tx=0).stop 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3925479740 1 -- 172.21.15.62:0/140753590 shutdown_connections 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3925479740 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55c535b67be0 0x55c535b6a020 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3925479740 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b57890 0x55c535b551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3925479740 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4f200 0x55c535b54cb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3925479740 1 --2- 172.21.15.62:0/140753590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 0x55c535b54770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3925479740 1 -- 172.21.15.62:0/140753590 >> 172.21.15.62:0/140753590 conn(0x55c535b0ca50 msgr2=0x55c535b4acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3905600640 0 client.25639 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3925479740 1 -- 172.21.15.62:0/140753590 shutdown_connections 2024-04-02T14:44:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3925479740 1 -- 172.21.15.62:0/140753590 wait complete. 2024-04-02T14:44:42.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.611+0000 7f3925479740 20 client.25639 trim_cache size 0 max 16384 2024-04-02T14:44:42.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:42 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2174624009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:44:42.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.626+0000 7f3925479740 1 Processor -- start 2024-04-02T14:44:42.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.626+0000 7f3925479740 1 -- start start 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f3925479740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 0x55c535b56ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f3925479740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 0x55c535b63330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f3925479740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b57890 0x55c535b3f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f3925479740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1ab610 con 0x55c535b4f200 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f3925479740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1ad150 con 0x55c535b4e520 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f3925479740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1abce0 con 0x55c535b57890 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f391aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 0x55c535b63330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f391be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 0x55c535b56ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f391be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 0x55c535b56ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:52132/0 (socket says 172.21.15.62:52132) 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f391be00640 1 -- 172.21.15.62:0/2455826000 learned_addr learned my addr 172.21.15.62:0/2455826000 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f391aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 0x55c535b63330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55054/0 (socket says 172.21.15.62:55054) 2024-04-02T14:44:42.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f391be00640 1 --2- 172.21.15.62:0/2455826000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b57890 0x55c535b3f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f391be00640 1 -- 172.21.15.62:0/2455826000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b57890 msgr2=0x55c535b3f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f391be00640 1 --2- 172.21.15.62:0/2455826000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b57890 0x55c535b3f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f391be00640 1 -- 172.21.15.62:0/2455826000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 msgr2=0x55c535b63330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f391be00640 1 --2- 172.21.15.62:0/2455826000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 0x55c535b63330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.627+0000 7f391be00640 1 -- 172.21.15.62:0/2455826000 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c535b4d550 con 0x55c535b4e520 2024-04-02T14:44:42.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.628+0000 7f391be00640 1 --2- 172.21.15.62:0/2455826000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 0x55c535b56ea0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3914009a70 tx=0x7f3914005ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ce97d231688188b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.628+0000 7f3907e00640 1 -- 172.21.15.62:0/2455826000 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391402fd50 con 0x55c535b4e520 2024-04-02T14:44:42.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.628+0000 7f3907e00640 1 -- 172.21.15.62:0/2455826000 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f39140049d0 con 0x55c535b4e520 2024-04-02T14:44:42.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.628+0000 7f3907e00640 1 -- 172.21.15.62:0/2455826000 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391402f460 con 0x55c535b4e520 2024-04-02T14:44:42.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.629+0000 7f3925479740 1 -- 172.21.15.62:0/2455826000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 msgr2=0x55c535b56ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.629+0000 7f3925479740 1 --2- 172.21.15.62:0/2455826000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 0x55c535b56ea0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3914009a70 tx=0x7f3914005ac0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.629+0000 7f3925479740 1 -- 172.21.15.62:0/2455826000 shutdown_connections 2024-04-02T14:44:42.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.629+0000 7f3925479740 1 --2- 172.21.15.62:0/2455826000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b57890 0x55c535b3f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.629+0000 7f3925479740 1 --2- 172.21.15.62:0/2455826000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 0x55c535b63330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.629+0000 7f3925479740 1 --2- 172.21.15.62:0/2455826000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b4e520 0x55c535b56ea0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.629+0000 7f3925479740 1 -- 172.21.15.62:0/2455826000 >> 172.21.15.62:0/2455826000 conn(0x55c535b0ca50 msgr2=0x55c535b3f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.629+0000 7f3925479740 1 -- 172.21.15.62:0/2455826000 shutdown_connections 2024-04-02T14:44:42.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.629+0000 7f3925479740 1 -- 172.21.15.62:0/2455826000 wait complete. 2024-04-02T14:44:42.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.629+0000 7f3925479740 1 -- start start 2024-04-02T14:44:42.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.629+0000 7f3925479740 1 Processor -- start 2024-04-02T14:44:42.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f3925479740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4e520 0x55c535b5f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f3925479740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 0x55c535b5fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f3925479740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b57890 0x55c535b509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f3925479740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1aca80 con 0x55c535b4f200 2024-04-02T14:44:42.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f3925479740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1ac3b0 con 0x55c535b57890 2024-04-02T14:44:42.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f3925479740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1a9610 con 0x55c535b4e520 2024-04-02T14:44:42.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f3925479740 10 client.0 _ll_register_callbacks cb 0x7f3908009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:42.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f391aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 0x55c535b5fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f391be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4e520 0x55c535b5f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f391be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4e520 0x55c535b5f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:60352/0 (socket says 172.21.15.62:60352) 2024-04-02T14:44:42.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f391be00640 1 -- 172.21.15.62:0/1017325342 learned_addr learned my addr 172.21.15.62:0/1017325342 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.630+0000 7f391be00640 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b57890 0x55c535b509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f391aa00640 1 -- 172.21.15.62:0/1017325342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4e520 msgr2=0x55c535b5f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f391aa00640 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4e520 0x55c535b5f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f391aa00640 1 -- 172.21.15.62:0/1017325342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b57890 msgr2=0x55c535b509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f391aa00640 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b57890 0x55c535b509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f391aa00640 1 -- 172.21.15.62:0/1017325342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c535b51050 con 0x55c535b4f200 2024-04-02T14:44:42.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f391aa00640 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 0x55c535b5fd80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3908000c00 tx=0x7f390801c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8db039e0236341 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f3905600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:42.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f3905600640 1 -- 172.21.15.62:0/1017325342 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908029030 con 0x55c535b4f200 2024-04-02T14:44:42.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f3905600640 1 -- 172.21.15.62:0/1017325342 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f3908018040 con 0x55c535b4f200 2024-04-02T14:44:42.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f3925479740 1 -- 172.21.15.62:0/1017325342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39140049d0 con 0x55c535b4f200 2024-04-02T14:44:42.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f3905600640 1 -- 172.21.15.62:0/1017325342 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3908033050 con 0x55c535b4f200 2024-04-02T14:44:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f3925479740 10 client.15537 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.631+0000 7f3925479740 1 -- 172.21.15.62:0/1017325342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c535b52330 con 0x55c535b4f200 2024-04-02T14:44:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.632+0000 7f3905600640 1 -- 172.21.15.62:0/1017325342 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f39140049d0 con 0x55c535b4f200 2024-04-02T14:44:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.632+0000 7f3905600640 1 -- 172.21.15.62:0/1017325342 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f39080035f0 con 0x55c535b4f200 2024-04-02T14:44:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.632+0000 7f3925479740 10 client.15537 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.632+0000 7f3925479740 10 client.15537 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:44:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.632+0000 7f3925479740 20 client.15537 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.632+0000 7f3925479740 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55c535bcf280 0x55c535bd16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.632+0000 7f3925479740 4 client.15537 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:44:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.632+0000 7f3925479740 1 -- 172.21.15.62:0/1017325342 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c535ab7c30 con 0x55c535bcf280 2024-04-02T14:44:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.632+0000 7f391be00640 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55c535bcf280 0x55c535bd16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f391be00640 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55c535bcf280 0x55c535bd16a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51b8de313927382 server_cookie=738f8195d6ccf624 in_seq=0 out_seq=0 2024-04-02T14:44:42.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3905600640 10 client.15537 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:44:42.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3905600640 1 -- 172.21.15.62:0/1017325342 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c535ab7c30 con 0x55c535bcf280 2024-04-02T14:44:42.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3905600640 10 client.15537 handle_command_reply: tid=0 2024-04-02T14:44:42.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3925479740 1 client.15537 shutdown 2024-04-02T14:44:42.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3925479740 1 -- 172.21.15.62:0/1017325342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 msgr2=0x55c535b5fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3925479740 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 0x55c535b5fd80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3908000c00 tx=0x7f390801c6d0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3925479740 1 -- 172.21.15.62:0/1017325342 shutdown_connections 2024-04-02T14:44:42.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3925479740 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55c535bcf280 0x55c535bd16a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3925479740 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c535b57890 0x55c535b509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3925479740 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c535b4f200 0x55c535b5fd80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3925479740 1 --2- 172.21.15.62:0/1017325342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c535b4e520 0x55c535b5f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3905600640 0 client.15537 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:44:42.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.633+0000 7f3925479740 1 -- 172.21.15.62:0/1017325342 >> 172.21.15.62:0/1017325342 conn(0x55c535b0ca50 msgr2=0x55c535b54fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3925479740 1 -- 172.21.15.62:0/1017325342 shutdown_connections 2024-04-02T14:44:42.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3925479740 1 -- 172.21.15.62:0/1017325342 wait complete. 2024-04-02T14:44:42.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3925479740 20 client.15537 trim_cache size 0 max 16384 2024-04-02T14:44:42.636 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:44:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3922800640 1 -- 172.21.15.62:0/224964034 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f39000a3e40 msgr2=0x7f39000a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3922800640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f39000a3e40 0x7f39000a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f390c0f0650 tx=0x7f390c0f8510 comp rx=0 tx=0).stop 2024-04-02T14:44:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3922800640 1 -- 172.21.15.62:0/224964034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c103f10 msgr2=0x7f391c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3922800640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c103f10 0x7f391c19a6b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f390c009230 tx=0x7f390c02dc80 comp rx=0 tx=0).stop 2024-04-02T14:44:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3922800640 1 -- 172.21.15.62:0/224964034 shutdown_connections 2024-04-02T14:44:42.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3922800640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f39000a3e40 0x7f39000a6300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3922800640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c19ef70 0x7f391c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3922800640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c108fe0 0x7f391c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3922800640 1 --2- 172.21.15.62:0/224964034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c103f10 0x7f391c19a6b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.634+0000 7f3922800640 1 -- 172.21.15.62:0/224964034 >> 172.21.15.62:0/224964034 conn(0x7f391c0ff770 msgr2=0x7f391c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.635+0000 7f3922800640 1 -- 172.21.15.62:0/224964034 shutdown_connections 2024-04-02T14:44:42.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.635+0000 7f3922800640 1 -- 172.21.15.62:0/224964034 wait complete. 2024-04-02T14:44:42.646 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:42 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2174624009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:44:42.648 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:44:42.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:42 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2174624009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:44:42.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.877+0000 7f7acf200640 1 -- 172.21.15.62:0/4259729081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ac810d3e0 msgr2=0x7f7ac8114fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.877+0000 7f7acf200640 1 --2- 172.21.15.62:0/4259729081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ac810d3e0 0x7f7ac8114fb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009900 tx=0x7f7ac002bc40 comp rx=0 tx=0).stop 2024-04-02T14:44:42.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.877+0000 7f7acf200640 1 -- 172.21.15.62:0/4259729081 shutdown_connections 2024-04-02T14:44:42.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.877+0000 7f7acf200640 1 --2- 172.21.15.62:0/4259729081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ac810d3e0 0x7f7ac8114fb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.877+0000 7f7acf200640 1 --2- 172.21.15.62:0/4259729081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ac8108cc0 0x7f7ac810cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.877+0000 7f7acf200640 1 --2- 172.21.15.62:0/4259729081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ac8108310 0x7f7ac81086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.877+0000 7f7acf200640 1 -- 172.21.15.62:0/4259729081 >> 172.21.15.62:0/4259729081 conn(0x7f7ac8076040 msgr2=0x7f7ac8078460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.877+0000 7f7acf200640 1 -- 172.21.15.62:0/4259729081 shutdown_connections 2024-04-02T14:44:42.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.877+0000 7f7acf200640 1 -- 172.21.15.62:0/4259729081 wait complete. 2024-04-02T14:44:42.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.878+0000 7f7acf200640 1 Processor -- start 2024-04-02T14:44:42.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.878+0000 7f7acf200640 1 -- start start 2024-04-02T14:44:42.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ac8108310 0x7f7ac81a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ac810d3e0 0x7f7ac81a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acf200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ac819e8b0 0x7f7ac819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81185b0 con 0x7f7ac8108310 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acf200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8118430 con 0x7f7ac819e8b0 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8118730 con 0x7f7ac810d3e0 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7ac7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ac8108310 0x7f7ac81a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7ac7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ac8108310 0x7f7ac81a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55076/0 (socket says 172.21.15.62:55076) 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7ac7e00640 1 -- 172.21.15.62:0/432389708 learned_addr learned my addr 172.21.15.62:0/432389708 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acd200640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ac819e8b0 0x7f7ac819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7ac7400640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ac810d3e0 0x7f7ac81a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acd200640 1 -- 172.21.15.62:0/432389708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ac810d3e0 msgr2=0x7f7ac81a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acd200640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ac810d3e0 0x7f7ac81a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acd200640 1 -- 172.21.15.62:0/432389708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ac8108310 msgr2=0x7f7ac81a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acd200640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ac8108310 0x7f7ac81a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7acd200640 1 -- 172.21.15.62:0/432389708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac819f3d0 con 0x7f7ac819e8b0 2024-04-02T14:44:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.879+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ac8108310 0x7f7ac81a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:42.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.880+0000 7f7acd200640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ac819e8b0 0x7f7ac819ed60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009230 tx=0x7f7ac002dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.880+0000 7f7ac4c00640 1 -- 172.21.15.62:0/432389708 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0043020 con 0x7f7ac819e8b0 2024-04-02T14:44:42.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.880+0000 7f7acf200640 1 -- 172.21.15.62:0/432389708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac819f660 con 0x7f7ac819e8b0 2024-04-02T14:44:42.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.880+0000 7f7acf200640 1 -- 172.21.15.62:0/432389708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac81ab900 con 0x7f7ac819e8b0 2024-04-02T14:44:42.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.880+0000 7f7ac4c00640 1 -- 172.21.15.62:0/432389708 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7ac0004250 con 0x7f7ac819e8b0 2024-04-02T14:44:42.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.880+0000 7f7ac4c00640 1 -- 172.21.15.62:0/432389708 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0034740 con 0x7f7ac819e8b0 2024-04-02T14:44:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.884+0000 7f7ac4c00640 1 -- 172.21.15.62:0/432389708 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f7ac0046020 con 0x7f7ac819e8b0 2024-04-02T14:44:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.884+0000 7f7ad1e1e740 1 Processor -- start 2024-04-02T14:44:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.884+0000 7f7ad1e1e740 1 -- start start 2024-04-02T14:44:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.884+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdd9f0 0x5579d9bdc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.884+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 0x5579d9bdd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.884+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9be6080 0x5579d9be8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.884+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5579d9b1d400 con 0x5579d9bdcd10 2024-04-02T14:44:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.884+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5579d9b01f10 con 0x5579d9bdd9f0 2024-04-02T14:44:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.884+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5579d9b416f0 con 0x5579d9be6080 2024-04-02T14:44:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.885+0000 7f7ac7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdd9f0 0x5579d9bdc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.885+0000 7f7ac7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 0x5579d9bdd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.885+0000 7f7ac7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdd9f0 0x5579d9bdc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:52164/0 (socket says 172.21.15.62:52164) 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.885+0000 7f7ac7400640 1 -- 172.21.15.62:0/1875854997 learned_addr learned my addr 172.21.15.62:0/1875854997 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.885+0000 7f7ac7400640 1 --2- 172.21.15.62:0/1875854997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9be6080 0x5579d9be8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.885+0000 7f7ac7e00640 1 -- 172.21.15.62:0/1875854997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9be6080 msgr2=0x5579d9be8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.885+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/1875854997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9be6080 0x5579d9be8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.885+0000 7f7ac7e00640 1 -- 172.21.15.62:0/1875854997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdd9f0 msgr2=0x5579d9bdc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.885+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/1875854997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdd9f0 0x5579d9bdc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.885+0000 7f7ac7e00640 1 -- 172.21.15.62:0/1875854997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579d98553f0 con 0x5579d9bdcd10 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.885+0000 7f7ac4c00640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7aac0a3ec0 0x7f7aac0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.886+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/1875854997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 0x5579d9bdd190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8011840 tx=0x7f7ab8011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34957113ac39298 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.886+0000 7f7ac4c00640 1 -- 172.21.15.62:0/432389708 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f7ac00f20a0 con 0x7f7ac819e8b0 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.886+0000 7f7ac7400640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7aac0a3ec0 0x7f7aac0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.886+0000 7f7abd600640 1 -- 172.21.15.62:0/1875854997 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80112f0 con 0x5579d9bdcd10 2024-04-02T14:44:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.886+0000 7f7abd600640 1 -- 172.21.15.62:0/1875854997 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7ab800f9a0 con 0x5579d9bdcd10 2024-04-02T14:44:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.886+0000 7f7abd600640 1 -- 172.21.15.62:0/1875854997 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8004680 con 0x5579d9bdcd10 2024-04-02T14:44:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.887+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1875854997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 msgr2=0x5579d9bdd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.887+0000 7f7ac7400640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7aac0a3ec0 0x7f7aac0a6380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00042e0 tx=0x7f7ab000b6e0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.887+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/1875854997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 0x5579d9bdd190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8011840 tx=0x7f7ab8011d00 comp rx=0 tx=0).stop 2024-04-02T14:44:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.887+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1875854997 shutdown_connections 2024-04-02T14:44:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.887+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/1875854997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9be6080 0x5579d9be8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.887+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/1875854997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 0x5579d9bdd190 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.887+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/1875854997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdd9f0 0x5579d9bdc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.887+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1875854997 >> 172.21.15.62:0/1875854997 conn(0x5579d9ba3a50 msgr2=0x5579d9bd7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.887+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1875854997 shutdown_connections 2024-04-02T14:44:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.887+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1875854997 wait complete. 2024-04-02T14:44:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.887+0000 7f7ad1e1e740 1 -- start start 2024-04-02T14:44:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.888+0000 7f7ad1e1e740 1 Processor -- start 2024-04-02T14:44:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.888+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 0x5579d9be2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.888+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdd9f0 0x5579d9be3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.888+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9be6080 0x5579d9be3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.888+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5579d9b3d6a0 con 0x5579d9bdcd10 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.888+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5579d9ae9980 con 0x5579d9bdd9f0 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.888+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5579d9afccb0 con 0x5579d9be6080 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.888+0000 7f7ad1e1e740 10 client.0 _ll_register_callbacks cb 0x5579d9afc6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.888+0000 7f7ac7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 0x5579d9be2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.889+0000 7f7ac7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 0x5579d9be2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55096/0 (socket says 172.21.15.62:55096) 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.889+0000 7f7ac7e00640 1 -- 172.21.15.62:0/1762812598 learned_addr learned my addr 172.21.15.62:0/1762812598 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.889+0000 7f7ac7400640 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdd9f0 0x5579d9be3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.889+0000 7f7acd200640 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9be6080 0x5579d9be3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.889+0000 7f7ac7e00640 1 -- 172.21.15.62:0/1762812598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9be6080 msgr2=0x5579d9be3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.889+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9be6080 0x5579d9be3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.889+0000 7f7ac7e00640 1 -- 172.21.15.62:0/1762812598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdd9f0 msgr2=0x5579d9be3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.889+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdd9f0 0x5579d9be3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.889+0000 7f7ac7e00640 1 -- 172.21.15.62:0/1762812598 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579d991f7d0 con 0x5579d9bdcd10 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.890+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 0x5579d9be2ed0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8000f80 tx=0x7f7ab8011320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb2b3f55bc27de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.890+0000 7f7ab6000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.890+0000 7f7ab6000640 1 -- 172.21.15.62:0/1762812598 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab802a060 con 0x5579d9bdcd10 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.890+0000 7f7ab6000640 1 -- 172.21.15.62:0/1762812598 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f7ab80043c0 con 0x5579d9bdcd10 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.890+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1762812598 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab800f9a0 con 0x5579d9bdcd10 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.890+0000 7f7ab6000640 1 -- 172.21.15.62:0/1762812598 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ab8004e50 con 0x5579d9bdcd10 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.890+0000 7f7ad1e1e740 10 client.15549 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.890+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1762812598 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579d9bf4e10 con 0x5579d9bdcd10 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.891+0000 7f7ab6000640 1 -- 172.21.15.62:0/1762812598 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7ab800f9a0 con 0x5579d9bdcd10 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.891+0000 7f7ab6000640 1 -- 172.21.15.62:0/1762812598 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7ab8018020 con 0x5579d9bdcd10 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.891+0000 7f7ad1e1e740 10 client.15549 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.891+0000 7f7ad1e1e740 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.891+0000 7f7ad1e1e740 20 client.15549 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.891+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5579d9bf6160 0x5579d9bf8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.891+0000 7f7ad1e1e740 4 client.15549 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.891+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1762812598 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579d9b01040 con 0x5579d9bf6160 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.892+0000 7f7ac7400640 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5579d9bf6160 0x5579d9bf8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.892+0000 7f7ac7400640 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5579d9bf6160 0x5579d9bf8570 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=206da072f8cb9d28 server_cookie=d2af2bf88a2647ee in_seq=0 out_seq=0 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.892+0000 7f7ab6000640 10 client.15549 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ab6000640 1 -- 172.21.15.62:0/1762812598 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579d9b01040 con 0x5579d9bf6160 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ab6000640 10 client.15549 handle_command_reply: tid=0 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ad1e1e740 1 client.15549 shutdown 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1762812598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 msgr2=0x5579d9be2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 0x5579d9be2ed0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8000f80 tx=0x7f7ab8011320 comp rx=0 tx=0).stop 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1762812598 shutdown_connections 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5579d9bf6160 0x5579d9bf8570 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9be6080 0x5579d9be3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdd9f0 0x5579d9be3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/1762812598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdcd10 0x5579d9be2ed0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ab6000640 0 client.15549 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1762812598 >> 172.21.15.62:0/1762812598 conn(0x5579d9ba3a50 msgr2=0x5579d9bd94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1762812598 shutdown_connections 2024-04-02T14:44:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.893+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/1762812598 wait complete. 2024-04-02T14:44:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.894+0000 7f7ad1e1e740 20 client.15549 trim_cache size 0 max 16384 2024-04-02T14:44:42.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.909+0000 7f7ad1e1e740 1 Processor -- start 2024-04-02T14:44:42.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.909+0000 7f7ad1e1e740 1 -- start start 2024-04-02T14:44:42.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.909+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdcd10 0x5579d9be5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.909+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdd9f0 0x5579d9bf1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.909+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9be6080 0x5579d9bd92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.909+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa400e650 con 0x5579d9be6080 2024-04-02T14:44:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.909+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa4010190 con 0x5579d9bdcd10 2024-04-02T14:44:42.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.909+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa400ed20 con 0x5579d9bdd9f0 2024-04-02T14:44:42.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdcd10 0x5579d9be5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7acd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9be6080 0x5579d9bd92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdd9f0 0x5579d9bf1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdd9f0 0x5579d9bf1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:60412/0 (socket says 172.21.15.62:60412) 2024-04-02T14:44:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7400640 1 -- 172.21.15.62:0/4123856366 learned_addr learned my addr 172.21.15.62:0/4123856366 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdcd10 0x5579d9be5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:52184/0 (socket says 172.21.15.62:52184) 2024-04-02T14:44:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7e00640 1 -- 172.21.15.62:0/4123856366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdd9f0 msgr2=0x5579d9bf1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/4123856366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdd9f0 0x5579d9bf1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7e00640 1 -- 172.21.15.62:0/4123856366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9be6080 msgr2=0x5579d9bd92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/4123856366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9be6080 0x5579d9bd92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7e00640 1 -- 172.21.15.62:0/4123856366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579d9bdbd40 con 0x5579d9bdcd10 2024-04-02T14:44:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7400640 1 --2- 172.21.15.62:0/4123856366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdd9f0 0x5579d9bf1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7acd200640 1 --2- 172.21.15.62:0/4123856366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9be6080 0x5579d9bd92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.910+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/4123856366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdcd10 0x5579d9be5600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8011d40 tx=0x7f7ab8005560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7abf00a0388a160c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7abcc00640 1 -- 172.21.15.62:0/4123856366 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8011390 con 0x5579d9bdcd10 2024-04-02T14:44:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7abcc00640 1 -- 172.21.15.62:0/4123856366 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7ab800f8c0 con 0x5579d9bdcd10 2024-04-02T14:44:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7abcc00640 1 -- 172.21.15.62:0/4123856366 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8003bd0 con 0x5579d9bdcd10 2024-04-02T14:44:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/4123856366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdcd10 msgr2=0x5579d9be5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/4123856366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdcd10 0x5579d9be5600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8011d40 tx=0x7f7ab8005560 comp rx=0 tx=0).stop 2024-04-02T14:44:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/4123856366 shutdown_connections 2024-04-02T14:44:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/4123856366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9be6080 0x5579d9bd92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/4123856366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdd9f0 0x5579d9bf1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/4123856366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9bdcd10 0x5579d9be5600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/4123856366 >> 172.21.15.62:0/4123856366 conn(0x5579d9ba3a50 msgr2=0x5579d9bd8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/4123856366 shutdown_connections 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.911+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/4123856366 wait complete. 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.912+0000 7f7ad1e1e740 1 -- start start 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.912+0000 7f7ad1e1e740 1 Processor -- start 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdcd10 0x5579d9bee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdd9f0 0x5579d9be0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ad1e1e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9be6080 0x5579d9be0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa400fac0 con 0x5579d9bdd9f0 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa400f3f0 con 0x5579d9be6080 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ad1e1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa400c630 con 0x5579d9bdcd10 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ad1e1e740 10 client.0 _ll_register_callbacks cb 0x7f7ab000bd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ac7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdd9f0 0x5579d9be0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ac7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdd9f0 0x5579d9be0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55114/0 (socket says 172.21.15.62:55114) 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ac7400640 1 -- 172.21.15.62:0/2255298069 learned_addr learned my addr 172.21.15.62:0/2255298069 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7acd200640 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9be6080 0x5579d9be0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdcd10 0x5579d9bee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ac7400640 1 -- 172.21.15.62:0/2255298069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdcd10 msgr2=0x5579d9bee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ac7400640 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdcd10 0x5579d9bee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ac7400640 1 -- 172.21.15.62:0/2255298069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9be6080 msgr2=0x5579d9be0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ac7400640 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9be6080 0x5579d9be0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.913+0000 7f7ac7400640 1 -- 172.21.15.62:0/2255298069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579d9bd7930 con 0x5579d9bdd9f0 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.914+0000 7f7ac7400640 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdd9f0 0x5579d9be0150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000b620 tx=0x7f7ab0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a38897de2b48b27 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.914+0000 7f7ab6000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.914+0000 7f7ab6000640 1 -- 172.21.15.62:0/2255298069 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0031030 con 0x5579d9bdd9f0 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.914+0000 7f7ab6000640 1 -- 172.21.15.62:0/2255298069 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f7ab001b030 con 0x5579d9bdd9f0 2024-04-02T14:44:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.914+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/2255298069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab800f8c0 con 0x5579d9bdd9f0 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.914+0000 7f7ab6000640 1 -- 172.21.15.62:0/2255298069 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ab003b050 con 0x5579d9bdd9f0 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.914+0000 7f7ad1e1e740 10 client.15555 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.914+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/2255298069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579d9be7000 con 0x5579d9bdd9f0 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.914+0000 7f7ab6000640 1 -- 172.21.15.62:0/2255298069 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7ab800f8c0 con 0x5579d9bdd9f0 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.915+0000 7f7ab6000640 1 -- 172.21.15.62:0/2255298069 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7ab00047a0 con 0x5579d9bdd9f0 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.915+0000 7f7ad1e1e740 10 client.15555 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.915+0000 7f7ad1e1e740 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.915+0000 7f7ad1e1e740 20 client.15555 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.915+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5579d9c5c990 0x5579d9c5edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.915+0000 7f7ad1e1e740 4 client.15555 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.915+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/2255298069 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579d9b4ec30 con 0x5579d9c5c990 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.915+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5579d9c5c990 0x5579d9c5edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ac7e00640 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5579d9c5c990 0x5579d9c5edb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e75a83b4319d711 server_cookie=2972cfd13b88b33a in_seq=0 out_seq=0 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ab6000640 10 client.15555 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ab6000640 1 -- 172.21.15.62:0/2255298069 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579d9b4ec30 con 0x5579d9c5c990 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ab6000640 10 client.15555 handle_command_reply: tid=0 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 1 client.15555 shutdown 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/2255298069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdd9f0 msgr2=0x5579d9be0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdd9f0 0x5579d9be0150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000b620 tx=0x7f7ab0004730 comp rx=0 tx=0).stop 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/2255298069 shutdown_connections 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5579d9c5c990 0x5579d9c5edb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5579d9be6080 0x5579d9be0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579d9bdd9f0 0x5579d9be0150 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ab6000640 0 client.15555 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 1 --2- 172.21.15.62:0/2255298069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d9bdcd10 0x5579d9bee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/2255298069 >> 172.21.15.62:0/2255298069 conn(0x5579d9ba3a50 msgr2=0x5579d9bd93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/2255298069 shutdown_connections 2024-04-02T14:44:42.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 1 -- 172.21.15.62:0/2255298069 wait complete. 2024-04-02T14:44:42.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.916+0000 7f7ad1e1e740 20 client.15555 trim_cache size 0 max 16384 2024-04-02T14:44:42.922 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:44:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 -- 172.21.15.62:0/432389708 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7aac0a3ec0 msgr2=0x7f7aac0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7aac0a3ec0 0x7f7aac0a6380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00042e0 tx=0x7f7ab000b6e0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 -- 172.21.15.62:0/432389708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ac819e8b0 msgr2=0x7f7ac819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ac819e8b0 0x7f7ac819ed60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009230 tx=0x7f7ac002dd10 comp rx=0 tx=0).stop 2024-04-02T14:44:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 -- 172.21.15.62:0/432389708 shutdown_connections 2024-04-02T14:44:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7aac0a3ec0 0x7f7aac0a6380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ac819e8b0 0x7f7ac819ed60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ac810d3e0 0x7f7ac81a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 --2- 172.21.15.62:0/432389708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ac8108310 0x7f7ac81a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:42.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 -- 172.21.15.62:0/432389708 >> 172.21.15.62:0/432389708 conn(0x7f7ac8076040 msgr2=0x7f7ac8076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:42.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 -- 172.21.15.62:0/432389708 shutdown_connections 2024-04-02T14:44:42.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:42.917+0000 7f7acf200640 1 -- 172.21.15.62:0/432389708 wait complete. 2024-04-02T14:44:43.617 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:43 smithi113 ceph-mon[36135]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-02T14:44:43.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:43 smithi204 ceph-mon[35948]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-02T14:44:43.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:43 smithi062 ceph-mon[32116]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-02T14:44:43.933 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:44:43.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:44:43.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:44:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.180+0000 7fbb5d200640 1 -- 172.21.15.62:0/2864261941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb58103f10 msgr2=0x7fbb581042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.180+0000 7fbb5d200640 1 --2- 172.21.15.62:0/2864261941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb58103f10 0x7fbb581042f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009900 tx=0x7fbb4802bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.180+0000 7fbb5d200640 1 -- 172.21.15.62:0/2864261941 shutdown_connections 2024-04-02T14:44:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.180+0000 7fbb5d200640 1 --2- 172.21.15.62:0/2864261941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb58108fe0 0x7fbb58110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.180+0000 7fbb5d200640 1 --2- 172.21.15.62:0/2864261941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb581048c0 0x7fbb58108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.180+0000 7fbb5d200640 1 --2- 172.21.15.62:0/2864261941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb58103f10 0x7fbb581042f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.180+0000 7fbb5d200640 1 -- 172.21.15.62:0/2864261941 >> 172.21.15.62:0/2864261941 conn(0x7fbb580ff770 msgr2=0x7fbb58101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.180+0000 7fbb5d200640 1 -- 172.21.15.62:0/2864261941 shutdown_connections 2024-04-02T14:44:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.180+0000 7fbb5d200640 1 -- 172.21.15.62:0/2864261941 wait complete. 2024-04-02T14:44:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.182+0000 7fbb5d200640 1 Processor -- start 2024-04-02T14:44:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.182+0000 7fbb5d200640 1 -- start start 2024-04-02T14:44:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.182+0000 7fbb5d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb58103f10 0x7fbb5819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.182+0000 7fbb5d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb58108fe0 0x7fbb5819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.182+0000 7fbb5d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb5819ef80 0x7fbb5819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.182+0000 7fbb5d200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb58114180 con 0x7fbb58103f10 2024-04-02T14:44:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.182+0000 7fbb5d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbb58114000 con 0x7fbb58108fe0 2024-04-02T14:44:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.182+0000 7fbb5d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb58114300 con 0x7fbb5819ef80 2024-04-02T14:44:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.183+0000 7fbb55e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb58103f10 0x7fbb5819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.183+0000 7fbb55400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb58108fe0 0x7fbb5819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.183+0000 7fbb55400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb58108fe0 0x7fbb5819ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:52206/0 (socket says 172.21.15.62:52206) 2024-04-02T14:44:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.183+0000 7fbb55400640 1 -- 172.21.15.62:0/958199515 learned_addr learned my addr 172.21.15.62:0/958199515 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.183+0000 7fbb55e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb58103f10 0x7fbb5819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55128/0 (socket says 172.21.15.62:55128) 2024-04-02T14:44:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.183+0000 7fbb56800640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb5819ef80 0x7fbb5819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.183+0000 7fbb55400640 1 -- 172.21.15.62:0/958199515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb5819ef80 msgr2=0x7fbb5819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.183+0000 7fbb55400640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb5819ef80 0x7fbb5819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.183+0000 7fbb55400640 1 -- 172.21.15.62:0/958199515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb58103f10 msgr2=0x7fbb5819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.183+0000 7fbb55400640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb58103f10 0x7fbb5819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.183+0000 7fbb55400640 1 -- 172.21.15.62:0/958199515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5819fa70 con 0x7fbb58108fe0 2024-04-02T14:44:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.184+0000 7fbb55400640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb58108fe0 0x7fbb5819ac00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e9a0 tx=0x7fbb4400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.184+0000 7fbb4ea00640 1 -- 172.21.15.62:0/958199515 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400cd80 con 0x7fbb58108fe0 2024-04-02T14:44:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.184+0000 7fbb4ea00640 1 -- 172.21.15.62:0/958199515 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb440035f0 con 0x7fbb58108fe0 2024-04-02T14:44:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.184+0000 7fbb4ea00640 1 -- 172.21.15.62:0/958199515 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44010700 con 0x7fbb58108fe0 2024-04-02T14:44:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.184+0000 7fbb5d200640 1 -- 172.21.15.62:0/958199515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5819fd60 con 0x7fbb58108fe0 2024-04-02T14:44:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.184+0000 7fbb5d200640 1 -- 172.21.15.62:0/958199515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb581a75a0 con 0x7fbb58108fe0 2024-04-02T14:44:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.187+0000 7fbb6002a740 1 Processor -- start 2024-04-02T14:44:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.187+0000 7fbb4ea00640 1 -- 172.21.15.62:0/958199515 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fbb440037a0 con 0x7fbb58108fe0 2024-04-02T14:44:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.187+0000 7fbb6002a740 1 -- start start 2024-04-02T14:44:44.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.188+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 0x55ca3b572d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.188+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b5737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.188+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b57c6c0 0x55ca3b57eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.188+0000 7fbb6002a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ca3b4d7310 con 0x55ca3b57c6c0 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.188+0000 7fbb6002a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ca3b47f690 con 0x55ca3b573350 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.188+0000 7fbb6002a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca3b4d32c0 con 0x55ca3b574030 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.188+0000 7fbb56800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 0x55ca3b572d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.188+0000 7fbb56800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 0x55ca3b572d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:60442/0 (socket says 172.21.15.62:60442) 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.188+0000 7fbb56800640 1 -- 172.21.15.62:0/936840331 learned_addr learned my addr 172.21.15.62:0/936840331 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.188+0000 7fbb55400640 1 --2- 172.21.15.62:0/936840331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b57c6c0 0x55ca3b57eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.188+0000 7fbb55e00640 1 --2- 172.21.15.62:0/936840331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b5737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.189+0000 7fbb56800640 1 -- 172.21.15.62:0/936840331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 msgr2=0x55ca3b5737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.189+0000 7fbb56800640 1 --2- 172.21.15.62:0/936840331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b5737d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.189+0000 7fbb56800640 1 -- 172.21.15.62:0/936840331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b57c6c0 msgr2=0x55ca3b57eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.189+0000 7fbb56800640 1 --2- 172.21.15.62:0/936840331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b57c6c0 0x55ca3b57eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.189+0000 7fbb56800640 1 -- 172.21.15.62:0/936840331 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca3b2b37e0 con 0x55ca3b574030 2024-04-02T14:44:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.189+0000 7fbb56800640 1 --2- 172.21.15.62:0/936840331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 0x55ca3b572d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009f50 tx=0x7fbb500118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7218e763b9cecdb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.189+0000 7fbb4ea00640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb3c0a3ec0 0x7fbb3c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.189+0000 7fbb43400640 1 -- 172.21.15.62:0/936840331 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50004d20 con 0x55ca3b574030 2024-04-02T14:44:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.189+0000 7fbb43400640 1 -- 172.21.15.62:0/936840331 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb50004050 con 0x55ca3b574030 2024-04-02T14:44:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.189+0000 7fbb43400640 1 -- 172.21.15.62:0/936840331 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50005220 con 0x55ca3b574030 2024-04-02T14:44:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.190+0000 7fbb4ea00640 1 -- 172.21.15.62:0/958199515 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fbb440d0d20 con 0x7fbb58108fe0 2024-04-02T14:44:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.190+0000 7fbb55e00640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb3c0a3ec0 0x7fbb3c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.190+0000 7fbb6002a740 1 -- 172.21.15.62:0/936840331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 msgr2=0x55ca3b572d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.190+0000 7fbb6002a740 1 --2- 172.21.15.62:0/936840331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 0x55ca3b572d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009f50 tx=0x7fbb500118d0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.190+0000 7fbb6002a740 1 -- 172.21.15.62:0/936840331 shutdown_connections 2024-04-02T14:44:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.190+0000 7fbb6002a740 1 --2- 172.21.15.62:0/936840331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b57c6c0 0x55ca3b57eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.190+0000 7fbb6002a740 1 --2- 172.21.15.62:0/936840331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b5737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.190+0000 7fbb6002a740 1 --2- 172.21.15.62:0/936840331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 0x55ca3b572d80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.190+0000 7fbb6002a740 1 -- 172.21.15.62:0/936840331 >> 172.21.15.62:0/936840331 conn(0x55ca3b507270 msgr2=0x55ca3b56dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.190+0000 7fbb6002a740 1 -- 172.21.15.62:0/936840331 shutdown_connections 2024-04-02T14:44:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.190+0000 7fbb6002a740 1 -- 172.21.15.62:0/936840331 wait complete. 2024-04-02T14:44:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.191+0000 7fbb55e00640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb3c0a3ec0 0x7fbb3c0a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb48006010 tx=0x7fbb48005e40 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.191+0000 7fbb6002a740 1 -- start start 2024-04-02T14:44:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.191+0000 7fbb6002a740 1 Processor -- start 2024-04-02T14:44:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b579590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b579ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b57c6c0 0x55ca3b57a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb6002a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ca3b57f530 con 0x55ca3b574030 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb6002a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ca3b492a60 con 0x55ca3b573350 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb6002a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca3b57f6b0 con 0x55ca3b57c6c0 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb6002a740 10 client.0 _ll_register_callbacks cb 0x55ca3b498030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb55400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b579ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb55400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b579ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55148/0 (socket says 172.21.15.62:55148) 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb55400640 1 -- 172.21.15.62:0/751852658 learned_addr learned my addr 172.21.15.62:0/751852658 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb56800640 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b57c6c0 0x55ca3b57a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb55e00640 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b579590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb55400640 1 -- 172.21.15.62:0/751852658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b57c6c0 msgr2=0x55ca3b57a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb55400640 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b57c6c0 0x55ca3b57a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb55400640 1 -- 172.21.15.62:0/751852658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 msgr2=0x55ca3b579590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb55400640 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b579590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.192+0000 7fbb55400640 1 -- 172.21.15.62:0/751852658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca3b313f70 con 0x55ca3b574030 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.193+0000 7fbb55400640 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b579ad0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e970 tx=0x7fbb440d43c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff714c94d11e06f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.193+0000 7fbb37e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.193+0000 7fbb37e00640 1 -- 172.21.15.62:0/751852658 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4409c5a0 con 0x55ca3b574030 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.193+0000 7fbb37e00640 1 -- 172.21.15.62:0/751852658 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fbb440cebb0 con 0x55ca3b574030 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.193+0000 7fbb6002a740 1 -- 172.21.15.62:0/751852658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb50004050 con 0x55ca3b574030 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.193+0000 7fbb37e00640 1 -- 172.21.15.62:0/751852658 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb4409c3c0 con 0x55ca3b574030 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.193+0000 7fbb6002a740 10 client.15567 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.193+0000 7fbb6002a740 1 -- 172.21.15.62:0/751852658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca3b1ab170 con 0x55ca3b574030 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.194+0000 7fbb37e00640 1 -- 172.21.15.62:0/751852658 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb50004050 con 0x55ca3b574030 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.194+0000 7fbb37e00640 1 -- 172.21.15.62:0/751852658 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbb440ce5c0 con 0x55ca3b574030 2024-04-02T14:44:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.194+0000 7fbb6002a740 10 client.15567 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.194+0000 7fbb6002a740 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.194+0000 7fbb6002a740 20 client.15567 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.194+0000 7fbb6002a740 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ca3b58cb40 0x55ca3b58ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.194+0000 7fbb6002a740 4 client.15567 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.194+0000 7fbb6002a740 1 -- 172.21.15.62:0/751852658 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca3b4e4850 con 0x55ca3b58cb40 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.194+0000 7fbb55e00640 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ca3b58cb40 0x55ca3b58ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.195+0000 7fbb55e00640 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ca3b58cb40 0x55ca3b58ef50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ef99fb4b0f8fd8 server_cookie=f462d6f5b895d1ed in_seq=0 out_seq=0 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.195+0000 7fbb37e00640 10 client.15567 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb37e00640 1 -- 172.21.15.62:0/751852658 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca3b4e4850 con 0x55ca3b58cb40 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb37e00640 10 client.15567 handle_command_reply: tid=0 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb6002a740 1 client.15567 shutdown 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb6002a740 1 -- 172.21.15.62:0/751852658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 msgr2=0x55ca3b579ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb6002a740 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b579ad0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e970 tx=0x7fbb440d43c0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb6002a740 1 -- 172.21.15.62:0/751852658 shutdown_connections 2024-04-02T14:44:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb6002a740 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ca3b58cb40 0x55ca3b58ef50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb6002a740 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b57c6c0 0x55ca3b57a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb6002a740 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b579ad0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb37e00640 0 client.15567 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb6002a740 1 --2- 172.21.15.62:0/751852658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b579590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb6002a740 1 -- 172.21.15.62:0/751852658 >> 172.21.15.62:0/751852658 conn(0x55ca3b507270 msgr2=0x55ca3b56faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb6002a740 1 -- 172.21.15.62:0/751852658 shutdown_connections 2024-04-02T14:44:44.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.196+0000 7fbb6002a740 1 -- 172.21.15.62:0/751852658 wait complete. 2024-04-02T14:44:44.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.197+0000 7fbb6002a740 20 client.15567 trim_cache size 0 max 16384 2024-04-02T14:44:44.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.216+0000 7fbb6002a740 1 Processor -- start 2024-04-02T14:44:44.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.216+0000 7fbb6002a740 1 -- start start 2024-04-02T14:44:44.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.216+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b573350 0x55ca3b588490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.216+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 0x55ca3b56f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.216+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b57c6c0 0x55ca3b594860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.216+0000 7fbb6002a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581af970 con 0x55ca3b573350 2024-04-02T14:44:44.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.216+0000 7fbb6002a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581b14b0 con 0x55ca3b57c6c0 2024-04-02T14:44:44.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.216+0000 7fbb6002a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581b0040 con 0x55ca3b574030 2024-04-02T14:44:44.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.217+0000 7fbb55e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b573350 0x55ca3b588490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.217+0000 7fbb55e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b573350 0x55ca3b588490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55156/0 (socket says 172.21.15.62:55156) 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.217+0000 7fbb55e00640 1 -- 172.21.15.62:0/1341030875 learned_addr learned my addr 172.21.15.62:0/1341030875 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.217+0000 7fbb56800640 1 --2- 172.21.15.62:0/1341030875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b57c6c0 0x55ca3b594860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.217+0000 7fbb55400640 1 --2- 172.21.15.62:0/1341030875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 0x55ca3b56f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.217+0000 7fbb55e00640 1 -- 172.21.15.62:0/1341030875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 msgr2=0x55ca3b56f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.217+0000 7fbb55e00640 1 --2- 172.21.15.62:0/1341030875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 0x55ca3b56f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.217+0000 7fbb55e00640 1 -- 172.21.15.62:0/1341030875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b57c6c0 msgr2=0x55ca3b594860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.217+0000 7fbb55e00640 1 --2- 172.21.15.62:0/1341030875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b57c6c0 0x55ca3b594860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.217+0000 7fbb55e00640 1 -- 172.21.15.62:0/1341030875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca3b572380 con 0x55ca3b573350 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.217+0000 7fbb55e00640 1 --2- 172.21.15.62:0/1341030875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b573350 0x55ca3b588490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009900 tx=0x7fbb48009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b433f9fd2ff5c1d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.218+0000 7fbb42a00640 1 -- 172.21.15.62:0/1341030875 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4805a070 con 0x55ca3b573350 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.218+0000 7fbb42a00640 1 -- 172.21.15.62:0/1341030875 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb480046c0 con 0x55ca3b573350 2024-04-02T14:44:44.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.218+0000 7fbb42a00640 1 -- 172.21.15.62:0/1341030875 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48003850 con 0x55ca3b573350 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.218+0000 7fbb6002a740 1 -- 172.21.15.62:0/1341030875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b573350 msgr2=0x55ca3b588490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.218+0000 7fbb6002a740 1 --2- 172.21.15.62:0/1341030875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b573350 0x55ca3b588490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009900 tx=0x7fbb48009260 comp rx=0 tx=0).stop 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.219+0000 7fbb6002a740 1 -- 172.21.15.62:0/1341030875 shutdown_connections 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.219+0000 7fbb6002a740 1 --2- 172.21.15.62:0/1341030875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b57c6c0 0x55ca3b594860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.219+0000 7fbb6002a740 1 --2- 172.21.15.62:0/1341030875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b574030 0x55ca3b56f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.219+0000 7fbb6002a740 1 --2- 172.21.15.62:0/1341030875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b573350 0x55ca3b588490 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.219+0000 7fbb6002a740 1 -- 172.21.15.62:0/1341030875 >> 172.21.15.62:0/1341030875 conn(0x55ca3b507270 msgr2=0x55ca3b56f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.219+0000 7fbb6002a740 1 -- 172.21.15.62:0/1341030875 shutdown_connections 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.219+0000 7fbb6002a740 1 -- 172.21.15.62:0/1341030875 wait complete. 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.219+0000 7fbb6002a740 1 -- start start 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.219+0000 7fbb6002a740 1 Processor -- start 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b584f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b581970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb6002a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b57c6c0 0x55ca3b582060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb6002a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581b0de0 con 0x55ca3b574030 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb6002a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581b0710 con 0x55ca3b573350 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb6002a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581ad970 con 0x55ca3b57c6c0 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb6002a740 10 client.0 _ll_register_callbacks cb 0x7fbb4400ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb55400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b581970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb55400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b581970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55168/0 (socket says 172.21.15.62:55168) 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb55400640 1 -- 172.21.15.62:0/3387471694 learned_addr learned my addr 172.21.15.62:0/3387471694 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb55e00640 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b584f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb56800640 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b57c6c0 0x55ca3b582060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb55400640 1 -- 172.21.15.62:0/3387471694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b57c6c0 msgr2=0x55ca3b582060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb55400640 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b57c6c0 0x55ca3b582060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb55400640 1 -- 172.21.15.62:0/3387471694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 msgr2=0x55ca3b584f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb55400640 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b584f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.220+0000 7fbb55400640 1 -- 172.21.15.62:0/3387471694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca3b5f30e0 con 0x55ca3b574030 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.221+0000 7fbb55400640 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b581970 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbb44098ec0 tx=0x7fbb44003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232a3d26c78107ff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.221+0000 7fbb37e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.221+0000 7fbb37e00640 1 -- 172.21.15.62:0/3387471694 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb440d3050 con 0x55ca3b574030 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.221+0000 7fbb37e00640 1 -- 172.21.15.62:0/3387471694 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fbb4409bfa0 con 0x55ca3b574030 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.221+0000 7fbb6002a740 1 -- 172.21.15.62:0/3387471694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb480046c0 con 0x55ca3b574030 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.221+0000 7fbb37e00640 1 -- 172.21.15.62:0/3387471694 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb44098d00 con 0x55ca3b574030 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.221+0000 7fbb6002a740 10 client.15573 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.221+0000 7fbb6002a740 1 -- 172.21.15.62:0/3387471694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca3b5f4270 con 0x55ca3b574030 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.221+0000 7fbb37e00640 1 -- 172.21.15.62:0/3387471694 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb480046c0 con 0x55ca3b574030 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.222+0000 7fbb37e00640 1 -- 172.21.15.62:0/3387471694 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbb4409b920 con 0x55ca3b574030 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.222+0000 7fbb6002a740 10 client.15573 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:44.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.222+0000 7fbb6002a740 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:44.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.222+0000 7fbb6002a740 20 client.15573 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:44.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.222+0000 7fbb6002a740 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ca3b5f4a20 0x55ca3b5f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.222+0000 7fbb6002a740 4 client.15573 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:44:44.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.222+0000 7fbb6002a740 1 -- 172.21.15.62:0/3387471694 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ca3b4deba0 con 0x55ca3b5f4a20 2024-04-02T14:44:44.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.222+0000 7fbb55e00640 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ca3b5f4a20 0x55ca3b5f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.222+0000 7fbb55e00640 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ca3b5f4a20 0x55ca3b5f6e40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38fe9d4cc4189d05 server_cookie=6aa85bfafbbf058 in_seq=0 out_seq=0 2024-04-02T14:44:44.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.222+0000 7fbb37e00640 10 client.15573 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:44.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb37e00640 1 -- 172.21.15.62:0/3387471694 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ca3b4deba0 con 0x55ca3b5f4a20 2024-04-02T14:44:44.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb37e00640 10 client.15573 handle_command_reply: tid=0 2024-04-02T14:44:44.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 1 client.15573 shutdown 2024-04-02T14:44:44.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 1 -- 172.21.15.62:0/3387471694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 msgr2=0x55ca3b581970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b581970 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbb44098ec0 tx=0x7fbb44003ea0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 1 -- 172.21.15.62:0/3387471694 shutdown_connections 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ca3b5f4a20 0x55ca3b5f6e40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca3b57c6c0 0x55ca3b582060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca3b574030 0x55ca3b581970 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 1 --2- 172.21.15.62:0/3387471694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ca3b573350 0x55ca3b584f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb37e00640 0 client.15573 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 1 -- 172.21.15.62:0/3387471694 >> 172.21.15.62:0/3387471694 conn(0x55ca3b507270 msgr2=0x55ca3b579de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 1 -- 172.21.15.62:0/3387471694 shutdown_connections 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 1 -- 172.21.15.62:0/3387471694 wait complete. 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.224+0000 7fbb6002a740 20 client.15573 trim_cache size 0 max 16384 2024-04-02T14:44:44.225 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:44:44.226 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-02T14:44:44.226 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "2fb16472-81f2-4b0a-8ff0-4636c85f9286", 2024-04-02T14:44:44.226 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-02T14:44:44.226 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 -- 172.21.15.62:0/958199515 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb3c0a3ec0 msgr2=0x7fbb3c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb3c0a3ec0 0x7fbb3c0a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb48006010 tx=0x7fbb48005e40 comp rx=0 tx=0).stop 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 -- 172.21.15.62:0/958199515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb58108fe0 msgr2=0x7fbb5819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb58108fe0 0x7fbb5819ac00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e9a0 tx=0x7fbb4400ee60 comp rx=0 tx=0).stop 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 -- 172.21.15.62:0/958199515 shutdown_connections 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb3c0a3ec0 0x7fbb3c0a6380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb5819ef80 0x7fbb5819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb58108fe0 0x7fbb5819ac00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 --2- 172.21.15.62:0/958199515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb58103f10 0x7fbb5819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 -- 172.21.15.62:0/958199515 >> 172.21.15.62:0/958199515 conn(0x7fbb580ff770 msgr2=0x7fbb58100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 -- 172.21.15.62:0/958199515 shutdown_connections 2024-04-02T14:44:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.225+0000 7fbb5d200640 1 -- 172.21.15.62:0/958199515 wait complete. 2024-04-02T14:44:44.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:44:44.239 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:44:44.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.460+0000 7fbe64a00640 1 -- 172.21.15.62:0/1385827222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe601047a0 msgr2=0x7fbe601087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.460+0000 7fbe64a00640 1 --2- 172.21.15.62:0/1385827222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe601047a0 0x7fbe601087e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009900 tx=0x7fbe4c02bc40 comp rx=0 tx=0).stop 2024-04-02T14:44:44.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.460+0000 7fbe64a00640 1 -- 172.21.15.62:0/1385827222 shutdown_connections 2024-04-02T14:44:44.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.460+0000 7fbe64a00640 1 --2- 172.21.15.62:0/1385827222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe60108ec0 0x7fbe60110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.460+0000 7fbe64a00640 1 --2- 172.21.15.62:0/1385827222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe601047a0 0x7fbe601087e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.460+0000 7fbe64a00640 1 --2- 172.21.15.62:0/1385827222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe60103df0 0x7fbe601041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.460+0000 7fbe64a00640 1 -- 172.21.15.62:0/1385827222 >> 172.21.15.62:0/1385827222 conn(0x7fbe600ff630 msgr2=0x7fbe60101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.460+0000 7fbe64a00640 1 -- 172.21.15.62:0/1385827222 shutdown_connections 2024-04-02T14:44:44.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.460+0000 7fbe64a00640 1 -- 172.21.15.62:0/1385827222 wait complete. 2024-04-02T14:44:44.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.462+0000 7fbe64a00640 1 Processor -- start 2024-04-02T14:44:44.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.462+0000 7fbe64a00640 1 -- start start 2024-04-02T14:44:44.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.462+0000 7fbe64a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe60103df0 0x7fbe6019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.462+0000 7fbe64a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe60108ec0 0x7fbe6019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.462+0000 7fbe64a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe6019ee30 0x7fbe6019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.463+0000 7fbe64a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60114070 con 0x7fbe60103df0 2024-04-02T14:44:44.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.463+0000 7fbe64a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60113ef0 con 0x7fbe60108ec0 2024-04-02T14:44:44.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.463+0000 7fbe64a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601141f0 con 0x7fbe6019ee30 2024-04-02T14:44:44.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.463+0000 7fbe5f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe60103df0 0x7fbe6019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.463+0000 7fbe5f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe60103df0 0x7fbe6019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55188/0 (socket says 172.21.15.62:55188) 2024-04-02T14:44:44.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.463+0000 7fbe5f400640 1 -- 172.21.15.62:0/540903902 learned_addr learned my addr 172.21.15.62:0/540903902 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:44.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.463+0000 7fbe5ea00640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe60108ec0 0x7fbe6019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.463+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe6019ee30 0x7fbe6019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.463+0000 7fbe5f400640 1 -- 172.21.15.62:0/540903902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe6019ee30 msgr2=0x7fbe6019f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe5f400640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe6019ee30 0x7fbe6019f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe5f400640 1 -- 172.21.15.62:0/540903902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe60108ec0 msgr2=0x7fbe6019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe5f400640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe60108ec0 0x7fbe6019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe5f400640 1 -- 172.21.15.62:0/540903902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6019f920 con 0x7fbe60103df0 2024-04-02T14:44:44.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe5ea00640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe60108ec0 0x7fbe6019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:44.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe6019ee30 0x7fbe6019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:44.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe5f400640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe60103df0 0x7fbe6019a570 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000db80 tx=0x7fbe5000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe57e00640 1 -- 172.21.15.62:0/540903902 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50017070 con 0x7fbe60103df0 2024-04-02T14:44:44.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe64a00640 1 -- 172.21.15.62:0/540903902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6019fc10 con 0x7fbe60103df0 2024-04-02T14:44:44.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe57e00640 1 -- 172.21.15.62:0/540903902 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe5000be30 con 0x7fbe60103df0 2024-04-02T14:44:44.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe57e00640 1 -- 172.21.15.62:0/540903902 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe500106c0 con 0x7fbe60103df0 2024-04-02T14:44:44.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.464+0000 7fbe64a00640 1 -- 172.21.15.62:0/540903902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe601a73d0 con 0x7fbe60103df0 2024-04-02T14:44:44.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.468+0000 7fbe68d98740 1 Processor -- start 2024-04-02T14:44:44.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.468+0000 7fbe57e00640 1 -- 172.21.15.62:0/540903902 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fbe50003970 con 0x7fbe60103df0 2024-04-02T14:44:44.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.468+0000 7fbe68d98740 1 -- start start 2024-04-02T14:44:44.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.468+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba0960 0x55b3bfba0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.468+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfba9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.468+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 0x55b3bfbac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.468+0000 7fbe68d98740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfb04770 con 0x55b3bfba9e60 2024-04-02T14:44:44.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.468+0000 7fbe68d98740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfaac700 con 0x55b3bfba1280 2024-04-02T14:44:44.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.468+0000 7fbe68d98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfb00720 con 0x55b3bfba0960 2024-04-02T14:44:44.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfba9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 0x55b3bfbac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba0960 0x55b3bfba0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba0960 0x55b3bfba0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:60498/0 (socket says 172.21.15.62:60498) 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 0x55b3bfbac250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55200/0 (socket says 172.21.15.62:55200) 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5ea00640 1 -- 172.21.15.62:0/1200426307 learned_addr learned my addr 172.21.15.62:0/1200426307 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfba9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:52260/0 (socket says 172.21.15.62:52260) 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5fe00640 1 -- 172.21.15.62:0/1200426307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 msgr2=0x55b3bfba9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/1200426307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfba9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5fe00640 1 -- 172.21.15.62:0/1200426307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 msgr2=0x55b3bfbac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/1200426307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 0x55b3bfbac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.469+0000 7fbe5fe00640 1 -- 172.21.15.62:0/1200426307 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3bf8e27d0 con 0x55b3bfba0960 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.470+0000 7fbe57e00640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbe440a3e40 0x7fbe440a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.470+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/1200426307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba0960 0x55b3bfba0d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009f40 tx=0x7fbe580118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb101b41f09e07dc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.470+0000 7fbe4be00640 1 -- 172.21.15.62:0/1200426307 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58005020 con 0x55b3bfba0960 2024-04-02T14:44:44.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.470+0000 7fbe5f400640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbe440a3e40 0x7fbe440a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.470+0000 7fbe57e00640 1 -- 172.21.15.62:0/540903902 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fbe500d0760 con 0x7fbe60103df0 2024-04-02T14:44:44.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.470+0000 7fbe4be00640 1 -- 172.21.15.62:0/1200426307 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe580047a0 con 0x55b3bfba0960 2024-04-02T14:44:44.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.470+0000 7fbe4be00640 1 -- 172.21.15.62:0/1200426307 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800f8a0 con 0x55b3bfba0960 2024-04-02T14:44:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.471+0000 7fbe68d98740 1 -- 172.21.15.62:0/1200426307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba0960 msgr2=0x55b3bfba0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.471+0000 7fbe5f400640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbe440a3e40 0x7fbe440a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe500cdd80 tx=0x7fbe500d6530 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.471+0000 7fbe68d98740 1 --2- 172.21.15.62:0/1200426307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba0960 0x55b3bfba0d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009f40 tx=0x7fbe580118d0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.471+0000 7fbe68d98740 1 -- 172.21.15.62:0/1200426307 shutdown_connections 2024-04-02T14:44:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.471+0000 7fbe68d98740 1 --2- 172.21.15.62:0/1200426307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 0x55b3bfbac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.471+0000 7fbe68d98740 1 --2- 172.21.15.62:0/1200426307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfba9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.471+0000 7fbe68d98740 1 --2- 172.21.15.62:0/1200426307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba0960 0x55b3bfba0d40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.471+0000 7fbe68d98740 1 -- 172.21.15.62:0/1200426307 >> 172.21.15.62:0/1200426307 conn(0x55b3bfb9b090 msgr2=0x55b3bfb9b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.471+0000 7fbe68d98740 1 -- 172.21.15.62:0/1200426307 shutdown_connections 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.471+0000 7fbe68d98740 1 -- 172.21.15.62:0/1200426307 wait complete. 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.471+0000 7fbe68d98740 1 -- start start 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.472+0000 7fbe68d98740 1 Processor -- start 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.472+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba0960 0x55b3bfba6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.472+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfba73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.472+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 0x55b3bfba7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.472+0000 7fbe68d98740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfbacc50 con 0x55b3bfba0960 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.472+0000 7fbe68d98740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfabfac0 con 0x55b3bfba1280 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.472+0000 7fbe68d98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfbacdd0 con 0x55b3bfba9e60 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.472+0000 7fbe68d98740 10 client.0 _ll_register_callbacks cb 0x55b3bfac2180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfba73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfba73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:52270/0 (socket says 172.21.15.62:52270) 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba0960 0x55b3bfba6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5ea00640 1 -- 172.21.15.62:0/440906105 learned_addr learned my addr 172.21.15.62:0/440906105 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 0x55b3bfba7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5ea00640 1 -- 172.21.15.62:0/440906105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 msgr2=0x55b3bfba7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5ea00640 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 0x55b3bfba7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5ea00640 1 -- 172.21.15.62:0/440906105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba0960 msgr2=0x55b3bfba6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5ea00640 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba0960 0x55b3bfba6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5ea00640 1 -- 172.21.15.62:0/440906105 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3bf8183f0 con 0x55b3bfba1280 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 0x55b3bfba7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe5ea00640 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfba73d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c036040 tx=0x7fbe4c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d774eb731f1336 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.473+0000 7fbe48c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.474+0000 7fbe48c00640 1 -- 172.21.15.62:0/440906105 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c046070 con 0x55b3bfba1280 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.474+0000 7fbe48c00640 1 -- 172.21.15.62:0/440906105 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fbe4c005250 con 0x55b3bfba1280 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.474+0000 7fbe68d98740 1 -- 172.21.15.62:0/440906105 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe580047a0 con 0x55b3bfba1280 2024-04-02T14:44:44.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.474+0000 7fbe48c00640 1 -- 172.21.15.62:0/440906105 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe4c052050 con 0x55b3bfba1280 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.474+0000 7fbe68d98740 10 client.25712 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.474+0000 7fbe68d98740 1 -- 172.21.15.62:0/440906105 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b3bf7d80f0 con 0x55b3bfba1280 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.474+0000 7fbe48c00640 1 -- 172.21.15.62:0/440906105 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe580047a0 con 0x55b3bfba1280 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.474+0000 7fbe48c00640 1 -- 172.21.15.62:0/440906105 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbe4c0035f0 con 0x55b3bfba1280 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.475+0000 7fbe68d98740 10 client.25712 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.475+0000 7fbe68d98740 10 client.25712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.475+0000 7fbe68d98740 20 client.25712 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.475+0000 7fbe68d98740 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b3bfbb9df0 0x55b3bfbbc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.475+0000 7fbe68d98740 4 client.25712 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.475+0000 7fbe68d98740 1 -- 172.21.15.62:0/440906105 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3bfaa5630 con 0x55b3bfbb9df0 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.475+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b3bfbb9df0 0x55b3bfbbc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.475+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b3bfbb9df0 0x55b3bfbbc200 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff02b40419143ce server_cookie=e581595c8821e7ca in_seq=0 out_seq=0 2024-04-02T14:44:44.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.475+0000 7fbe48c00640 10 client.25712 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.476+0000 7fbe48c00640 1 -- 172.21.15.62:0/440906105 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3bfaa5630 con 0x55b3bfbb9df0 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.476+0000 7fbe48c00640 10 client.25712 handle_command_reply: tid=0 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.476+0000 7fbe68d98740 1 client.25712 shutdown 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe68d98740 1 -- 172.21.15.62:0/440906105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 msgr2=0x55b3bfba73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe68d98740 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfba73d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c036040 tx=0x7fbe4c009680 comp rx=0 tx=0).stop 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe68d98740 1 -- 172.21.15.62:0/440906105 shutdown_connections 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe68d98740 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b3bfbb9df0 0x55b3bfbbc200 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe68d98740 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 0x55b3bfba7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe68d98740 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfba73d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe48c00640 0 client.25712 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe68d98740 1 --2- 172.21.15.62:0/440906105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba0960 0x55b3bfba6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe68d98740 1 -- 172.21.15.62:0/440906105 >> 172.21.15.62:0/440906105 conn(0x55b3bfb9b090 msgr2=0x55b3bfb9a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe68d98740 1 -- 172.21.15.62:0/440906105 shutdown_connections 2024-04-02T14:44:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe68d98740 1 -- 172.21.15.62:0/440906105 wait complete. 2024-04-02T14:44:44.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.477+0000 7fbe68d98740 20 client.25712 trim_cache size 0 max 16384 2024-04-02T14:44:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.496+0000 7fbe68d98740 1 Processor -- start 2024-04-02T14:44:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe68d98740 1 -- start start 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba0960 0x55b3bfba95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba1280 0x55b3bfbb5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 0x55b3bfbc1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe68d98740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601af800 con 0x55b3bfba9e60 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe68d98740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601b1340 con 0x55b3bfba0960 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe68d98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601afed0 con 0x55b3bfba1280 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe5ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba1280 0x55b3bfbb5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe5ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba1280 0x55b3bfbb5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:60514/0 (socket says 172.21.15.62:60514) 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe5ea00640 1 -- 172.21.15.62:0/2117939812 learned_addr learned my addr 172.21.15.62:0/2117939812 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/2117939812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba0960 0x55b3bfba95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.497+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/2117939812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 0x55b3bfbc1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.498+0000 7fbe5ea00640 1 -- 172.21.15.62:0/2117939812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba0960 msgr2=0x55b3bfba95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.498+0000 7fbe5ea00640 1 --2- 172.21.15.62:0/2117939812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba0960 0x55b3bfba95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.498+0000 7fbe5ea00640 1 -- 172.21.15.62:0/2117939812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 msgr2=0x55b3bfbc1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.498+0000 7fbe5ea00640 1 --2- 172.21.15.62:0/2117939812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 0x55b3bfbc1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.498+0000 7fbe5ea00640 1 -- 172.21.15.62:0/2117939812 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3bfbac250 con 0x55b3bfba1280 2024-04-02T14:44:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.498+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/2117939812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 0x55b3bfbc1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.498+0000 7fbe5ea00640 1 --2- 172.21.15.62:0/2117939812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba1280 0x55b3bfbb5b10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c02c180 tx=0x7fbe4c009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d88b71d5f3a5d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.498+0000 7fbe4b400640 1 -- 172.21.15.62:0/2117939812 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c046030 con 0x55b3bfba1280 2024-04-02T14:44:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.498+0000 7fbe4b400640 1 -- 172.21.15.62:0/2117939812 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe4c0044b0 con 0x55b3bfba1280 2024-04-02T14:44:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.498+0000 7fbe4b400640 1 -- 172.21.15.62:0/2117939812 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0049f0 con 0x55b3bfba1280 2024-04-02T14:44:44.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.499+0000 7fbe68d98740 1 -- 172.21.15.62:0/2117939812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba1280 msgr2=0x55b3bfbb5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.499+0000 7fbe68d98740 1 --2- 172.21.15.62:0/2117939812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba1280 0x55b3bfbb5b10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c02c180 tx=0x7fbe4c009290 comp rx=0 tx=0).stop 2024-04-02T14:44:44.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.499+0000 7fbe68d98740 1 -- 172.21.15.62:0/2117939812 shutdown_connections 2024-04-02T14:44:44.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.499+0000 7fbe68d98740 1 --2- 172.21.15.62:0/2117939812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba9e60 0x55b3bfbc1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.499+0000 7fbe68d98740 1 --2- 172.21.15.62:0/2117939812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba1280 0x55b3bfbb5b10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.499+0000 7fbe68d98740 1 --2- 172.21.15.62:0/2117939812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba0960 0x55b3bfba95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.499+0000 7fbe68d98740 1 -- 172.21.15.62:0/2117939812 >> 172.21.15.62:0/2117939812 conn(0x55b3bfb9b090 msgr2=0x55b3bfb9a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.499+0000 7fbe68d98740 1 -- 172.21.15.62:0/2117939812 shutdown_connections 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.499+0000 7fbe68d98740 1 -- 172.21.15.62:0/2117939812 wait complete. 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.500+0000 7fbe68d98740 1 -- start start 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.500+0000 7fbe68d98740 1 Processor -- start 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.500+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba0960 0x55b3bfbb2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.500+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfbaeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.500+0000 7fbe68d98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 0x55b3bfbaf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.500+0000 7fbe68d98740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601b0c70 con 0x55b3bfba0960 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.500+0000 7fbe68d98740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601b05a0 con 0x55b3bfba1280 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.500+0000 7fbe68d98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601ad800 con 0x55b3bfba9e60 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.500+0000 7fbe68d98740 10 client.0 _ll_register_callbacks cb 0x7fbe4c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfbaeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba0960 0x55b3bfbb2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfbaeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:52294/0 (socket says 172.21.15.62:52294) 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5ea00640 1 -- 172.21.15.62:0/982417055 learned_addr learned my addr 172.21.15.62:0/982417055 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5ea00640 1 -- 172.21.15.62:0/982417055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 msgr2=0x55b3bfbaf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 0x55b3bfbaf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5ea00640 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 0x55b3bfbaf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5ea00640 1 -- 172.21.15.62:0/982417055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba0960 msgr2=0x55b3bfbb2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5ea00640 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba0960 0x55b3bfbb2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5ea00640 1 -- 172.21.15.62:0/982417055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3bfba0320 con 0x55b3bfba1280 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 0x55b3bfbaf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe5ea00640 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfbaeff0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c005ec0 tx=0x7fbe4c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc76f292f886b9b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.501+0000 7fbe48c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe48c00640 1 -- 172.21.15.62:0/982417055 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c046040 con 0x55b3bfba1280 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe48c00640 1 -- 172.21.15.62:0/982417055 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fbe4c005450 con 0x55b3bfba1280 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe68d98740 1 -- 172.21.15.62:0/982417055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe4c0044b0 con 0x55b3bfba1280 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe48c00640 1 -- 172.21.15.62:0/982417055 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe4c003df0 con 0x55b3bfba1280 2024-04-02T14:44:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe68d98740 10 client.25718 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe68d98740 1 -- 172.21.15.62:0/982417055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b3bfc21020 con 0x55b3bfba1280 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe48c00640 1 -- 172.21.15.62:0/982417055 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe4c0044b0 con 0x55b3bfba1280 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe48c00640 1 -- 172.21.15.62:0/982417055 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbe4c0347c0 con 0x55b3bfba1280 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe68d98740 10 client.25718 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe68d98740 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe68d98740 20 client.25718 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe68d98740 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b3bfc216f0 0x55b3bfc23b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe68d98740 4 client.25718 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.502+0000 7fbe68d98740 1 -- 172.21.15.62:0/982417055 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3bfb11cb0 con 0x55b3bfc216f0 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.503+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b3bfc216f0 0x55b3bfc23b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.503+0000 7fbe5fe00640 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b3bfc216f0 0x55b3bfc23b10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05606617e258558 server_cookie=452af3f2f9412aca in_seq=0 out_seq=0 2024-04-02T14:44:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.503+0000 7fbe48c00640 10 client.25718 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe48c00640 1 -- 172.21.15.62:0/982417055 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b3bfb11cb0 con 0x55b3bfc216f0 2024-04-02T14:44:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe48c00640 10 client.25718 handle_command_reply: tid=0 2024-04-02T14:44:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe68d98740 1 client.25718 shutdown 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe68d98740 1 -- 172.21.15.62:0/982417055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 msgr2=0x55b3bfbaeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe68d98740 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfbaeff0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c005ec0 tx=0x7fbe4c003820 comp rx=0 tx=0).stop 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe68d98740 1 -- 172.21.15.62:0/982417055 shutdown_connections 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe68d98740 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b3bfc216f0 0x55b3bfc23b10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe68d98740 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3bfba9e60 0x55b3bfbaf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe68d98740 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3bfba1280 0x55b3bfbaeff0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe68d98740 1 --2- 172.21.15.62:0/982417055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3bfba0960 0x55b3bfbb2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe68d98740 1 -- 172.21.15.62:0/982417055 >> 172.21.15.62:0/982417055 conn(0x55b3bfb9b090 msgr2=0x55b3bfba6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe48c00640 0 client.25718 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.504+0000 7fbe68d98740 1 -- 172.21.15.62:0/982417055 shutdown_connections 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe68d98740 1 -- 172.21.15.62:0/982417055 wait complete. 2024-04-02T14:44:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe68d98740 20 client.25718 trim_cache size 0 max 16384 2024-04-02T14:44:44.505 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:44:44.505 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T14:44:44.505 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-02T14:44:44.505 INFO:teuthology.orchestra.run.smithi062.stdout: "2fb16472-81f2-4b0a-8ff0-4636c85f9286": { 2024-04-02T14:44:44.505 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-04-02T14:44:44.505 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "2fb16472-81f2-4b0a-8ff0-4636c85f9286", 2024-04-02T14:44:44.505 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-04-02T14:44:44.505 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:44:44.505 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:44:44.505 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:44:44.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe64a00640 1 -- 172.21.15.62:0/540903902 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbe440a3e40 msgr2=0x7fbe440a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe64a00640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbe440a3e40 0x7fbe440a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe500cdd80 tx=0x7fbe500d6530 comp rx=0 tx=0).stop 2024-04-02T14:44:44.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe64a00640 1 -- 172.21.15.62:0/540903902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe60103df0 msgr2=0x7fbe6019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:44.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe64a00640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe60103df0 0x7fbe6019a570 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000db80 tx=0x7fbe5000df80 comp rx=0 tx=0).stop 2024-04-02T14:44:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe64a00640 1 -- 172.21.15.62:0/540903902 shutdown_connections 2024-04-02T14:44:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe64a00640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbe440a3e40 0x7fbe440a6300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe64a00640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe6019ee30 0x7fbe6019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe64a00640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe60108ec0 0x7fbe6019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe64a00640 1 --2- 172.21.15.62:0/540903902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe60103df0 0x7fbe6019a570 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.505+0000 7fbe64a00640 1 -- 172.21.15.62:0/540903902 >> 172.21.15.62:0/540903902 conn(0x7fbe600ff630 msgr2=0x7fbe60101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.506+0000 7fbe64a00640 1 -- 172.21.15.62:0/540903902 shutdown_connections 2024-04-02T14:44:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:44:44.506+0000 7fbe64a00640 1 -- 172.21.15.62:0/540903902 wait complete. 2024-04-02T14:44:44.519 INFO:tasks.cephfs.filesystem:scrub status for tag:2fb16472-81f2-4b0a-8ff0-4636c85f9286 - {'path': '/', 'tag': '2fb16472-81f2-4b0a-8ff0-4636c85f9286', 'options': 'recursive,force'} 2024-04-02T14:44:45.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:45 smithi204 ceph-mon[35948]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T14:44:45.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:45 smithi204 ceph-mon[35948]: scrub queued for path: / 2024-04-02T14:44:45.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:45 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:45.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:45 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:44:45.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:45 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:45.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:45 smithi062 ceph-mon[32116]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T14:44:45.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:45 smithi062 ceph-mon[32116]: scrub queued for path: / 2024-04-02T14:44:45.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:45 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:45.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:45 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:44:45.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:45 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:45.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:45 smithi113 ceph-mon[36135]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T14:44:45.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:45 smithi113 ceph-mon[36135]: scrub queued for path: / 2024-04-02T14:44:45.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:45 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:45.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:45 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:44:45.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:45 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:44:46.692 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:46 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:44:46.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:46 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:44:46.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:46 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:44:47.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:47 smithi204 ceph-mon[35948]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-02T14:44:47.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:47 smithi062 ceph-mon[32116]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-02T14:44:47.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:47 smithi113 ceph-mon[36135]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-02T14:44:49.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:49 smithi204 ceph-mon[35948]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-02T14:44:49.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:49 smithi062 ceph-mon[32116]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-02T14:44:49.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:49 smithi113 ceph-mon[36135]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-02T14:44:51.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:51 smithi204 ceph-mon[35948]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-02T14:44:51.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:51 smithi062 ceph-mon[32116]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-02T14:44:51.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:51 smithi113 ceph-mon[36135]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-02T14:44:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.253+0000 7fe903c00640 1 -- 172.21.15.62:0/2892307730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8fc107b70 msgr2=0x7fe8fc107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.253+0000 7fe903c00640 1 --2- 172.21.15.62:0/2892307730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8fc107b70 0x7fe8fc107f50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009900 tx=0x7fe8f002bc30 comp rx=0 tx=0).stop 2024-04-02T14:44:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.253+0000 7fe903c00640 1 -- 172.21.15.62:0/2892307730 shutdown_connections 2024-04-02T14:44:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.253+0000 7fe903c00640 1 --2- 172.21.15.62:0/2892307730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8fc102490 0x7fe8fc10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.253+0000 7fe903c00640 1 --2- 172.21.15.62:0/2892307730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8fc101ad0 0x7fe8fc101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.253+0000 7fe903c00640 1 --2- 172.21.15.62:0/2892307730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8fc107b70 0x7fe8fc107f50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.253+0000 7fe903c00640 1 -- 172.21.15.62:0/2892307730 >> 172.21.15.62:0/2892307730 conn(0x7fe8fc0fd5c0 msgr2=0x7fe8fc0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.253+0000 7fe903c00640 1 -- 172.21.15.62:0/2892307730 shutdown_connections 2024-04-02T14:44:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.253+0000 7fe903c00640 1 -- 172.21.15.62:0/2892307730 wait complete. 2024-04-02T14:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.254+0000 7fe903c00640 1 Processor -- start 2024-04-02T14:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.255+0000 7fe903c00640 1 -- start start 2024-04-02T14:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.255+0000 7fe903c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8fc101ad0 0x7fe8fc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.255+0000 7fe903c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8fc102490 0x7fe8fc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.255+0000 7fe903c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8fc107b70 0x7fe8fc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.255+0000 7fe903c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc112010 con 0x7fe8fc107b70 2024-04-02T14:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.255+0000 7fe903c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc111e90 con 0x7fe8fc101ad0 2024-04-02T14:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.255+0000 7fe903c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc112190 con 0x7fe8fc102490 2024-04-02T14:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.255+0000 7fe901200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8fc101ad0 0x7fe8fc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.255+0000 7fe901c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8fc107b70 0x7fe8fc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.255+0000 7fe901c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8fc107b70 0x7fe8fc19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55236/0 (socket says 172.21.15.62:55236) 2024-04-02T14:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.255+0000 7fe901c00640 1 -- 172.21.15.62:0/3511910286 learned_addr learned my addr 172.21.15.62:0/3511910286 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.256+0000 7fe8fbe00640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8fc102490 0x7fe8fc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.256+0000 7fe901200640 1 -- 172.21.15.62:0/3511910286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8fc102490 msgr2=0x7fe8fc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.256+0000 7fe901200640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8fc102490 0x7fe8fc1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.256+0000 7fe901200640 1 -- 172.21.15.62:0/3511910286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8fc107b70 msgr2=0x7fe8fc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.256+0000 7fe901200640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8fc107b70 0x7fe8fc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.256+0000 7fe901200640 1 -- 172.21.15.62:0/3511910286 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc19d430 con 0x7fe8fc101ad0 2024-04-02T14:44:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.256+0000 7fe901c00640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8fc107b70 0x7fe8fc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.256+0000 7fe901200640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8fc101ad0 0x7fe8fc198490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0036040 tx=0x7fe8f002dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.257+0000 7fe8f9600640 1 -- 172.21.15.62:0/3511910286 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0043020 con 0x7fe8fc101ad0 2024-04-02T14:44:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.257+0000 7fe903c00640 1 -- 172.21.15.62:0/3511910286 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc19d6c0 con 0x7fe8fc101ad0 2024-04-02T14:44:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.257+0000 7fe8f9600640 1 -- 172.21.15.62:0/3511910286 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe8f0004e50 con 0x7fe8fc101ad0 2024-04-02T14:44:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.257+0000 7fe8f9600640 1 -- 172.21.15.62:0/3511910286 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0003970 con 0x7fe8fc101ad0 2024-04-02T14:44:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.257+0000 7fe903c00640 1 -- 172.21.15.62:0/3511910286 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc1a4fb0 con 0x7fe8fc101ad0 2024-04-02T14:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.260+0000 7fe906982740 1 Processor -- start 2024-04-02T14:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.260+0000 7fe906982740 1 -- start start 2024-04-02T14:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.260+0000 7fe906982740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e2b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.260+0000 7fe906982740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e33e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.260+0000 7fe906982740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e343b0 0x55f5d9e367a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.260+0000 7fe906982740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9d8fd70 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.260+0000 7fe906982740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9d382a0 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.260+0000 7fe906982740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9d8bd20 con 0x55f5d9e343b0 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe901c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e2b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe901c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e2b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45154/0 (socket says 172.21.15.62:45154) 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe901c00640 1 -- 172.21.15.62:0/176619773 learned_addr learned my addr 172.21.15.62:0/176619773 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe8fbe00640 1 --2- 172.21.15.62:0/176619773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e343b0 0x55f5d9e367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe901c00640 1 -- 172.21.15.62:0/176619773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e343b0 msgr2=0x55f5d9e367a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe901c00640 1 --2- 172.21.15.62:0/176619773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e343b0 0x55f5d9e367a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe901c00640 1 -- 172.21.15.62:0/176619773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2b7d0 msgr2=0x55f5d9e33e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe901c00640 1 --2- 172.21.15.62:0/176619773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e33e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe901c00640 1 -- 172.21.15.62:0/176619773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d9d4c840 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe8fbe00640 1 --2- 172.21.15.62:0/176619773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e343b0 0x55f5d9e367a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe8f9600640 1 -- 172.21.15.62:0/3511910286 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fe8f0046020 con 0x7fe8fc101ad0 2024-04-02T14:44:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.261+0000 7fe901c00640 1 --2- 172.21.15.62:0/176619773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e2b290 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4011850 tx=0x7fe8f4011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0df5d390d4873ce server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.262+0000 7fe8ee000640 1 -- 172.21.15.62:0/176619773 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4011330 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.262+0000 7fe8ee000640 1 -- 172.21.15.62:0/176619773 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe8f4010430 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.262+0000 7fe8ee000640 1 -- 172.21.15.62:0/176619773 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f400fea0 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.262+0000 7fe8f9600640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe8e40a3ee0 0x7fe8e40a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.262+0000 7fe906982740 1 -- 172.21.15.62:0/176619773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 msgr2=0x55f5d9e2b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.262+0000 7fe906982740 1 --2- 172.21.15.62:0/176619773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e2b290 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4011850 tx=0x7fe8f4011d10 comp rx=0 tx=0).stop 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.263+0000 7fe8f9600640 1 -- 172.21.15.62:0/3511910286 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fe8f00f2810 con 0x7fe8fc101ad0 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.263+0000 7fe906982740 1 -- 172.21.15.62:0/176619773 shutdown_connections 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.263+0000 7fe906982740 1 --2- 172.21.15.62:0/176619773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e343b0 0x55f5d9e367a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.263+0000 7fe906982740 1 --2- 172.21.15.62:0/176619773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e33e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.263+0000 7fe906982740 1 --2- 172.21.15.62:0/176619773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e2b290 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.263+0000 7fe906982740 1 -- 172.21.15.62:0/176619773 >> 172.21.15.62:0/176619773 conn(0x55f5d9e255e0 msgr2=0x55f5d9e259c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.263+0000 7fe901200640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe8e40a3ee0 0x7fe8e40a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.263+0000 7fe906982740 1 -- 172.21.15.62:0/176619773 shutdown_connections 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.263+0000 7fe906982740 1 -- 172.21.15.62:0/176619773 wait complete. 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.264+0000 7fe901200640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe8e40a3ee0 0x7fe8e40a63a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00effa0 tx=0x7fe8f00057c0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.264+0000 7fe906982740 1 -- start start 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.264+0000 7fe906982740 1 Processor -- start 2024-04-02T14:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.264+0000 7fe906982740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e313a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.264+0000 7fe906982740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e318e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.264+0000 7fe906982740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e343b0 0x55f5d9e31e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.264+0000 7fe906982740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9e371a0 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.264+0000 7fe906982740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9d4b660 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.264+0000 7fe906982740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9e37320 con 0x55f5d9e343b0 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.264+0000 7fe906982740 10 client.0 _ll_register_callbacks cb 0x55f5d9d4b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe901c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e313a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe901c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e313a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45162/0 (socket says 172.21.15.62:45162) 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe901c00640 1 -- 172.21.15.62:0/2040209757 learned_addr learned my addr 172.21.15.62:0/2040209757 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe8fbe00640 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe901c00640 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e343b0 0x55f5d9e31e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe8fbe00640 1 -- 172.21.15.62:0/2040209757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e343b0 msgr2=0x55f5d9e31e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe8fbe00640 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e343b0 0x55f5d9e31e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe8fbe00640 1 -- 172.21.15.62:0/2040209757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 msgr2=0x55f5d9e313a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe8fbe00640 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e313a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe8fbe00640 1 -- 172.21.15.62:0/2040209757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5d9d5ae80 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe901c00640 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e313a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe8fbe00640 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e318e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc1070d0 tx=0x7fe8e8012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c680a248b43d5e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.265+0000 7fe8e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.266+0000 7fe8e2a00640 1 -- 172.21.15.62:0/2040209757 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8004110 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.266+0000 7fe8e2a00640 1 -- 172.21.15.62:0/2040209757 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fe8e80042b0 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.266+0000 7fe906982740 1 -- 172.21.15.62:0/2040209757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f4010430 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.266+0000 7fe8e2a00640 1 -- 172.21.15.62:0/2040209757 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8e80044b0 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.266+0000 7fe906982740 10 client.25727 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.266+0000 7fe906982740 1 -- 172.21.15.62:0/2040209757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5d9aa4210 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.267+0000 7fe8e2a00640 1 -- 172.21.15.62:0/2040209757 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe8f4010430 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.267+0000 7fe8e2a00640 1 -- 172.21.15.62:0/2040209757 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe8e8003780 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.268+0000 7fe906982740 10 client.25727 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.268+0000 7fe906982740 10 client.25727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.268+0000 7fe906982740 20 client.25727 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.268+0000 7fe906982740 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f5d9e48b00 0x55f5d9e4aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.268+0000 7fe906982740 4 client.25727 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.268+0000 7fe906982740 1 -- 172.21.15.62:0/2040209757 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5d9d3fcd0 con 0x55f5d9e48b00 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.268+0000 7fe901c00640 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f5d9e48b00 0x55f5d9e4aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.268+0000 7fe901c00640 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f5d9e48b00 0x55f5d9e4aef0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4596d4f5cc947d71 server_cookie=b5b7de6f9dfc2251 in_seq=0 out_seq=0 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.268+0000 7fe8e2a00640 10 client.25727 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.269+0000 7fe8e2a00640 1 -- 172.21.15.62:0/2040209757 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5d9d3fcd0 con 0x55f5d9e48b00 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.269+0000 7fe8e2a00640 10 client.25727 handle_command_reply: tid=0 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.269+0000 7fe906982740 1 client.25727 shutdown 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe906982740 1 -- 172.21.15.62:0/2040209757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2b7d0 msgr2=0x55f5d9e318e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe906982740 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e318e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc1070d0 tx=0x7fe8e8012b60 comp rx=0 tx=0).stop 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe906982740 1 -- 172.21.15.62:0/2040209757 shutdown_connections 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe906982740 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f5d9e48b00 0x55f5d9e4aef0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe906982740 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e343b0 0x55f5d9e31e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe906982740 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e318e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe906982740 1 --2- 172.21.15.62:0/2040209757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e313a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe8e2a00640 0 client.25727 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe906982740 1 -- 172.21.15.62:0/2040209757 >> 172.21.15.62:0/2040209757 conn(0x55f5d9e255e0 msgr2=0x55f5d9e27330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe906982740 1 -- 172.21.15.62:0/2040209757 shutdown_connections 2024-04-02T14:44:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe906982740 1 -- 172.21.15.62:0/2040209757 wait complete. 2024-04-02T14:44:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.270+0000 7fe906982740 20 client.25727 trim_cache size 0 max 16384 2024-04-02T14:44:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.286+0000 7fe906982740 1 Processor -- start 2024-04-02T14:44:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.286+0000 7fe906982740 1 -- start start 2024-04-02T14:44:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.287+0000 7fe906982740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e33ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.287+0000 7fe906982740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.287+0000 7fe906982740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e343b0 0x55f5d9e31a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.287+0000 7fe906982740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1ab520 con 0x55f5d9e343b0 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.287+0000 7fe906982740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1ad060 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.287+0000 7fe906982740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1abbf0 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.287+0000 7fe901c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e33ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.287+0000 7fe901c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e33ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55412/0 (socket says 172.21.15.62:55412) 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.287+0000 7fe901c00640 1 -- 172.21.15.62:0/3160616841 learned_addr learned my addr 172.21.15.62:0/3160616841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.287+0000 7fe8fbe00640 1 --2- 172.21.15.62:0/3160616841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.287+0000 7fe901c00640 1 --2- 172.21.15.62:0/3160616841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e343b0 0x55f5d9e31a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.288+0000 7fe901c00640 1 -- 172.21.15.62:0/3160616841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e2b7d0 msgr2=0x55f5d9e444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.288+0000 7fe901c00640 1 --2- 172.21.15.62:0/3160616841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.288+0000 7fe901c00640 1 -- 172.21.15.62:0/3160616841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e343b0 msgr2=0x55f5d9e31a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.288+0000 7fe901c00640 1 --2- 172.21.15.62:0/3160616841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e343b0 0x55f5d9e31a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.288+0000 7fe901c00640 1 -- 172.21.15.62:0/3160616841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d9b70620 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.288+0000 7fe901c00640 1 --2- 172.21.15.62:0/3160616841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e33ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4010ee0 tx=0x7fe8f4009a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27766d5c02325c5f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.288+0000 7fe8ed600640 1 -- 172.21.15.62:0/3160616841 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f401f040 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.288+0000 7fe8ed600640 1 -- 172.21.15.62:0/3160616841 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe8f40109c0 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.288+0000 7fe8ed600640 1 -- 172.21.15.62:0/3160616841 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40052b0 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.289+0000 7fe906982740 1 -- 172.21.15.62:0/3160616841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2aeb0 msgr2=0x55f5d9e33ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.289+0000 7fe906982740 1 --2- 172.21.15.62:0/3160616841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e33ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4010ee0 tx=0x7fe8f4009a30 comp rx=0 tx=0).stop 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.289+0000 7fe906982740 1 -- 172.21.15.62:0/3160616841 shutdown_connections 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.289+0000 7fe906982740 1 --2- 172.21.15.62:0/3160616841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e343b0 0x55f5d9e31a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.289+0000 7fe906982740 1 --2- 172.21.15.62:0/3160616841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.289+0000 7fe906982740 1 --2- 172.21.15.62:0/3160616841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e33ad0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.289+0000 7fe906982740 1 -- 172.21.15.62:0/3160616841 >> 172.21.15.62:0/3160616841 conn(0x55f5d9e255e0 msgr2=0x55f5d9e31360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.289+0000 7fe906982740 1 -- 172.21.15.62:0/3160616841 shutdown_connections 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.289+0000 7fe906982740 1 -- 172.21.15.62:0/3160616841 wait complete. 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.290+0000 7fe906982740 1 -- start start 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.290+0000 7fe906982740 1 Processor -- start 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.290+0000 7fe906982740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e2e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.290+0000 7fe906982740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e2e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.290+0000 7fe906982740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e343b0 0x55f5d9e2ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.290+0000 7fe906982740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1ac990 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.290+0000 7fe906982740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1ac2c0 con 0x55f5d9e343b0 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.290+0000 7fe906982740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1a9520 con 0x55f5d9e2aeb0 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.290+0000 7fe906982740 10 client.0 _ll_register_callbacks cb 0x7fe8e8009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe8fbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e2e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe8fbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e2e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45170/0 (socket says 172.21.15.62:45170) 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe8fbe00640 1 -- 172.21.15.62:0/3372139722 learned_addr learned my addr 172.21.15.62:0/3372139722 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe901c00640 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e2e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe901c00640 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e343b0 0x55f5d9e2ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe8fbe00640 1 -- 172.21.15.62:0/3372139722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e2aeb0 msgr2=0x55f5d9e2e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe8fbe00640 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e2e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe8fbe00640 1 -- 172.21.15.62:0/3372139722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e343b0 msgr2=0x55f5d9e2ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe8fbe00640 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e343b0 0x55f5d9e2ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe8fbe00640 1 -- 172.21.15.62:0/3372139722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5d9eae110 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe901c00640 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e2e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe8fbe00640 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e2e710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55f5d9e32970 tx=0x7fe8e8004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d107aadfd8ab6ba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.291+0000 7fe8e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.292+0000 7fe8e2a00640 1 -- 172.21.15.62:0/3372139722 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8035060 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.292+0000 7fe906982740 1 -- 172.21.15.62:0/3372139722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f40109c0 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.292+0000 7fe8e2a00640 1 -- 172.21.15.62:0/3372139722 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fe8e8003710 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.292+0000 7fe8e2a00640 1 -- 172.21.15.62:0/3372139722 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8e802c040 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.292+0000 7fe906982740 10 client.15609 fetch_fsmap learned FSMap version 28 2024-04-02T14:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.292+0000 7fe906982740 1 -- 172.21.15.62:0/3372139722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5d9eaf400 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.292+0000 7fe8e2a00640 1 -- 172.21.15.62:0/3372139722 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe8f40109c0 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.292+0000 7fe8e2a00640 1 -- 172.21.15.62:0/3372139722 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe8e8004650 con 0x55f5d9e2b7d0 2024-04-02T14:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.292+0000 7fe906982740 10 client.15609 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.292+0000 7fe906982740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.292+0000 7fe906982740 20 client.15609 populate_metadata read hostname 'smithi062' 2024-04-02T14:44:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.293+0000 7fe906982740 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f5d9eafbb0 0x55f5d9eb1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:44:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.293+0000 7fe906982740 4 client.15609 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:44:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.293+0000 7fe906982740 1 -- 172.21.15.62:0/3372139722 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5d9d9d2b0 con 0x55f5d9eafbb0 2024-04-02T14:44:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.293+0000 7fe901c00640 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f5d9eafbb0 0x55f5d9eb1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:44:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.293+0000 7fe901c00640 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f5d9eafbb0 0x55f5d9eb1fd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237f79977e125401 server_cookie=2655c4d97886ddfe in_seq=0 out_seq=0 2024-04-02T14:44:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.293+0000 7fe8e2a00640 10 client.15609 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe8e2a00640 1 -- 172.21.15.62:0/3372139722 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5d9d9d2b0 con 0x55f5d9eafbb0 2024-04-02T14:44:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe8e2a00640 10 client.15609 handle_command_reply: tid=0 2024-04-02T14:44:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe906982740 1 client.15609 shutdown 2024-04-02T14:44:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe906982740 1 -- 172.21.15.62:0/3372139722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2b7d0 msgr2=0x55f5d9e2e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe906982740 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e2e710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55f5d9e32970 tx=0x7fe8e8004420 comp rx=0 tx=0).stop 2024-04-02T14:44:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe906982740 1 -- 172.21.15.62:0/3372139722 shutdown_connections 2024-04-02T14:44:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe906982740 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f5d9eafbb0 0x55f5d9eb1fd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe906982740 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f5d9e343b0 0x55f5d9e2ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe906982740 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5d9e2b7d0 0x55f5d9e2e710 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe906982740 1 --2- 172.21.15.62:0/3372139722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5d9e2aeb0 0x55f5d9e2e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe8e2a00640 0 client.15609 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:44:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe906982740 1 -- 172.21.15.62:0/3372139722 >> 172.21.15.62:0/3372139722 conn(0x55f5d9e255e0 msgr2=0x55f5d9e31bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe906982740 1 -- 172.21.15.62:0/3372139722 shutdown_connections 2024-04-02T14:44:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.301+0000 7fe906982740 1 -- 172.21.15.62:0/3372139722 wait complete. 2024-04-02T14:44:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe906982740 20 client.15609 trim_cache size 0 max 16384 2024-04-02T14:44:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:44:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:44:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:44:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:44:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 -- 172.21.15.62:0/3511910286 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe8e40a3ee0 msgr2=0x7fe8e40a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe8e40a3ee0 0x7fe8e40a63a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00effa0 tx=0x7fe8f00057c0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 -- 172.21.15.62:0/3511910286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8fc101ad0 msgr2=0x7fe8fc198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:44:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8fc101ad0 0x7fe8fc198490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0036040 tx=0x7fe8f002dc00 comp rx=0 tx=0).stop 2024-04-02T14:44:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 -- 172.21.15.62:0/3511910286 shutdown_connections 2024-04-02T14:44:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe8e40a3ee0 0x7fe8e40a63a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8fc107b70 0x7fe8fc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8fc102490 0x7fe8fc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 --2- 172.21.15.62:0/3511910286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8fc101ad0 0x7fe8fc198490 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:44:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 -- 172.21.15.62:0/3511910286 >> 172.21.15.62:0/3511910286 conn(0x7fe8fc0fd5c0 msgr2=0x7fe8fc0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:44:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 -- 172.21.15.62:0/3511910286 shutdown_connections 2024-04-02T14:44:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:44:52.302+0000 7fe903c00640 1 -- 172.21.15.62:0/3511910286 wait complete. 2024-04-02T14:44:53.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:53 smithi204 ceph-mon[35948]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-02T14:44:53.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:53 smithi062 ceph-mon[32116]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-02T14:44:53.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:53 smithi113 ceph-mon[36135]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-02T14:44:55.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:55 smithi204 ceph-mon[35948]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-02T14:44:55.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:55 smithi062 ceph-mon[32116]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-02T14:44:55.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:55 smithi113 ceph-mon[36135]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-02T14:44:57.678 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:57 smithi113 ceph-mon[36135]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-02T14:44:57.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:57 smithi204 ceph-mon[35948]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-02T14:44:57.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:57 smithi062 ceph-mon[32116]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-02T14:44:59.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:44:59 smithi204 ceph-mon[35948]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-02T14:44:59.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:44:59 smithi062 ceph-mon[32116]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-02T14:44:59.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:44:59 smithi113 ceph-mon[36135]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-02T14:45:00.839 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:45:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:45:00.452+0000 7f0c2e400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:45:00.839 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:45:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:45:00.452+0000 7f0c2e400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:45:00.840 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:45:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:45:00.452+0000 7f0c2e400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:45:00.840 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:45:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:45:00.453+0000 7f0c2e400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:45:00.840 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:45:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:45:00.453+0000 7f0c2e400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T14:45:01.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:01 smithi204 ceph-mon[35948]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-02T14:45:01.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:01 smithi062 ceph-mon[32116]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-02T14:45:01.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:01 smithi113 ceph-mon[36135]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-02T14:45:03.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:03 smithi204 ceph-mon[35948]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-02T14:45:03.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:03 smithi204 ceph-mon[35948]: mgrmap e26: x(active, since 62s), standbys: z, y 2024-04-02T14:45:03.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:03 smithi062 ceph-mon[32116]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-02T14:45:03.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:03 smithi062 ceph-mon[32116]: mgrmap e26: x(active, since 62s), standbys: z, y 2024-04-02T14:45:03.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:03 smithi113 ceph-mon[36135]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-02T14:45:03.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:03 smithi113 ceph-mon[36135]: mgrmap e26: x(active, since 62s), standbys: z, y 2024-04-02T14:45:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.456+0000 7f1fd6a00640 1 -- 172.21.15.62:0/394428235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd0102ab0 msgr2=0x7f1fd0102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.456+0000 7f1fd6a00640 1 --2- 172.21.15.62:0/394428235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd0102ab0 0x7f1fd0102e90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009900 tx=0x7f1fc402bcd0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.456+0000 7f1fd6a00640 1 -- 172.21.15.62:0/394428235 shutdown_connections 2024-04-02T14:45:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.456+0000 7f1fd6a00640 1 --2- 172.21.15.62:0/394428235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd010bed0 0x7f1fd010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.456+0000 7f1fd6a00640 1 --2- 172.21.15.62:0/394428235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fd01033d0 0x7f1fd010b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.456+0000 7f1fd6a00640 1 --2- 172.21.15.62:0/394428235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd0102ab0 0x7f1fd0102e90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.456+0000 7f1fd6a00640 1 -- 172.21.15.62:0/394428235 >> 172.21.15.62:0/394428235 conn(0x7f1fd00fc310 msgr2=0x7f1fd00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.456+0000 7f1fd6a00640 1 -- 172.21.15.62:0/394428235 shutdown_connections 2024-04-02T14:45:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.456+0000 7f1fd6a00640 1 -- 172.21.15.62:0/394428235 wait complete. 2024-04-02T14:45:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.457+0000 7f1fd6a00640 1 Processor -- start 2024-04-02T14:45:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.457+0000 7f1fd6a00640 1 -- start start 2024-04-02T14:45:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.457+0000 7f1fd6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fd0102ab0 0x7f1fd019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.457+0000 7f1fd6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd01033d0 0x7f1fd019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd010bed0 0x7f1fd019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd6a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd0111df0 con 0x7f1fd0102ab0 2024-04-02T14:45:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd6a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd0111c70 con 0x7f1fd01033d0 2024-04-02T14:45:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd6a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd0111f70 con 0x7f1fd010bed0 2024-04-02T14:45:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fd0102ab0 0x7f1fd019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fd0102ab0 0x7f1fd019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57480/0 (socket says 172.21.15.62:57480) 2024-04-02T14:45:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd5600640 1 -- 172.21.15.62:0/3123454167 learned_addr learned my addr 172.21.15.62:0/3123454167 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd01033d0 0x7f1fd019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd6000640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd010bed0 0x7f1fd019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd5600640 1 -- 172.21.15.62:0/3123454167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd010bed0 msgr2=0x7f1fd019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd5600640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd010bed0 0x7f1fd019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd5600640 1 -- 172.21.15.62:0/3123454167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd01033d0 msgr2=0x7f1fd019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd5600640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd01033d0 0x7f1fd019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.458+0000 7f1fd5600640 1 -- 172.21.15.62:0/3123454167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd019f500 con 0x7f1fd0102ab0 2024-04-02T14:45:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.459+0000 7f1fd5600640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fd0102ab0 0x7f1fd019a510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4005940 tx=0x7f1fc4004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.459+0000 7f1fce000640 1 -- 172.21.15.62:0/3123454167 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4043020 con 0x7f1fd0102ab0 2024-04-02T14:45:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.459+0000 7f1fce000640 1 -- 172.21.15.62:0/3123454167 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1fc4004250 con 0x7f1fd0102ab0 2024-04-02T14:45:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.459+0000 7f1fce000640 1 -- 172.21.15.62:0/3123454167 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4003a80 con 0x7f1fd0102ab0 2024-04-02T14:45:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.459+0000 7f1fd6a00640 1 -- 172.21.15.62:0/3123454167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd019f790 con 0x7f1fd0102ab0 2024-04-02T14:45:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.459+0000 7f1fd6a00640 1 -- 172.21.15.62:0/3123454167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd01a7080 con 0x7f1fd0102ab0 2024-04-02T14:45:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.463+0000 7f1fdad84740 1 Processor -- start 2024-04-02T14:45:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.463+0000 7f1fdad84740 1 -- start start 2024-04-02T14:45:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.463+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b5bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.463+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b5c210 0x55b377b648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.463+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b64df0 0x55b377b671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.463+0000 7f1fdad84740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b377ab7d70 con 0x55b377b64df0 2024-04-02T14:45:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.463+0000 7f1fdad84740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b377a602a0 con 0x55b377b5b8f0 2024-04-02T14:45:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.463+0000 7f1fdad84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b377ab3d20 con 0x55b377b5c210 2024-04-02T14:45:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.464+0000 7f1fd6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b5bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.464+0000 7f1fd6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b5bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:47478/0 (socket says 172.21.15.62:47478) 2024-04-02T14:45:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.464+0000 7f1fd6000640 1 -- 172.21.15.62:0/1821322626 learned_addr learned my addr 172.21.15.62:0/1821322626 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.464+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/1821322626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b64df0 0x55b377b671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.464+0000 7f1fd6000640 1 -- 172.21.15.62:0/1821322626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b5c210 msgr2=0x55b377b648b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T14:45:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.464+0000 7f1fd6000640 1 --2- 172.21.15.62:0/1821322626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b5c210 0x55b377b648b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.464+0000 7f1fd6000640 1 -- 172.21.15.62:0/1821322626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b64df0 msgr2=0x55b377b671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.464+0000 7f1fd6000640 1 --2- 172.21.15.62:0/1821322626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b64df0 0x55b377b671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.464+0000 7f1fd6000640 1 -- 172.21.15.62:0/1821322626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b377a74840 con 0x55b377b5b8f0 2024-04-02T14:45:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.464+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/1821322626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b64df0 0x55b377b671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.465+0000 7f1fce000640 1 -- 172.21.15.62:0/3123454167 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f1fc4046020 con 0x7f1fd0102ab0 2024-04-02T14:45:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.465+0000 7f1fd6000640 1 --2- 172.21.15.62:0/1821322626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b5bcd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009f40 tx=0x7f1fc8010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1e4b3a0b432b88 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.465+0000 7f1fc2600640 1 -- 172.21.15.62:0/1821322626 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8003b70 con 0x55b377b5b8f0 2024-04-02T14:45:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.465+0000 7f1fc2600640 1 -- 172.21.15.62:0/1821322626 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1fc8003d10 con 0x55b377b5b8f0 2024-04-02T14:45:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.465+0000 7f1fc2600640 1 -- 172.21.15.62:0/1821322626 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8011b60 con 0x55b377b5b8f0 2024-04-02T14:45:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.466+0000 7f1fdad84740 1 -- 172.21.15.62:0/1821322626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 msgr2=0x55b377b5bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.466+0000 7f1fdad84740 1 --2- 172.21.15.62:0/1821322626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b5bcd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009f40 tx=0x7f1fc8010860 comp rx=0 tx=0).stop 2024-04-02T14:45:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.466+0000 7f1fdad84740 1 -- 172.21.15.62:0/1821322626 shutdown_connections 2024-04-02T14:45:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.466+0000 7f1fdad84740 1 --2- 172.21.15.62:0/1821322626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b64df0 0x55b377b671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.466+0000 7f1fdad84740 1 --2- 172.21.15.62:0/1821322626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b5c210 0x55b377b648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.466+0000 7f1fdad84740 1 --2- 172.21.15.62:0/1821322626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b5bcd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.466+0000 7f1fdad84740 1 -- 172.21.15.62:0/1821322626 >> 172.21.15.62:0/1821322626 conn(0x55b377b56020 msgr2=0x55b377b56400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.467+0000 7f1fdad84740 1 -- 172.21.15.62:0/1821322626 shutdown_connections 2024-04-02T14:45:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.467+0000 7f1fdad84740 1 -- 172.21.15.62:0/1821322626 wait complete. 2024-04-02T14:45:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.467+0000 7f1fce000640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1fb80a3f20 0x7f1fb80a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.467+0000 7f1fce000640 1 -- 172.21.15.62:0/3123454167 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f1fc40f2880 con 0x7f1fd0102ab0 2024-04-02T14:45:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.467+0000 7f1fdad84740 1 -- start start 2024-04-02T14:45:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.467+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1fb80a3f20 0x7f1fb80a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.468+0000 7f1fdad84740 1 Processor -- start 2024-04-02T14:45:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.468+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b61e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.468+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 0x55b377b623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.468+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b64df0 0x55b377b628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.468+0000 7f1fdad84740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b377b67be0 con 0x55b377b5c210 2024-04-02T14:45:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.468+0000 7f1fdad84740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b377a73660 con 0x55b377b5b8f0 2024-04-02T14:45:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.468+0000 7f1fdad84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b377b67d60 con 0x55b377b64df0 2024-04-02T14:45:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.468+0000 7f1fdad84740 10 client.0 _ll_register_callbacks cb 0x55b377a73050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.468+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1fb80a3f20 0x7f1fb80a63e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55b377b571b0 tx=0x7f1fbc005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fd5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b61e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fd5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b61e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:47486/0 (socket says 172.21.15.62:47486) 2024-04-02T14:45:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fd5600640 1 -- 172.21.15.62:0/2423700206 learned_addr learned my addr 172.21.15.62:0/2423700206 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 0x55b377b623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fd5600640 1 -- 172.21.15.62:0/2423700206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b64df0 msgr2=0x55b377b628e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T14:45:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fd5600640 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b64df0 0x55b377b628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fd5600640 1 -- 172.21.15.62:0/2423700206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 msgr2=0x55b377b623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fd5600640 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 0x55b377b623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fd5600640 1 -- 172.21.15.62:0/2423700206 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b377a82e80 con 0x55b377b5b8f0 2024-04-02T14:45:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 0x55b377b623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fd5600640 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b61e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4000c00 tx=0x7f1fc40be160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69bbdbba31bf6677 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.469+0000 7f1fb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.470+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2423700206 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc40beba0 con 0x55b377b5b8f0 2024-04-02T14:45:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.470+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2423700206 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f1fc40c6f20 con 0x55b377b5b8f0 2024-04-02T14:45:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.470+0000 7f1fdad84740 1 -- 172.21.15.62:0/2423700206 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fc8003d10 con 0x55b377b5b8f0 2024-04-02T14:45:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.470+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2423700206 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fc40c2af0 con 0x55b377b5b8f0 2024-04-02T14:45:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.470+0000 7f1fdad84740 10 client.25751 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.470+0000 7f1fdad84740 1 -- 172.21.15.62:0/2423700206 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b3777cc210 con 0x55b377b5b8f0 2024-04-02T14:45:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.470+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2423700206 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1fc8003d10 con 0x55b377b5b8f0 2024-04-02T14:45:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.471+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2423700206 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1fc40f00f0 con 0x55b377b5b8f0 2024-04-02T14:45:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.471+0000 7f1fdad84740 10 client.25751 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.471+0000 7f1fdad84740 10 client.25751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.471+0000 7f1fdad84740 20 client.25751 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.471+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b377b75070 0x55b377b77510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.471+0000 7f1fdad84740 4 client.25751 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.471+0000 7f1fdad84740 1 -- 172.21.15.62:0/2423700206 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b377a67cd0 con 0x55b377b75070 2024-04-02T14:45:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.471+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b377b75070 0x55b377b77510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.472+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b377b75070 0x55b377b77510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67be70247070cab6 server_cookie=6a49f45d116def27 in_seq=0 out_seq=0 2024-04-02T14:45:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.472+0000 7f1fb6a00640 10 client.25751 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.473+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2423700206 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b377a67cd0 con 0x55b377b75070 2024-04-02T14:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.473+0000 7f1fb6a00640 10 client.25751 handle_command_reply: tid=0 2024-04-02T14:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.473+0000 7f1fdad84740 1 client.25751 shutdown 2024-04-02T14:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fdad84740 1 -- 172.21.15.62:0/2423700206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 msgr2=0x55b377b61e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b61e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4000c00 tx=0x7f1fc40be160 comp rx=0 tx=0).stop 2024-04-02T14:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fdad84740 1 -- 172.21.15.62:0/2423700206 shutdown_connections 2024-04-02T14:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b377b75070 0x55b377b77510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b64df0 0x55b377b628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 0x55b377b623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2423700206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5b8f0 0x55b377b61e60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fb6a00640 0 client.25751 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fdad84740 1 -- 172.21.15.62:0/2423700206 >> 172.21.15.62:0/2423700206 conn(0x55b377b56020 msgr2=0x55b377b57d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fdad84740 1 -- 172.21.15.62:0/2423700206 shutdown_connections 2024-04-02T14:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fdad84740 1 -- 172.21.15.62:0/2423700206 wait complete. 2024-04-02T14:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.474+0000 7f1fdad84740 20 client.25751 trim_cache size 0 max 16384 2024-04-02T14:45:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fdad84740 1 Processor -- start 2024-04-02T14:45:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fdad84740 1 -- start start 2024-04-02T14:45:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b5b8f0 0x55b377b64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 0x55b377b70aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b64df0 0x55b377b7cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fdad84740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01ab380 con 0x55b377b5c210 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fdad84740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01acec0 con 0x55b377b64df0 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fdad84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01aba50 con 0x55b377b5b8f0 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fd4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 0x55b377b70aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fd6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b64df0 0x55b377b7cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fd4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 0x55b377b70aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57500/0 (socket says 172.21.15.62:57500) 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.505+0000 7f1fd4c00640 1 -- 172.21.15.62:0/3848252985 learned_addr learned my addr 172.21.15.62:0/3848252985 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.506+0000 7f1fd5600640 1 --2- 172.21.15.62:0/3848252985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b5b8f0 0x55b377b64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.506+0000 7f1fd4c00640 1 -- 172.21.15.62:0/3848252985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b5b8f0 msgr2=0x55b377b64590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.506+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/3848252985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b5b8f0 0x55b377b64590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.506+0000 7f1fd4c00640 1 -- 172.21.15.62:0/3848252985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b64df0 msgr2=0x55b377b7cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.506+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/3848252985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b64df0 0x55b377b7cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.506+0000 7f1fd4c00640 1 -- 172.21.15.62:0/3848252985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b377898620 con 0x55b377b5c210 2024-04-02T14:45:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.506+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/3848252985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 0x55b377b70aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55b377b616c0 tx=0x7f1fbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ea0b3ac56935cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.507+0000 7f1fc1c00640 1 -- 172.21.15.62:0/3848252985 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0042e0 con 0x55b377b5c210 2024-04-02T14:45:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.507+0000 7f1fc1c00640 1 -- 172.21.15.62:0/3848252985 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1fbc031070 con 0x55b377b5c210 2024-04-02T14:45:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.507+0000 7f1fc1c00640 1 -- 172.21.15.62:0/3848252985 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc01fbf0 con 0x55b377b5c210 2024-04-02T14:45:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.508+0000 7f1fdad84740 1 -- 172.21.15.62:0/3848252985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 msgr2=0x55b377b70aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.508+0000 7f1fdad84740 1 --2- 172.21.15.62:0/3848252985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 0x55b377b70aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55b377b616c0 tx=0x7f1fbc017000 comp rx=0 tx=0).stop 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.508+0000 7f1fdad84740 1 -- 172.21.15.62:0/3848252985 shutdown_connections 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.508+0000 7f1fdad84740 1 --2- 172.21.15.62:0/3848252985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b64df0 0x55b377b7cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.508+0000 7f1fdad84740 1 --2- 172.21.15.62:0/3848252985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5c210 0x55b377b70aa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.508+0000 7f1fdad84740 1 --2- 172.21.15.62:0/3848252985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b5b8f0 0x55b377b64590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.508+0000 7f1fdad84740 1 -- 172.21.15.62:0/3848252985 >> 172.21.15.62:0/3848252985 conn(0x55b377b56020 msgr2=0x55b377b4c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.508+0000 7f1fdad84740 1 -- 172.21.15.62:0/3848252985 shutdown_connections 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.508+0000 7f1fdad84740 1 -- 172.21.15.62:0/3848252985 wait complete. 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.509+0000 7f1fdad84740 1 -- start start 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.509+0000 7f1fdad84740 1 Processor -- start 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.509+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5b8f0 0x55b377b6d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.509+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5c210 0x55b377b68830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.509+0000 7f1fdad84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b64df0 0x55b377b68e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.509+0000 7f1fdad84740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01ac7f0 con 0x55b377b5b8f0 2024-04-02T14:45:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.509+0000 7f1fdad84740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01ac120 con 0x55b377b5c210 2024-04-02T14:45:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.509+0000 7f1fdad84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01a93a0 con 0x55b377b64df0 2024-04-02T14:45:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fdad84740 10 client.0 _ll_register_callbacks cb 0x7f1fc40be5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fd5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5b8f0 0x55b377b6d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fd5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5b8f0 0x55b377b6d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57502/0 (socket says 172.21.15.62:57502) 2024-04-02T14:45:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fd5600640 1 -- 172.21.15.62:0/2903628541 learned_addr learned my addr 172.21.15.62:0/2903628541 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5c210 0x55b377b68830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fd6000640 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b64df0 0x55b377b68e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fd5600640 1 -- 172.21.15.62:0/2903628541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b64df0 msgr2=0x55b377b68e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fd5600640 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b64df0 0x55b377b68e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fd5600640 1 -- 172.21.15.62:0/2903628541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5c210 msgr2=0x55b377b68830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fd5600640 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5c210 0x55b377b68830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fd5600640 1 -- 172.21.15.62:0/2903628541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b377b69640 con 0x55b377b5b8f0 2024-04-02T14:45:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.510+0000 7f1fd6000640 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b64df0 0x55b377b68e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.511+0000 7f1fd5600640 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5b8f0 0x55b377b6d470 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1fc40bc9f0 tx=0x7f1fc40be1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d842b71f27017bed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.511+0000 7f1fb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:45:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.511+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2903628541 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc40be9a0 con 0x55b377b5b8f0 2024-04-02T14:45:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.511+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2903628541 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f1fc40beb40 con 0x55b377b5b8f0 2024-04-02T14:45:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.511+0000 7f1fdad84740 1 -- 172.21.15.62:0/2903628541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fbc031070 con 0x55b377b5b8f0 2024-04-02T14:45:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.511+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2903628541 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fc40cf070 con 0x55b377b5b8f0 2024-04-02T14:45:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.511+0000 7f1fdad84740 10 client.15633 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.511+0000 7f1fdad84740 1 -- 172.21.15.62:0/2903628541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b377bdb760 con 0x55b377b5b8f0 2024-04-02T14:45:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.512+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2903628541 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1fbc031070 con 0x55b377b5b8f0 2024-04-02T14:45:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.512+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2903628541 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1fc40bfa70 con 0x55b377b5b8f0 2024-04-02T14:45:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.512+0000 7f1fdad84740 10 client.15633 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.512+0000 7f1fdad84740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.512+0000 7f1fdad84740 20 client.15633 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.512+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b377bdbc30 0x55b377bde020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.512+0000 7f1fdad84740 4 client.15633 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:45:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.512+0000 7f1fdad84740 1 -- 172.21.15.62:0/2903628541 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b377ac52b0 con 0x55b377bdbc30 2024-04-02T14:45:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.512+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b377bdbc30 0x55b377bde020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.513+0000 7f1fd4c00640 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b377bdbc30 0x55b377bde020 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb4b442307a35e9b server_cookie=9974cb799fe4a838 in_seq=0 out_seq=0 2024-04-02T14:45:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.513+0000 7f1fb6a00640 10 client.15633 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fb6a00640 1 -- 172.21.15.62:0/2903628541 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b377ac52b0 con 0x55b377bdbc30 2024-04-02T14:45:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fb6a00640 10 client.15633 handle_command_reply: tid=0 2024-04-02T14:45:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fdad84740 1 client.15633 shutdown 2024-04-02T14:45:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fdad84740 1 -- 172.21.15.62:0/2903628541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5b8f0 msgr2=0x55b377b6d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5b8f0 0x55b377b6d470 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1fc40bc9f0 tx=0x7f1fc40be1d0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fdad84740 1 -- 172.21.15.62:0/2903628541 shutdown_connections 2024-04-02T14:45:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b377bdbc30 0x55b377bde020 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b377b64df0 0x55b377b68e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b377b5c210 0x55b377b68830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fdad84740 1 --2- 172.21.15.62:0/2903628541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b377b5b8f0 0x55b377b6d470 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fb6a00640 0 client.15633 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.519+0000 7f1fdad84740 1 -- 172.21.15.62:0/2903628541 >> 172.21.15.62:0/2903628541 conn(0x55b377b56020 msgr2=0x55b377b4bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.520+0000 7f1fdad84740 1 -- 172.21.15.62:0/2903628541 shutdown_connections 2024-04-02T14:45:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.520+0000 7f1fdad84740 1 -- 172.21.15.62:0/2903628541 wait complete. 2024-04-02T14:45:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.520+0000 7f1fdad84740 20 client.15633 trim_cache size 0 max 16384 2024-04-02T14:45:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:45:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:45:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:45:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 -- 172.21.15.62:0/3123454167 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1fb80a3f20 msgr2=0x7f1fb80a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1fb80a3f20 0x7f1fb80a63e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55b377b571b0 tx=0x7f1fbc005f90 comp rx=0 tx=0).stop 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 -- 172.21.15.62:0/3123454167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fd0102ab0 msgr2=0x7f1fd019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fd0102ab0 0x7f1fd019a510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4005940 tx=0x7f1fc4004970 comp rx=0 tx=0).stop 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 -- 172.21.15.62:0/3123454167 shutdown_connections 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1fb80a3f20 0x7f1fb80a63e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd010bed0 0x7f1fd019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd01033d0 0x7f1fd019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 --2- 172.21.15.62:0/3123454167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fd0102ab0 0x7f1fd019a510 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 -- 172.21.15.62:0/3123454167 >> 172.21.15.62:0/3123454167 conn(0x7f1fd00fc310 msgr2=0x7f1fd0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 -- 172.21.15.62:0/3123454167 shutdown_connections 2024-04-02T14:45:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:05.521+0000 7f1fd6a00640 1 -- 172.21.15.62:0/3123454167 wait complete. 2024-04-02T14:45:05.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:05 smithi204 ceph-mon[35948]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-02T14:45:05.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:05 smithi062 ceph-mon[32116]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-02T14:45:05.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:05 smithi113 ceph-mon[36135]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-02T14:45:07.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:07 smithi204 ceph-mon[35948]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-02T14:45:07.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:07 smithi062 ceph-mon[32116]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-02T14:45:07.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:07 smithi113 ceph-mon[36135]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-02T14:45:09.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:09 smithi204 ceph-mon[35948]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-02T14:45:09.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:09 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:45:09.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:09 smithi062 ceph-mon[32116]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-02T14:45:09.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:09 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:45:09.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:09 smithi113 ceph-mon[36135]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-02T14:45:09.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:09 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:45:10.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:10 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:45:10.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:10 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:45:10.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:10 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:45:10.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:10 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:45:10.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:10 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:45:10.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:10 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:45:10.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:10 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:45:10.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:10 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:45:10.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:10 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:45:11.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:11 smithi204 ceph-mon[35948]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-02T14:45:11.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:11 smithi062 ceph-mon[32116]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-02T14:45:11.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:11 smithi113 ceph-mon[36135]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-02T14:45:13.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:13 smithi204 ceph-mon[35948]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-02T14:45:13.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:13 smithi062 ceph-mon[32116]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-02T14:45:13.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:13 smithi113 ceph-mon[36135]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-02T14:45:14.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:45:14.521 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:45:14.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.741+0000 7fc522800640 1 -- 172.21.15.62:0/3968138744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc51c101240 msgr2=0x7fc51c105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.741+0000 7fc522800640 1 --2- 172.21.15.62:0/3968138744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc51c101240 0x7fc51c105700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc508009860 tx=0x7fc50802bc10 comp rx=0 tx=0).stop 2024-04-02T14:45:14.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.741+0000 7fc522800640 1 -- 172.21.15.62:0/3968138744 shutdown_connections 2024-04-02T14:45:14.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.741+0000 7fc522800640 1 --2- 172.21.15.62:0/3968138744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c105d20 0x7fc51c1061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.741+0000 7fc522800640 1 --2- 172.21.15.62:0/3968138744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc51c101240 0x7fc51c105700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.741+0000 7fc522800640 1 --2- 172.21.15.62:0/3968138744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c100920 0x7fc51c100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.741+0000 7fc522800640 1 -- 172.21.15.62:0/3968138744 >> 172.21.15.62:0/3968138744 conn(0x7fc51c0fc350 msgr2=0x7fc51c0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:14.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.741+0000 7fc522800640 1 -- 172.21.15.62:0/3968138744 shutdown_connections 2024-04-02T14:45:14.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.742+0000 7fc522800640 1 -- 172.21.15.62:0/3968138744 wait complete. 2024-04-02T14:45:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.742+0000 7fc522800640 1 Processor -- start 2024-04-02T14:45:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.743+0000 7fc522800640 1 -- start start 2024-04-02T14:45:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.743+0000 7fc522800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c100920 0x7fc51c1a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.743+0000 7fc522800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c101240 0x7fc51c1a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.743+0000 7fc522800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc51c105d20 0x7fc51c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.743+0000 7fc522800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c118470 con 0x7fc51c101240 2024-04-02T14:45:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.743+0000 7fc522800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1182f0 con 0x7fc51c105d20 2024-04-02T14:45:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.743+0000 7fc522800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1185f0 con 0x7fc51c100920 2024-04-02T14:45:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.744+0000 7fc520a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c101240 0x7fc51c1a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.744+0000 7fc520a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c101240 0x7fc51c1a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59908/0 (socket says 172.21.15.62:59908) 2024-04-02T14:45:14.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.744+0000 7fc520a00640 1 -- 172.21.15.62:0/182599080 learned_addr learned my addr 172.21.15.62:0/182599080 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:14.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.744+0000 7fc521e00640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc51c105d20 0x7fc51c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.744+0000 7fc521400640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c100920 0x7fc51c1a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.744+0000 7fc520a00640 1 -- 172.21.15.62:0/182599080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c100920 msgr2=0x7fc51c1a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.744+0000 7fc520a00640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c100920 0x7fc51c1a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.744+0000 7fc520a00640 1 -- 172.21.15.62:0/182599080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc51c105d20 msgr2=0x7fc51c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.744+0000 7fc520a00640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc51c105d20 0x7fc51c19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.744+0000 7fc520a00640 1 -- 172.21.15.62:0/182599080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c19efc0 con 0x7fc51c101240 2024-04-02T14:45:14.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.744+0000 7fc520a00640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c101240 0x7fc51c1a4bc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc508037040 tx=0x7fc50802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:14.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.745+0000 7fc51a000640 1 -- 172.21.15.62:0/182599080 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508039040 con 0x7fc51c101240 2024-04-02T14:45:14.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.745+0000 7fc51a000640 1 -- 172.21.15.62:0/182599080 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5080044d0 con 0x7fc51c101240 2024-04-02T14:45:14.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.745+0000 7fc51a000640 1 -- 172.21.15.62:0/182599080 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50802ce60 con 0x7fc51c101240 2024-04-02T14:45:14.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.745+0000 7fc522800640 1 -- 172.21.15.62:0/182599080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51c19f250 con 0x7fc51c101240 2024-04-02T14:45:14.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.745+0000 7fc522800640 1 -- 172.21.15.62:0/182599080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51c1ab4d0 con 0x7fc51c101240 2024-04-02T14:45:14.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.749+0000 7fc526a7a740 1 Processor -- start 2024-04-02T14:45:14.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.749+0000 7fc526a7a740 1 -- start start 2024-04-02T14:45:14.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.749+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf438d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.749+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf441920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.750+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf444250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.750+0000 7fc526a7a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5645bf39c770 con 0x5645bf438960 2024-04-02T14:45:14.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.750+0000 7fc526a7a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5645bf344700 con 0x5645bf441e60 2024-04-02T14:45:14.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.750+0000 7fc526a7a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5645bf398720 con 0x5645bf439280 2024-04-02T14:45:14.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.750+0000 7fc51a000640 1 -- 172.21.15.62:0/182599080 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7fc508005030 con 0x7fc51c101240 2024-04-02T14:45:14.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.750+0000 7fc521e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf438d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.750+0000 7fc521400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf441920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.750+0000 7fc521400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf441920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55334/0 (socket says 172.21.15.62:55334) 2024-04-02T14:45:14.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.750+0000 7fc521e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf438d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59918/0 (socket says 172.21.15.62:59918) 2024-04-02T14:45:14.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.750+0000 7fc521400640 1 -- 172.21.15.62:0/964227207 learned_addr learned my addr 172.21.15.62:0/964227207 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:14.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.751+0000 7fc520a00640 1 --2- 172.21.15.62:0/964227207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf444250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.751+0000 7fc521400640 1 -- 172.21.15.62:0/964227207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 msgr2=0x5645bf444250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.751+0000 7fc521400640 1 --2- 172.21.15.62:0/964227207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf444250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.751+0000 7fc521400640 1 -- 172.21.15.62:0/964227207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 msgr2=0x5645bf438d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.751+0000 7fc521400640 1 --2- 172.21.15.62:0/964227207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf438d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.751+0000 7fc521400640 1 -- 172.21.15.62:0/964227207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645bf17a7d0 con 0x5645bf439280 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.751+0000 7fc520a00640 1 --2- 172.21.15.62:0/964227207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf444250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.751+0000 7fc521e00640 1 --2- 172.21.15.62:0/964227207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf438d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.751+0000 7fc521400640 1 --2- 172.21.15.62:0/964227207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf441920 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc510009cc0 tx=0x7fc510011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=419be72dbe5ea74 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.751+0000 7fc50e600640 1 -- 172.21.15.62:0/964227207 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510005220 con 0x5645bf439280 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.752+0000 7fc50e600640 1 -- 172.21.15.62:0/964227207 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc510004ca0 con 0x5645bf439280 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.752+0000 7fc50e600640 1 -- 172.21.15.62:0/964227207 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510019910 con 0x5645bf439280 2024-04-02T14:45:14.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.752+0000 7fc51a000640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5040a3f50 0x7fc5040a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.752+0000 7fc51a000640 1 -- 172.21.15.62:0/182599080 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc5080f1af0 con 0x7fc51c101240 2024-04-02T14:45:14.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.752+0000 7fc526a7a740 1 -- 172.21.15.62:0/964227207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 msgr2=0x5645bf441920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.752+0000 7fc526a7a740 1 --2- 172.21.15.62:0/964227207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf441920 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc510009cc0 tx=0x7fc510011810 comp rx=0 tx=0).stop 2024-04-02T14:45:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.753+0000 7fc526a7a740 1 -- 172.21.15.62:0/964227207 shutdown_connections 2024-04-02T14:45:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.753+0000 7fc521e00640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5040a3f50 0x7fc5040a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.753+0000 7fc526a7a740 1 --2- 172.21.15.62:0/964227207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf444250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.753+0000 7fc526a7a740 1 --2- 172.21.15.62:0/964227207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf441920 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.753+0000 7fc526a7a740 1 --2- 172.21.15.62:0/964227207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf438d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.753+0000 7fc526a7a740 1 -- 172.21.15.62:0/964227207 >> 172.21.15.62:0/964227207 conn(0x5645bf433090 msgr2=0x5645bf433470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.753+0000 7fc526a7a740 1 -- 172.21.15.62:0/964227207 shutdown_connections 2024-04-02T14:45:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.753+0000 7fc526a7a740 1 -- 172.21.15.62:0/964227207 wait complete. 2024-04-02T14:45:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.754+0000 7fc526a7a740 1 -- start start 2024-04-02T14:45:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.754+0000 7fc526a7a740 1 Processor -- start 2024-04-02T14:45:14.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.754+0000 7fc521e00640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5040a3f50 0x7fc5040a6410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc5140099a0 tx=0x7fc514009500 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.754+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf438960 0x5645bf43eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf439280 0x5645bf43f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf43f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc526a7a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5645bf444c50 con 0x5645bf439280 2024-04-02T14:45:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc526a7a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5645bf357ac0 con 0x5645bf441e60 2024-04-02T14:45:14.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc526a7a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5645bf444dd0 con 0x5645bf438960 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc526a7a740 10 client.0 _ll_register_callbacks cb 0x5645bf35a180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc520a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf439280 0x5645bf43f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc521400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf438960 0x5645bf43eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc521400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf438960 0x5645bf43eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55342/0 (socket says 172.21.15.62:55342) 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc521400640 1 -- 172.21.15.62:0/1891546687 learned_addr learned my addr 172.21.15.62:0/1891546687 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc521e00640 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf43f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc521400640 1 -- 172.21.15.62:0/1891546687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 msgr2=0x5645bf43f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc521400640 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf43f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc521400640 1 -- 172.21.15.62:0/1891546687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf439280 msgr2=0x5645bf43f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc521400640 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf439280 0x5645bf43f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.755+0000 7fc521400640 1 -- 172.21.15.62:0/1891546687 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645bf0b03f0 con 0x5645bf438960 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.756+0000 7fc521400640 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf438960 0x5645bf43eee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc51001b040 tx=0x7fc510019430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fae4a36885b98932 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:14.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.756+0000 7fc502a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:45:14.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.756+0000 7fc502a00640 1 -- 172.21.15.62:0/1891546687 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510029070 con 0x5645bf438960 2024-04-02T14:45:14.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.756+0000 7fc526a7a740 1 -- 172.21.15.62:0/1891546687 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc510004ca0 con 0x5645bf438960 2024-04-02T14:45:14.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.756+0000 7fc502a00640 1 -- 172.21.15.62:0/1891546687 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc510019860 con 0x5645bf438960 2024-04-02T14:45:14.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.757+0000 7fc502a00640 1 -- 172.21.15.62:0/1891546687 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc510035050 con 0x5645bf438960 2024-04-02T14:45:14.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.757+0000 7fc526a7a740 10 client.25744 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:14.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.757+0000 7fc526a7a740 1 -- 172.21.15.62:0/1891546687 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645bf0700f0 con 0x5645bf438960 2024-04-02T14:45:14.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.757+0000 7fc502a00640 1 -- 172.21.15.62:0/1891546687 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc510004ca0 con 0x5645bf438960 2024-04-02T14:45:14.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.757+0000 7fc502a00640 1 -- 172.21.15.62:0/1891546687 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc510023790 con 0x5645bf438960 2024-04-02T14:45:14.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.758+0000 7fc526a7a740 10 client.25744 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:14.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.758+0000 7fc526a7a740 10 client.25744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:14.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.758+0000 7fc526a7a740 20 client.25744 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:14.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.758+0000 7fc526a7a740 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5645bf451d50 0x5645bf454140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.758+0000 7fc526a7a740 4 client.25744 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:14.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.758+0000 7fc526a7a740 1 -- 172.21.15.62:0/1891546687 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645bf33d630 con 0x5645bf451d50 2024-04-02T14:45:14.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.758+0000 7fc520a00640 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5645bf451d50 0x5645bf454140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.759+0000 7fc520a00640 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5645bf451d50 0x5645bf454140 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e4ca7db3c0e9b27 server_cookie=4549e93cd9b8aea8 in_seq=0 out_seq=0 2024-04-02T14:45:14.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.759+0000 7fc502a00640 10 client.25744 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.759+0000 7fc502a00640 1 -- 172.21.15.62:0/1891546687 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645bf33d630 con 0x5645bf451d50 2024-04-02T14:45:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.759+0000 7fc502a00640 10 client.25744 handle_command_reply: tid=0 2024-04-02T14:45:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.759+0000 7fc526a7a740 1 client.25744 shutdown 2024-04-02T14:45:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc526a7a740 1 -- 172.21.15.62:0/1891546687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf438960 msgr2=0x5645bf43eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc526a7a740 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf438960 0x5645bf43eee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc51001b040 tx=0x7fc510019430 comp rx=0 tx=0).stop 2024-04-02T14:45:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc526a7a740 1 -- 172.21.15.62:0/1891546687 shutdown_connections 2024-04-02T14:45:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc526a7a740 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5645bf451d50 0x5645bf454140 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc526a7a740 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf43f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc526a7a740 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf439280 0x5645bf43f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc526a7a740 1 --2- 172.21.15.62:0/1891546687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf438960 0x5645bf43eee0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc526a7a740 1 -- 172.21.15.62:0/1891546687 >> 172.21.15.62:0/1891546687 conn(0x5645bf433090 msgr2=0x5645bf434de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc502a00640 0 client.25744 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc526a7a740 1 -- 172.21.15.62:0/1891546687 shutdown_connections 2024-04-02T14:45:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc526a7a740 1 -- 172.21.15.62:0/1891546687 wait complete. 2024-04-02T14:45:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.760+0000 7fc526a7a740 20 client.25744 trim_cache size 0 max 16384 2024-04-02T14:45:14.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.779+0000 7fc526a7a740 1 Processor -- start 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.779+0000 7fc526a7a740 1 -- start start 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.779+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf441610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.779+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf433470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf44db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc526a7a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1b3c50 con 0x5645bf438960 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc526a7a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1b5790 con 0x5645bf441e60 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc526a7a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1b4320 con 0x5645bf439280 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc521400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf441610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc521400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf441610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59946/0 (socket says 172.21.15.62:59946) 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc521400640 1 -- 172.21.15.62:0/786777933 learned_addr learned my addr 172.21.15.62:0/786777933 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc520a00640 1 --2- 172.21.15.62:0/786777933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf433470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc521e00640 1 --2- 172.21.15.62:0/786777933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf44db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc521400640 1 -- 172.21.15.62:0/786777933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 msgr2=0x5645bf433470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc521400640 1 --2- 172.21.15.62:0/786777933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf433470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.780+0000 7fc521400640 1 -- 172.21.15.62:0/786777933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 msgr2=0x5645bf44db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.781+0000 7fc521400640 1 --2- 172.21.15.62:0/786777933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf44db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.781+0000 7fc521400640 1 -- 172.21.15.62:0/786777933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645bf444b30 con 0x5645bf438960 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.781+0000 7fc521400640 1 --2- 172.21.15.62:0/786777933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf441610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc510011850 tx=0x7fc510005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894ecd3903cdd5dd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.781+0000 7fc50dc00640 1 -- 172.21.15.62:0/786777933 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510032030 con 0x5645bf438960 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.781+0000 7fc50dc00640 1 -- 172.21.15.62:0/786777933 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc510023d80 con 0x5645bf438960 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.781+0000 7fc50dc00640 1 -- 172.21.15.62:0/786777933 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510029070 con 0x5645bf438960 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.782+0000 7fc526a7a740 1 -- 172.21.15.62:0/786777933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 msgr2=0x5645bf441610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.782+0000 7fc526a7a740 1 --2- 172.21.15.62:0/786777933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf441610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc510011850 tx=0x7fc510005c50 comp rx=0 tx=0).stop 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.782+0000 7fc526a7a740 1 -- 172.21.15.62:0/786777933 shutdown_connections 2024-04-02T14:45:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.782+0000 7fc526a7a740 1 --2- 172.21.15.62:0/786777933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf44db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.782+0000 7fc526a7a740 1 --2- 172.21.15.62:0/786777933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf433470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.782+0000 7fc526a7a740 1 --2- 172.21.15.62:0/786777933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf441610 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.782+0000 7fc526a7a740 1 -- 172.21.15.62:0/786777933 >> 172.21.15.62:0/786777933 conn(0x5645bf433090 msgr2=0x5645bf43d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:14.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.782+0000 7fc526a7a740 1 -- 172.21.15.62:0/786777933 shutdown_connections 2024-04-02T14:45:14.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.782+0000 7fc526a7a740 1 -- 172.21.15.62:0/786777933 wait complete. 2024-04-02T14:45:14.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.783+0000 7fc526a7a740 1 -- start start 2024-04-02T14:45:14.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.783+0000 7fc526a7a740 1 Processor -- start 2024-04-02T14:45:14.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.784+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf44a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.784+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf44a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.784+0000 7fc526a7a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf43b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.784+0000 7fc526a7a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1b50c0 con 0x5645bf438960 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.784+0000 7fc526a7a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1b49f0 con 0x5645bf441e60 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.784+0000 7fc526a7a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1b1c30 con 0x5645bf439280 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.784+0000 7fc526a7a740 10 client.0 _ll_register_callbacks cb 0x7fc510009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.784+0000 7fc521400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf44a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.784+0000 7fc521400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf44a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59948/0 (socket says 172.21.15.62:59948) 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.784+0000 7fc521400640 1 -- 172.21.15.62:0/105648952 learned_addr learned my addr 172.21.15.62:0/105648952 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.785+0000 7fc520a00640 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf44a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.785+0000 7fc521e00640 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf43b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.785+0000 7fc521400640 1 -- 172.21.15.62:0/105648952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 msgr2=0x5645bf44a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.785+0000 7fc521400640 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf44a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.785+0000 7fc521400640 1 -- 172.21.15.62:0/105648952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 msgr2=0x5645bf43b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.785+0000 7fc521400640 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf43b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.785+0000 7fc521400640 1 -- 172.21.15.62:0/105648952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645bf43b7a0 con 0x5645bf438960 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.785+0000 7fc521400640 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf44a020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc510019620 tx=0x7fc510019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd1738458abf5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.785+0000 7fc502a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.786+0000 7fc502a00640 1 -- 172.21.15.62:0/105648952 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510032070 con 0x5645bf438960 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.786+0000 7fc502a00640 1 -- 172.21.15.62:0/105648952 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc510004b70 con 0x5645bf438960 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.786+0000 7fc526a7a740 1 -- 172.21.15.62:0/105648952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc510023d80 con 0x5645bf438960 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.786+0000 7fc502a00640 1 -- 172.21.15.62:0/105648952 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc510029070 con 0x5645bf438960 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.786+0000 7fc526a7a740 10 client.15651 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.786+0000 7fc526a7a740 1 -- 172.21.15.62:0/105648952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645bf4b7a50 con 0x5645bf438960 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.786+0000 7fc502a00640 1 -- 172.21.15.62:0/105648952 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc510023d80 con 0x5645bf438960 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.786+0000 7fc502a00640 1 -- 172.21.15.62:0/105648952 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc510019ac0 con 0x5645bf438960 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.787+0000 7fc526a7a740 10 client.15651 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.787+0000 7fc526a7a740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:14.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.787+0000 7fc526a7a740 20 client.15651 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:14.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.787+0000 7fc526a7a740 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5645bf4b8100 0x5645bf4ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.787+0000 7fc526a7a740 4 client.15651 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:45:14.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.787+0000 7fc526a7a740 1 -- 172.21.15.62:0/105648952 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5645bf3a9cb0 con 0x5645bf4b8100 2024-04-02T14:45:14.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.787+0000 7fc520a00640 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5645bf4b8100 0x5645bf4ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc520a00640 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5645bf4b8100 0x5645bf4ba4f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f480c77e1352f15b server_cookie=846510217c7a2188 in_seq=0 out_seq=0 2024-04-02T14:45:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc502a00640 10 client.15651 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc502a00640 1 -- 172.21.15.62:0/105648952 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5645bf3a9cb0 con 0x5645bf4b8100 2024-04-02T14:45:14.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc502a00640 10 client.15651 handle_command_reply: tid=0 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc526a7a740 1 client.15651 shutdown 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc526a7a740 1 -- 172.21.15.62:0/105648952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 msgr2=0x5645bf44a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc526a7a740 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf44a020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc510019620 tx=0x7fc510019570 comp rx=0 tx=0).stop 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc526a7a740 1 -- 172.21.15.62:0/105648952 shutdown_connections 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc526a7a740 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5645bf4b8100 0x5645bf4ba4f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc526a7a740 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645bf441e60 0x5645bf43b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc526a7a740 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5645bf439280 0x5645bf44a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc526a7a740 1 --2- 172.21.15.62:0/105648952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645bf438960 0x5645bf44a020 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc526a7a740 1 -- 172.21.15.62:0/105648952 >> 172.21.15.62:0/105648952 conn(0x5645bf433090 msgr2=0x5645bf431a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.788+0000 7fc502a00640 0 client.15651 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.789+0000 7fc526a7a740 1 -- 172.21.15.62:0/105648952 shutdown_connections 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.789+0000 7fc526a7a740 1 -- 172.21.15.62:0/105648952 wait complete. 2024-04-02T14:45:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.789+0000 7fc526a7a740 20 client.15651 trim_cache size 0 max 16384 2024-04-02T14:45:14.788 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:45:14.788 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-02T14:45:14.788 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-02T14:45:14.788 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.789+0000 7fc522800640 1 -- 172.21.15.62:0/182599080 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5040a3f50 msgr2=0x7fc5040a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.789+0000 7fc522800640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5040a3f50 0x7fc5040a6410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc5140099a0 tx=0x7fc514009500 comp rx=0 tx=0).stop 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.789+0000 7fc522800640 1 -- 172.21.15.62:0/182599080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c101240 msgr2=0x7fc51c1a4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.789+0000 7fc522800640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c101240 0x7fc51c1a4bc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc508037040 tx=0x7fc50802c860 comp rx=0 tx=0).stop 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.790+0000 7fc522800640 1 -- 172.21.15.62:0/182599080 shutdown_connections 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.790+0000 7fc522800640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc5040a3f50 0x7fc5040a6410 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.790+0000 7fc522800640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc51c105d20 0x7fc51c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.790+0000 7fc522800640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c101240 0x7fc51c1a4bc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.790+0000 7fc522800640 1 --2- 172.21.15.62:0/182599080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c100920 0x7fc51c1a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.790+0000 7fc522800640 1 -- 172.21.15.62:0/182599080 >> 172.21.15.62:0/182599080 conn(0x7fc51c0fc350 msgr2=0x7fc51c1036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.790+0000 7fc522800640 1 -- 172.21.15.62:0/182599080 shutdown_connections 2024-04-02T14:45:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.790+0000 7fc522800640 1 -- 172.21.15.62:0/182599080 wait complete. 2024-04-02T14:45:14.805 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:45:14.805 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:45:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.985+0000 7f16ed000640 1 -- 172.21.15.62:0/524706146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e8069ea0 msgr2=0x7f16e806a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.985+0000 7f16ed000640 1 --2- 172.21.15.62:0/524706146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e8069ea0 0x7f16e806a280 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009860 tx=0x7f16d802bc00 comp rx=0 tx=0).stop 2024-04-02T14:45:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.985+0000 7f16ed000640 1 -- 172.21.15.62:0/524706146 shutdown_connections 2024-04-02T14:45:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.985+0000 7f16ed000640 1 --2- 172.21.15.62:0/524706146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16e810e510 0x7f16e8110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.985+0000 7f16ed000640 1 --2- 172.21.15.62:0/524706146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16e806a7c0 0x7f16e810dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.985+0000 7f16ed000640 1 --2- 172.21.15.62:0/524706146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e8069ea0 0x7f16e806a280 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.985+0000 7f16ed000640 1 -- 172.21.15.62:0/524706146 >> 172.21.15.62:0/524706146 conn(0x7f16e8077b30 msgr2=0x7f16e8077f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:14.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.985+0000 7f16ed000640 1 -- 172.21.15.62:0/524706146 shutdown_connections 2024-04-02T14:45:14.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.985+0000 7f16ed000640 1 -- 172.21.15.62:0/524706146 wait complete. 2024-04-02T14:45:14.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.987+0000 7f16ed000640 1 Processor -- start 2024-04-02T14:45:14.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.987+0000 7f16ed000640 1 -- start start 2024-04-02T14:45:14.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.987+0000 7f16ed000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16e8069ea0 0x7f16e819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.987+0000 7f16ed000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e806a7c0 0x7f16e819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.987+0000 7f16ed000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16e810e510 0x7f16e819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.987+0000 7f16ed000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81141f0 con 0x7f16e8069ea0 2024-04-02T14:45:14.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.987+0000 7f16ed000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8114070 con 0x7f16e810e510 2024-04-02T14:45:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.987+0000 7f16ed000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8114370 con 0x7f16e806a7c0 2024-04-02T14:45:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.988+0000 7f16e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16e8069ea0 0x7f16e819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.988+0000 7f16e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16e8069ea0 0x7f16e819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59966/0 (socket says 172.21.15.62:59966) 2024-04-02T14:45:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.988+0000 7f16e5e00640 1 -- 172.21.15.62:0/4235722973 learned_addr learned my addr 172.21.15.62:0/4235722973 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.988+0000 7f16e5400640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e806a7c0 0x7f16e819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.988+0000 7f16e6800640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16e810e510 0x7f16e819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.988+0000 7f16e5400640 1 -- 172.21.15.62:0/4235722973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16e810e510 msgr2=0x7f16e819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.988+0000 7f16e5400640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16e810e510 0x7f16e819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.988+0000 7f16e5400640 1 -- 172.21.15.62:0/4235722973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16e8069ea0 msgr2=0x7f16e819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.988+0000 7f16e5400640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16e8069ea0 0x7f16e819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.988+0000 7f16e5400640 1 -- 172.21.15.62:0/4235722973 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e819f6d0 con 0x7f16e806a7c0 2024-04-02T14:45:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.988+0000 7f16e5e00640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16e8069ea0 0x7f16e819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.989+0000 7f16e5400640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e806a7c0 0x7f16e819ac70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16d400e9f0 tx=0x7f16d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.989+0000 7f16dea00640 1 -- 172.21.15.62:0/4235722973 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d400ccc0 con 0x7f16e806a7c0 2024-04-02T14:45:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.989+0000 7f16dea00640 1 -- 172.21.15.62:0/4235722973 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16d4010930 con 0x7f16e806a7c0 2024-04-02T14:45:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.989+0000 7f16ed000640 1 -- 172.21.15.62:0/4235722973 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e819f9c0 con 0x7f16e806a7c0 2024-04-02T14:45:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.989+0000 7f16dea00640 1 -- 172.21.15.62:0/4235722973 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d40189f0 con 0x7f16e806a7c0 2024-04-02T14:45:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.989+0000 7f16ed000640 1 -- 172.21.15.62:0/4235722973 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e81a7250 con 0x7f16e806a7c0 2024-04-02T14:45:14.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.991+0000 7f16ed000640 1 -- 172.21.15.62:0/4235722973 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e8068060 con 0x7f16e806a7c0 2024-04-02T14:45:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.992+0000 7f16dea00640 1 -- 172.21.15.62:0/4235722973 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f16d4010ae0 con 0x7f16e806a7c0 2024-04-02T14:45:14.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.994+0000 7f16dea00640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16cc0a3ed0 0x7f16cc0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.994+0000 7f16dea00640 1 -- 172.21.15.62:0/4235722973 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f16d40d0de0 con 0x7f16e806a7c0 2024-04-02T14:45:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.995+0000 7f16e5e00640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16cc0a3ed0 0x7f16cc0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:14.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.996+0000 7f16e5e00640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16cc0a3ed0 0x7f16cc0a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16d80095f0 tx=0x7f16d802e040 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:14.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:14.998+0000 7f16dea00640 1 -- 172.21.15.62:0/4235722973 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f16d409d290 con 0x7f16e806a7c0 2024-04-02T14:45:15.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.260+0000 7f16ed000640 1 -- 172.21.15.62:0/4235722973 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16e81096a0 con 0x7f16e806a7c0 2024-04-02T14:45:15.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.261+0000 7f16dea00640 1 -- 172.21.15.62:0/4235722973 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f16d409cb90 con 0x7f16e806a7c0 2024-04-02T14:45:15.258 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:45:15.258 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:44:00.377074+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584,24535,24533,24553,24565]},"id":1}]} 2024-04-02T14:45:15.259 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-04-02T14:45:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.264+0000 7f16ed000640 1 -- 172.21.15.62:0/4235722973 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16cc0a3ed0 msgr2=0x7f16cc0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.264+0000 7f16ed000640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16cc0a3ed0 0x7f16cc0a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16d80095f0 tx=0x7f16d802e040 comp rx=0 tx=0).stop 2024-04-02T14:45:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.264+0000 7f16ed000640 1 -- 172.21.15.62:0/4235722973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e806a7c0 msgr2=0x7f16e819ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.264+0000 7f16ed000640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e806a7c0 0x7f16e819ac70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16d400e9f0 tx=0x7f16d400eeb0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.264+0000 7f16ed000640 1 -- 172.21.15.62:0/4235722973 shutdown_connections 2024-04-02T14:45:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.264+0000 7f16ed000640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f16cc0a3ed0 0x7f16cc0a6390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.264+0000 7f16ed000640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16e810e510 0x7f16e819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.264+0000 7f16ed000640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e806a7c0 0x7f16e819ac70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.264+0000 7f16ed000640 1 --2- 172.21.15.62:0/4235722973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16e8069ea0 0x7f16e819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.264+0000 7f16ed000640 1 -- 172.21.15.62:0/4235722973 >> 172.21.15.62:0/4235722973 conn(0x7f16e8077b30 msgr2=0x7f16e810fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.265+0000 7f16ed000640 1 -- 172.21.15.62:0/4235722973 shutdown_connections 2024-04-02T14:45:15.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.265+0000 7f16ed000640 1 -- 172.21.15.62:0/4235722973 wait complete. 2024-04-02T14:45:15.278 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:45:15.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.490+0000 7fc8dfe00640 1 -- 172.21.15.62:0/768015354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e010e2f0 msgr2=0x7fc8e01106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.490+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/768015354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e010e2f0 0x7fc8e01106e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009be0 tx=0x7fc8d802bef0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.490+0000 7fc8dfe00640 1 -- 172.21.15.62:0/768015354 shutdown_connections 2024-04-02T14:45:15.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.490+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/768015354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e010e2f0 0x7fc8e01106e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.490+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/768015354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e01010f0 0x7fc8e010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.490+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/768015354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e0100740 0x7fc8e0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.490+0000 7fc8dfe00640 1 -- 172.21.15.62:0/768015354 >> 172.21.15.62:0/768015354 conn(0x7fc8e00fc350 msgr2=0x7fc8e00fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.490+0000 7fc8dfe00640 1 -- 172.21.15.62:0/768015354 shutdown_connections 2024-04-02T14:45:15.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.490+0000 7fc8dfe00640 1 -- 172.21.15.62:0/768015354 wait complete. 2024-04-02T14:45:15.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.491+0000 7fc8dfe00640 1 Processor -- start 2024-04-02T14:45:15.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.492+0000 7fc8dfe00640 1 -- start start 2024-04-02T14:45:15.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.492+0000 7fc8dfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0100740 0x7fc8e019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.492+0000 7fc8dfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e01010f0 0x7fc8e019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.492+0000 7fc8dfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e010e2f0 0x7fc8e019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.492+0000 7fc8dfe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01140d0 con 0x7fc8e01010f0 2024-04-02T14:45:15.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.492+0000 7fc8dfe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0113f50 con 0x7fc8e0100740 2024-04-02T14:45:15.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.492+0000 7fc8dfe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0114250 con 0x7fc8e010e2f0 2024-04-02T14:45:15.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.492+0000 7fc8dea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0100740 0x7fc8e019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.492+0000 7fc8dea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0100740 0x7fc8e019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37020/0 (socket says 172.21.15.62:37020) 2024-04-02T14:45:15.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.492+0000 7fc8dea00640 1 -- 172.21.15.62:0/974633915 learned_addr learned my addr 172.21.15.62:0/974633915 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:15.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.493+0000 7fc8de000640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e01010f0 0x7fc8e019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.493+0000 7fc8df400640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e010e2f0 0x7fc8e019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.493+0000 7fc8de000640 1 -- 172.21.15.62:0/974633915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e010e2f0 msgr2=0x7fc8e019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.493+0000 7fc8de000640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e010e2f0 0x7fc8e019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.493+0000 7fc8de000640 1 -- 172.21.15.62:0/974633915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0100740 msgr2=0x7fc8e019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.493+0000 7fc8de000640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0100740 0x7fc8e019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.493+0000 7fc8de000640 1 -- 172.21.15.62:0/974633915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e019f560 con 0x7fc8e01010f0 2024-04-02T14:45:15.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.493+0000 7fc8dea00640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0100740 0x7fc8e019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:15.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.494+0000 7fc8de000640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e01010f0 0x7fc8e019aab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00e9f0 tx=0x7fc8cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.494+0000 7fc8d7400640 1 -- 172.21.15.62:0/974633915 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc00ccc0 con 0x7fc8e01010f0 2024-04-02T14:45:15.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.494+0000 7fc8d7400640 1 -- 172.21.15.62:0/974633915 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8cc005020 con 0x7fc8e01010f0 2024-04-02T14:45:15.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.494+0000 7fc8dfe00640 1 -- 172.21.15.62:0/974633915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e019f850 con 0x7fc8e01010f0 2024-04-02T14:45:15.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.494+0000 7fc8d7400640 1 -- 172.21.15.62:0/974633915 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc004b40 con 0x7fc8e01010f0 2024-04-02T14:45:15.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.495+0000 7fc8dfe00640 1 -- 172.21.15.62:0/974633915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e01a70e0 con 0x7fc8e01010f0 2024-04-02T14:45:15.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.499+0000 7fc8e8837740 1 Processor -- start 2024-04-02T14:45:15.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.499+0000 7fc8e8837740 1 -- start start 2024-04-02T14:45:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.500+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b73498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.500+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7349ec0 0x55d4b734a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.500+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 0x55d4b7355620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.500+0000 7fc8e8837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d4b728a400 con 0x55d4b7353230 2024-04-02T14:45:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.500+0000 7fc8e8837740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d4b726ef10 con 0x55d4b734aba0 2024-04-02T14:45:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.500+0000 7fc8e8837740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d4b72ae6f0 con 0x55d4b7349ec0 2024-04-02T14:45:15.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.500+0000 7fc8dea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7349ec0 0x55d4b734a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.500+0000 7fc8df400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b73498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.500+0000 7fc8df400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b73498f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37036/0 (socket says 172.21.15.62:37036) 2024-04-02T14:45:15.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.500+0000 7fc8df400640 1 -- 172.21.15.62:0/2107167637 learned_addr learned my addr 172.21.15.62:0/2107167637 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:15.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.500+0000 7fc8d7400640 1 -- 172.21.15.62:0/974633915 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7fc8cc004ce0 con 0x7fc8e01010f0 2024-04-02T14:45:15.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.501+0000 7fc8df400640 1 -- 172.21.15.62:0/2107167637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7349ec0 msgr2=0x55d4b734a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.501+0000 7fc8df400640 1 --2- 172.21.15.62:0/2107167637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7349ec0 0x55d4b734a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.501+0000 7fc8df400640 1 -- 172.21.15.62:0/2107167637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 msgr2=0x55d4b7355620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.501+0000 7fc8de000640 1 --2- 172.21.15.62:0/2107167637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 0x55d4b7355620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.501+0000 7fc8df400640 1 --2- 172.21.15.62:0/2107167637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 0x55d4b7355620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.501+0000 7fc8df400640 1 -- 172.21.15.62:0/2107167637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4b6fc23f0 con 0x55d4b734aba0 2024-04-02T14:45:15.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.501+0000 7fc8de000640 1 --2- 172.21.15.62:0/2107167637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 0x55d4b7355620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:45:15.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.502+0000 7fc8df400640 1 --2- 172.21.15.62:0/2107167637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b73498f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0101690 tx=0x7fc8d8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=862f03845865e556 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.502+0000 7fc8cbe00640 1 -- 172.21.15.62:0/2107167637 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8043030 con 0x55d4b734aba0 2024-04-02T14:45:15.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.502+0000 7fc8cbe00640 1 -- 172.21.15.62:0/2107167637 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8d8033bf0 con 0x55d4b734aba0 2024-04-02T14:45:15.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.502+0000 7fc8cbe00640 1 -- 172.21.15.62:0/2107167637 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d802c890 con 0x55d4b734aba0 2024-04-02T14:45:15.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.503+0000 7fc8d7400640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc8c40a3f50 0x7fc8c40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.503+0000 7fc8e8837740 1 -- 172.21.15.62:0/2107167637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 msgr2=0x55d4b73498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.503+0000 7fc8e8837740 1 --2- 172.21.15.62:0/2107167637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b73498f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0101690 tx=0x7fc8d8009310 comp rx=0 tx=0).stop 2024-04-02T14:45:15.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.503+0000 7fc8d7400640 1 -- 172.21.15.62:0/974633915 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc8cc0d0930 con 0x7fc8e01010f0 2024-04-02T14:45:15.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.503+0000 7fc8e8837740 1 -- 172.21.15.62:0/2107167637 shutdown_connections 2024-04-02T14:45:15.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.503+0000 7fc8e8837740 1 --2- 172.21.15.62:0/2107167637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 0x55d4b7355620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.503+0000 7fc8e8837740 1 --2- 172.21.15.62:0/2107167637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7349ec0 0x55d4b734a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.503+0000 7fc8e8837740 1 --2- 172.21.15.62:0/2107167637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b73498f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.503+0000 7fc8e8837740 1 -- 172.21.15.62:0/2107167637 >> 172.21.15.62:0/2107167637 conn(0x55d4b7310a50 msgr2=0x55d4b7344910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.503+0000 7fc8dea00640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc8c40a3f50 0x7fc8c40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.504+0000 7fc8e8837740 1 -- 172.21.15.62:0/2107167637 shutdown_connections 2024-04-02T14:45:15.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.505+0000 7fc8dea00640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc8c40a3f50 0x7fc8c40a6410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009fd0 tx=0x7fc8d0014040 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.505+0000 7fc8e8837740 1 -- 172.21.15.62:0/2107167637 wait complete. 2024-04-02T14:45:15.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.505+0000 7fc8e8837740 1 -- start start 2024-04-02T14:45:15.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.506+0000 7fc8e8837740 1 Processor -- start 2024-04-02T14:45:15.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.506+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7349ec0 0x55d4b7350260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.506+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b734aba0 0x55d4b73507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.506+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7353230 0x55d4b7350ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.506+0000 7fc8e8837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d4b72aa6a0 con 0x55d4b7349ec0 2024-04-02T14:45:15.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.506+0000 7fc8e8837740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d4b7256980 con 0x55d4b7353230 2024-04-02T14:45:15.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.506+0000 7fc8e8837740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d4b7269cb0 con 0x55d4b734aba0 2024-04-02T14:45:15.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.506+0000 7fc8e8837740 10 client.0 _ll_register_callbacks cb 0x55d4b72696a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:15.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.507+0000 7fc8dea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7349ec0 0x55d4b7350260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.507+0000 7fc8dea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7349ec0 0x55d4b7350260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60002/0 (socket says 172.21.15.62:60002) 2024-04-02T14:45:15.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.507+0000 7fc8dea00640 1 -- 172.21.15.62:0/3738124546 learned_addr learned my addr 172.21.15.62:0/3738124546 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:15.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.507+0000 7fc8de000640 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b734aba0 0x55d4b73507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.507+0000 7fc8df400640 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7353230 0x55d4b7350ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.507+0000 7fc8dea00640 1 -- 172.21.15.62:0/3738124546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b734aba0 msgr2=0x55d4b73507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.507+0000 7fc8dea00640 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b734aba0 0x55d4b73507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.507+0000 7fc8dea00640 1 -- 172.21.15.62:0/3738124546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7353230 msgr2=0x55d4b7350ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.507+0000 7fc8dea00640 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7353230 0x55d4b7350ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.507+0000 7fc8dea00640 1 -- 172.21.15.62:0/3738124546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4b708c7d0 con 0x55d4b7349ec0 2024-04-02T14:45:15.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.507+0000 7fc8de000640 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b734aba0 0x55d4b73507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:15.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.508+0000 7fc8dea00640 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7349ec0 0x55d4b7350260 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000fb20 tx=0x7fc8d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddee140ffb1970c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.508+0000 7fc8c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:45:15.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.508+0000 7fc8c8c00640 1 -- 172.21.15.62:0/3738124546 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d002b060 con 0x55d4b7349ec0 2024-04-02T14:45:15.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.508+0000 7fc8c8c00640 1 -- 172.21.15.62:0/3738124546 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc8d0011cd0 con 0x55d4b7349ec0 2024-04-02T14:45:15.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.508+0000 7fc8e8837740 1 -- 172.21.15.62:0/3738124546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d8033bf0 con 0x55d4b7349ec0 2024-04-02T14:45:15.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.509+0000 7fc8c8c00640 1 -- 172.21.15.62:0/3738124546 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8d0030460 con 0x55d4b7349ec0 2024-04-02T14:45:15.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.509+0000 7fc8e8837740 10 client.15666 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:15.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.509+0000 7fc8e8837740 1 -- 172.21.15.62:0/3738124546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d4b736a700 con 0x55d4b7349ec0 2024-04-02T14:45:15.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.509+0000 7fc8c8c00640 1 -- 172.21.15.62:0/3738124546 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8d0030b00 con 0x55d4b7349ec0 2024-04-02T14:45:15.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.510+0000 7fc8c8c00640 1 -- 172.21.15.62:0/3738124546 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc8d0025030 con 0x55d4b7349ec0 2024-04-02T14:45:15.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.510+0000 7fc8e8837740 10 client.15666 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:15.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.510+0000 7fc8e8837740 10 client.15666 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:45:15.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.510+0000 7fc8e8837740 20 client.15666 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:15.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.510+0000 7fc8e8837740 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55d4b736bbc0 0x55d4b736dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.510+0000 7fc8e8837740 4 client.15666 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:15.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.510+0000 7fc8e8837740 1 -- 172.21.15.62:0/3738124546 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc8d8033bf0 con 0x55d4b736bbc0 2024-04-02T14:45:15.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.510+0000 7fc8de000640 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55d4b736bbc0 0x55d4b736dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.511+0000 7fc8de000640 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55d4b736bbc0 0x55d4b736dfb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b49d060c8f84d18 server_cookie=4f82ab41d9699f3c in_seq=0 out_seq=0 2024-04-02T14:45:15.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.511+0000 7fc8c8c00640 10 client.15666 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:45:15.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8c8c00640 1 -- 172.21.15.62:0/3738124546 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc8d8033bf0 con 0x55d4b736bbc0 2024-04-02T14:45:15.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8c8c00640 10 client.15666 handle_command_reply: tid=0 2024-04-02T14:45:15.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8e8837740 1 client.15666 shutdown 2024-04-02T14:45:15.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8e8837740 1 -- 172.21.15.62:0/3738124546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7349ec0 msgr2=0x55d4b7350260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8e8837740 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7349ec0 0x55d4b7350260 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000fb20 tx=0x7fc8d0004730 comp rx=0 tx=0).stop 2024-04-02T14:45:15.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8e8837740 1 -- 172.21.15.62:0/3738124546 shutdown_connections 2024-04-02T14:45:15.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8e8837740 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55d4b736bbc0 0x55d4b736dfb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8e8837740 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7353230 0x55d4b7350ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8e8837740 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b734aba0 0x55d4b73507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8e8837740 1 --2- 172.21.15.62:0/3738124546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7349ec0 0x55d4b7350260 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8e8837740 1 -- 172.21.15.62:0/3738124546 >> 172.21.15.62:0/3738124546 conn(0x55d4b7310a50 msgr2=0x55d4b7346660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.512+0000 7fc8c8c00640 0 client.15666 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:45:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.513+0000 7fc8e8837740 1 -- 172.21.15.62:0/3738124546 shutdown_connections 2024-04-02T14:45:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.513+0000 7fc8e8837740 1 -- 172.21.15.62:0/3738124546 wait complete. 2024-04-02T14:45:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.513+0000 7fc8e8837740 20 client.15666 trim_cache size 0 max 16384 2024-04-02T14:45:15.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.541+0000 7fc8e8837740 1 Processor -- start 2024-04-02T14:45:15.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.542+0000 7fc8e8837740 1 -- start start 2024-04-02T14:45:15.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.542+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7349ec0 0x55d4b7352990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.542+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b734aba0 0x55d4b7373880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.542+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7353230 0x55d4b7373dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.542+0000 7fc8e8837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01ab390 con 0x55d4b734aba0 2024-04-02T14:45:15.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.542+0000 7fc8e8837740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01aced0 con 0x55d4b7349ec0 2024-04-02T14:45:15.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.542+0000 7fc8e8837740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01aba60 con 0x55d4b7353230 2024-04-02T14:45:15.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.542+0000 7fc8dea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7349ec0 0x55d4b7352990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.542+0000 7fc8dea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7349ec0 0x55d4b7352990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37048/0 (socket says 172.21.15.62:37048) 2024-04-02T14:45:15.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.543+0000 7fc8dea00640 1 -- 172.21.15.62:0/541516488 learned_addr learned my addr 172.21.15.62:0/541516488 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.543+0000 7fc8df400640 1 --2- 172.21.15.62:0/541516488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7353230 0x55d4b7373dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.543+0000 7fc8de000640 1 --2- 172.21.15.62:0/541516488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b734aba0 0x55d4b7373880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.543+0000 7fc8dea00640 1 -- 172.21.15.62:0/541516488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7353230 msgr2=0x55d4b7373dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.543+0000 7fc8dea00640 1 --2- 172.21.15.62:0/541516488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7353230 0x55d4b7373dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.543+0000 7fc8dea00640 1 -- 172.21.15.62:0/541516488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b734aba0 msgr2=0x55d4b7373880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.543+0000 7fc8dea00640 1 --2- 172.21.15.62:0/541516488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b734aba0 0x55d4b7373880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.543+0000 7fc8dea00640 1 -- 172.21.15.62:0/541516488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4b73454e0 con 0x55d4b7349ec0 2024-04-02T14:45:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.543+0000 7fc8dea00640 1 --2- 172.21.15.62:0/541516488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7349ec0 0x55d4b7352990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc8d001cb00 tx=0x7fc8d0009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=142f8949a8fea21a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.543+0000 7fc8cb400640 1 -- 172.21.15.62:0/541516488 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d001c420 con 0x55d4b7349ec0 2024-04-02T14:45:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.544+0000 7fc8cb400640 1 -- 172.21.15.62:0/541516488 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8d0005520 con 0x55d4b7349ec0 2024-04-02T14:45:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.544+0000 7fc8cb400640 1 -- 172.21.15.62:0/541516488 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0004a00 con 0x55d4b7349ec0 2024-04-02T14:45:15.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.544+0000 7fc8e8837740 1 -- 172.21.15.62:0/541516488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7349ec0 msgr2=0x55d4b7352990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.544+0000 7fc8e8837740 1 --2- 172.21.15.62:0/541516488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7349ec0 0x55d4b7352990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc8d001cb00 tx=0x7fc8d0009390 comp rx=0 tx=0).stop 2024-04-02T14:45:15.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.545+0000 7fc8e8837740 1 -- 172.21.15.62:0/541516488 shutdown_connections 2024-04-02T14:45:15.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.545+0000 7fc8e8837740 1 --2- 172.21.15.62:0/541516488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7353230 0x55d4b7373dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.545+0000 7fc8e8837740 1 --2- 172.21.15.62:0/541516488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b734aba0 0x55d4b7373880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.545+0000 7fc8e8837740 1 --2- 172.21.15.62:0/541516488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b7349ec0 0x55d4b7352990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.545+0000 7fc8e8837740 1 -- 172.21.15.62:0/541516488 >> 172.21.15.62:0/541516488 conn(0x55d4b7310a50 msgr2=0x55d4b734eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.545+0000 7fc8e8837740 1 -- 172.21.15.62:0/541516488 shutdown_connections 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.545+0000 7fc8e8837740 1 -- 172.21.15.62:0/541516488 wait complete. 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.545+0000 7fc8e8837740 1 -- start start 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.546+0000 7fc8e8837740 1 Processor -- start 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.546+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7349ec0 0x55d4b7353610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.546+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b7353b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.546+0000 7fc8e8837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 0x55d4b7354090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.546+0000 7fc8e8837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01ac800 con 0x55d4b7353230 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.546+0000 7fc8e8837740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01ac130 con 0x55d4b734aba0 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.546+0000 7fc8e8837740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01a9370 con 0x55d4b7349ec0 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.546+0000 7fc8e8837740 10 client.0 _ll_register_callbacks cb 0x7fc8d000f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:15.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.547+0000 7fc8df400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 0x55d4b7354090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.547+0000 7fc8de000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b7353b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.547+0000 7fc8de000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b7353b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37062/0 (socket says 172.21.15.62:37062) 2024-04-02T14:45:15.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.547+0000 7fc8df400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 0x55d4b7354090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60024/0 (socket says 172.21.15.62:60024) 2024-04-02T14:45:15.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.547+0000 7fc8de000640 1 -- 172.21.15.62:0/1740939946 learned_addr learned my addr 172.21.15.62:0/1740939946 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:15.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.547+0000 7fc8dea00640 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7349ec0 0x55d4b7353610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.547+0000 7fc8de000640 1 -- 172.21.15.62:0/1740939946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7349ec0 msgr2=0x55d4b7353610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.547+0000 7fc8de000640 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7349ec0 0x55d4b7353610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.547+0000 7fc8de000640 1 -- 172.21.15.62:0/1740939946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 msgr2=0x55d4b7354090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.547+0000 7fc8de000640 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 0x55d4b7354090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.547+0000 7fc8de000640 1 -- 172.21.15.62:0/1740939946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4b7362b10 con 0x55d4b734aba0 2024-04-02T14:45:15.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.548+0000 7fc8de000640 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b7353b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc004fd0 tx=0x7fc8cc009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82e14a301e1853ab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.548+0000 7fc8c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:15.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.548+0000 7fc8c8c00640 1 -- 172.21.15.62:0/1740939946 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc01c040 con 0x55d4b734aba0 2024-04-02T14:45:15.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.548+0000 7fc8c8c00640 1 -- 172.21.15.62:0/1740939946 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fc8cc09c940 con 0x55d4b734aba0 2024-04-02T14:45:15.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.548+0000 7fc8e8837740 1 -- 172.21.15.62:0/1740939946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d0005520 con 0x55d4b734aba0 2024-04-02T14:45:15.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.548+0000 7fc8c8c00640 1 -- 172.21.15.62:0/1740939946 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8cc09cde0 con 0x55d4b734aba0 2024-04-02T14:45:15.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.548+0000 7fc8e8837740 10 client.25793 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:15.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.548+0000 7fc8e8837740 1 -- 172.21.15.62:0/1740939946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d4b7363e50 con 0x55d4b734aba0 2024-04-02T14:45:15.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.548+0000 7fc8c8c00640 1 -- 172.21.15.62:0/1740939946 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8d0005520 con 0x55d4b734aba0 2024-04-02T14:45:15.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.549+0000 7fc8c8c00640 1 -- 172.21.15.62:0/1740939946 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc8cc09bb30 con 0x55d4b734aba0 2024-04-02T14:45:15.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.549+0000 7fc8e8837740 10 client.25793 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:15.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.549+0000 7fc8e8837740 10 client.25793 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:45:15.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.549+0000 7fc8e8837740 20 client.25793 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:15.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.549+0000 7fc8e8837740 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55d4b73d2140 0x55d4b73d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.549+0000 7fc8e8837740 4 client.25793 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:45:15.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.549+0000 7fc8e8837740 1 -- 172.21.15.62:0/1740939946 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4b726e040 con 0x55d4b73d2140 2024-04-02T14:45:15.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.549+0000 7fc8dea00640 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55d4b73d2140 0x55d4b73d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8dea00640 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55d4b73d2140 0x55d4b73d4560 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6be3e045cba41b80 server_cookie=da84f53fb3f21523 in_seq=0 out_seq=0 2024-04-02T14:45:15.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8c8c00640 10 client.25793 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:45:15.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8c8c00640 1 -- 172.21.15.62:0/1740939946 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4b726e040 con 0x55d4b73d2140 2024-04-02T14:45:15.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8c8c00640 10 client.25793 handle_command_reply: tid=0 2024-04-02T14:45:15.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8e8837740 1 client.25793 shutdown 2024-04-02T14:45:15.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8e8837740 1 -- 172.21.15.62:0/1740939946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 msgr2=0x55d4b7353b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8e8837740 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b7353b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc004fd0 tx=0x7fc8cc009f60 comp rx=0 tx=0).stop 2024-04-02T14:45:15.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8e8837740 1 -- 172.21.15.62:0/1740939946 shutdown_connections 2024-04-02T14:45:15.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8e8837740 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55d4b73d2140 0x55d4b73d4560 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8c8c00640 0 client.25793 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:45:15.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8e8837740 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4b7353230 0x55d4b7354090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8e8837740 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4b734aba0 0x55d4b7353b50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8e8837740 1 --2- 172.21.15.62:0/1740939946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4b7349ec0 0x55d4b7353610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.550+0000 7fc8e8837740 1 -- 172.21.15.62:0/1740939946 >> 172.21.15.62:0/1740939946 conn(0x55d4b7310a50 msgr2=0x55d4b7345c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.551+0000 7fc8e8837740 1 -- 172.21.15.62:0/1740939946 shutdown_connections 2024-04-02T14:45:15.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.551+0000 7fc8e8837740 1 -- 172.21.15.62:0/1740939946 wait complete. 2024-04-02T14:45:15.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.551+0000 7fc8e8837740 20 client.25793 trim_cache size 0 max 16384 2024-04-02T14:45:15.559 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:45:15.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 -- 172.21.15.62:0/974633915 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc8c40a3f50 msgr2=0x7fc8c40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc8c40a3f50 0x7fc8c40a6410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009fd0 tx=0x7fc8d0014040 comp rx=0 tx=0).stop 2024-04-02T14:45:15.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 -- 172.21.15.62:0/974633915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e01010f0 msgr2=0x7fc8e019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e01010f0 0x7fc8e019aab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00e9f0 tx=0x7fc8cc00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 -- 172.21.15.62:0/974633915 shutdown_connections 2024-04-02T14:45:15.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fc8c40a3f50 0x7fc8c40a6410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e010e2f0 0x7fc8e019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8e01010f0 0x7fc8e019aab0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 --2- 172.21.15.62:0/974633915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e0100740 0x7fc8e019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 -- 172.21.15.62:0/974633915 >> 172.21.15.62:0/974633915 conn(0x7fc8e00fc350 msgr2=0x7fc8e00fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 -- 172.21.15.62:0/974633915 shutdown_connections 2024-04-02T14:45:15.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.552+0000 7fc8dfe00640 1 -- 172.21.15.62:0/974633915 wait complete. 2024-04-02T14:45:15.570 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:15 smithi062 ceph-mon[32116]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-02T14:45:15.570 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:15 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/4235722973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:45:15.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:45:15.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:15 smithi204 ceph-mon[35948]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-02T14:45:15.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:15 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/4235722973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:45:15.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.813+0000 7f44c1200640 1 -- 172.21.15.62:0/1380668038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc103df0 msgr2=0x7f44bc1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.814+0000 7f44c1200640 1 --2- 172.21.15.62:0/1380668038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc103df0 0x7f44bc1041d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009900 tx=0x7f44ac02bc30 comp rx=0 tx=0).stop 2024-04-02T14:45:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.814+0000 7f44c1200640 1 -- 172.21.15.62:0/1380668038 shutdown_connections 2024-04-02T14:45:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.814+0000 7f44c1200640 1 --2- 172.21.15.62:0/1380668038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44bc108ec0 0x7f44bc110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.814+0000 7f44c1200640 1 --2- 172.21.15.62:0/1380668038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44bc1047a0 0x7f44bc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.814+0000 7f44c1200640 1 --2- 172.21.15.62:0/1380668038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc103df0 0x7f44bc1041d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.814+0000 7f44c1200640 1 -- 172.21.15.62:0/1380668038 >> 172.21.15.62:0/1380668038 conn(0x7f44bc0ff610 msgr2=0x7f44bc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.814+0000 7f44c1200640 1 -- 172.21.15.62:0/1380668038 shutdown_connections 2024-04-02T14:45:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.814+0000 7f44c1200640 1 -- 172.21.15.62:0/1380668038 wait complete. 2024-04-02T14:45:15.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.815+0000 7f44c1200640 1 Processor -- start 2024-04-02T14:45:15.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.815+0000 7f44c1200640 1 -- start start 2024-04-02T14:45:15.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.816+0000 7f44c1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44bc103df0 0x7f44bc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.816+0000 7f44c1200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44bc108ec0 0x7f44bc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.816+0000 7f44c1200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc19ee70 0x7f44bc19f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.816+0000 7f44c1200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc114060 con 0x7f44bc103df0 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.816+0000 7f44c1200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc113ee0 con 0x7f44bc19ee70 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.816+0000 7f44c1200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1141e0 con 0x7f44bc108ec0 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.816+0000 7f44bb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44bc103df0 0x7f44bc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.816+0000 7f44bbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc19ee70 0x7f44bc19f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.816+0000 7f44bbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc19ee70 0x7f44bc19f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37082/0 (socket says 172.21.15.62:37082) 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.816+0000 7f44bb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44bc103df0 0x7f44bc19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60052/0 (socket says 172.21.15.62:60052) 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.816+0000 7f44bbe00640 1 -- 172.21.15.62:0/924617165 learned_addr learned my addr 172.21.15.62:0/924617165 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.817+0000 7f44baa00640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44bc108ec0 0x7f44bc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.817+0000 7f44bbe00640 1 -- 172.21.15.62:0/924617165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44bc108ec0 msgr2=0x7f44bc19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.817+0000 7f44bbe00640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44bc108ec0 0x7f44bc19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.817+0000 7f44bbe00640 1 -- 172.21.15.62:0/924617165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44bc103df0 msgr2=0x7f44bc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.817+0000 7f44bbe00640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44bc103df0 0x7f44bc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.817+0000 7f44bbe00640 1 -- 172.21.15.62:0/924617165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44bc19f960 con 0x7f44bc19ee70 2024-04-02T14:45:15.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.817+0000 7f44bbe00640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc19ee70 0x7f44bc19f320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009fd0 tx=0x7f44b400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.818+0000 7f44b3e00640 1 -- 172.21.15.62:0/924617165 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4017070 con 0x7f44bc19ee70 2024-04-02T14:45:15.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.818+0000 7f44b3e00640 1 -- 172.21.15.62:0/924617165 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f44b4004020 con 0x7f44bc19ee70 2024-04-02T14:45:15.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.818+0000 7f44b3e00640 1 -- 172.21.15.62:0/924617165 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4010790 con 0x7f44bc19ee70 2024-04-02T14:45:15.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.818+0000 7f44c1200640 1 -- 172.21.15.62:0/924617165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44bc19fc50 con 0x7f44bc19ee70 2024-04-02T14:45:15.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.818+0000 7f44c1200640 1 -- 172.21.15.62:0/924617165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44bc1a73f0 con 0x7f44bc19ee70 2024-04-02T14:45:15.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.822+0000 7f44c557f740 1 Processor -- start 2024-04-02T14:45:15.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.823+0000 7f44b3e00640 1 -- 172.21.15.62:0/924617165 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f44b40041d0 con 0x7f44bc19ee70 2024-04-02T14:45:15.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.823+0000 7f44c557f740 1 -- start start 2024-04-02T14:45:15.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.823+0000 7f44c557f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c7c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.823+0000 7f44c557f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7cec0 0x560469c7d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.823+0000 7f44c557f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c88620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.823+0000 7f44c557f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560469bbd400 con 0x560469c7dba0 2024-04-02T14:45:15.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.823+0000 7f44c557f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560469ba1f10 con 0x560469c86230 2024-04-02T14:45:15.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.823+0000 7f44c557f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560469be16f0 con 0x560469c7cec0 2024-04-02T14:45:15.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.824+0000 7f44baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c7c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.824+0000 7f44baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c7c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60056/0 (socket says 172.21.15.62:60056) 2024-04-02T14:45:15.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.824+0000 7f44baa00640 1 -- 172.21.15.62:0/2264415122 learned_addr learned my addr 172.21.15.62:0/2264415122 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:15.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.824+0000 7f44baa00640 1 --2- 172.21.15.62:0/2264415122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c88620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.824+0000 7f44baa00640 1 -- 172.21.15.62:0/2264415122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7cec0 msgr2=0x560469c7d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.824+0000 7f44baa00640 1 --2- 172.21.15.62:0/2264415122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7cec0 0x560469c7d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.824+0000 7f44baa00640 1 -- 172.21.15.62:0/2264415122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 msgr2=0x560469c88620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.824+0000 7f44baa00640 1 --2- 172.21.15.62:0/2264415122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c88620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.824+0000 7f44baa00640 1 -- 172.21.15.62:0/2264415122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604698f53f0 con 0x560469c7dba0 2024-04-02T14:45:15.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.825+0000 7f44baa00640 1 --2- 172.21.15.62:0/2264415122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c7c8f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f44a8013520 tx=0x7f44a8009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9079e1bd21c1575f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.825+0000 7f44b0c00640 1 -- 172.21.15.62:0/2264415122 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8013270 con 0x560469c7dba0 2024-04-02T14:45:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.825+0000 7f44b3e00640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44a00a3f50 0x7f44a00a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.825+0000 7f44b0c00640 1 -- 172.21.15.62:0/2264415122 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f44a8010990 con 0x560469c7dba0 2024-04-02T14:45:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.825+0000 7f44b0c00640 1 -- 172.21.15.62:0/2264415122 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8011ac0 con 0x560469c7dba0 2024-04-02T14:45:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.825+0000 7f44b3e00640 1 -- 172.21.15.62:0/924617165 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f44b40d0ac0 con 0x7f44bc19ee70 2024-04-02T14:45:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.826+0000 7f44bb400640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44a00a3f50 0x7f44a00a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.826+0000 7f44c557f740 1 -- 172.21.15.62:0/2264415122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 msgr2=0x560469c7c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.826+0000 7f44c557f740 1 --2- 172.21.15.62:0/2264415122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c7c8f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f44a8013520 tx=0x7f44a8009760 comp rx=0 tx=0).stop 2024-04-02T14:45:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.827+0000 7f44c557f740 1 -- 172.21.15.62:0/2264415122 shutdown_connections 2024-04-02T14:45:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.827+0000 7f44c557f740 1 --2- 172.21.15.62:0/2264415122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c88620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.827+0000 7f44c557f740 1 --2- 172.21.15.62:0/2264415122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7cec0 0x560469c7d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.827+0000 7f44c557f740 1 --2- 172.21.15.62:0/2264415122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c7c8f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.827+0000 7f44c557f740 1 -- 172.21.15.62:0/2264415122 >> 172.21.15.62:0/2264415122 conn(0x560469c43a50 msgr2=0x560469c77910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.827+0000 7f44bb400640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44a00a3f50 0x7f44a00a6410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f44ac006010 tx=0x7f44ac005b50 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.827+0000 7f44c557f740 1 -- 172.21.15.62:0/2264415122 shutdown_connections 2024-04-02T14:45:15.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.827+0000 7f44c557f740 1 -- 172.21.15.62:0/2264415122 wait complete. 2024-04-02T14:45:15.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.828+0000 7f44c557f740 1 -- start start 2024-04-02T14:45:15.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.828+0000 7f44c557f740 1 Processor -- start 2024-04-02T14:45:15.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.828+0000 7f44c557f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7cec0 0x560469c832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.828+0000 7f44c557f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44c557f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c83d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44c557f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560469bdd6a0 con 0x560469c7dba0 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44c557f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560469b89980 con 0x560469c86230 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44c557f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560469b9ccb0 con 0x560469c7cec0 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44c557f740 10 client.0 _ll_register_callbacks cb 0x560469b9c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44bb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7cec0 0x560469c832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44bb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7cec0 0x560469c832b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55442/0 (socket says 172.21.15.62:55442) 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44bb400640 1 -- 172.21.15.62:0/2809947007 learned_addr learned my addr 172.21.15.62:0/2809947007 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44baa00640 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44bbe00640 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c83d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44bb400640 1 -- 172.21.15.62:0/2809947007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 msgr2=0x560469c83d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44bb400640 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c83d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44bb400640 1 -- 172.21.15.62:0/2809947007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 msgr2=0x560469c837f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44bb400640 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c837f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44bb400640 1 -- 172.21.15.62:0/2809947007 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604699bf7d0 con 0x560469c7cec0 2024-04-02T14:45:15.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.829+0000 7f44baa00640 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:15.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.830+0000 7f44bb400640 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7cec0 0x560469c832b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009900 tx=0x7f44ac02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86aaa97fc2139308 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.830+0000 7f44a5600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:45:15.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.830+0000 7f44a5600640 1 -- 172.21.15.62:0/2809947007 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac02ce60 con 0x560469c7cec0 2024-04-02T14:45:15.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.830+0000 7f44c557f740 1 -- 172.21.15.62:0/2809947007 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a8010990 con 0x560469c7cec0 2024-04-02T14:45:15.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.830+0000 7f44a5600640 1 -- 172.21.15.62:0/2809947007 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f44ac0042c0 con 0x560469c7cec0 2024-04-02T14:45:15.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.831+0000 7f44a5600640 1 -- 172.21.15.62:0/2809947007 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44ac0044c0 con 0x560469c7cec0 2024-04-02T14:45:15.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.831+0000 7f44c557f740 10 client.25783 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:15.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.831+0000 7f44c557f740 1 -- 172.21.15.62:0/2809947007 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560469c94f40 con 0x560469c7cec0 2024-04-02T14:45:15.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.831+0000 7f44a5600640 1 -- 172.21.15.62:0/2809947007 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f44a8010990 con 0x560469c7cec0 2024-04-02T14:45:15.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.831+0000 7f44a5600640 1 -- 172.21.15.62:0/2809947007 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f44ac054030 con 0x560469c7cec0 2024-04-02T14:45:15.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.831+0000 7f44c557f740 10 client.25783 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:15.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.831+0000 7f44c557f740 10 client.25783 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:45:15.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.831+0000 7f44c557f740 20 client.25783 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:15.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.831+0000 7f44c557f740 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560469c963a0 0x560469c98810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.831+0000 7f44c557f740 4 client.25783 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:15.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.831+0000 7f44c557f740 1 -- 172.21.15.62:0/2809947007 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560469ba1040 con 0x560469c963a0 2024-04-02T14:45:15.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.832+0000 7f44baa00640 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560469c963a0 0x560469c98810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.833+0000 7f44baa00640 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560469c963a0 0x560469c98810 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8c3ecd06ad4949b server_cookie=a30a9b400e8554ee in_seq=0 out_seq=0 2024-04-02T14:45:15.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.833+0000 7f44a5600640 10 client.25783 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:45:15.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.833+0000 7f44a5600640 1 -- 172.21.15.62:0/2809947007 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560469ba1040 con 0x560469c963a0 2024-04-02T14:45:15.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.833+0000 7f44a5600640 10 client.25783 handle_command_reply: tid=0 2024-04-02T14:45:15.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.833+0000 7f44c557f740 1 client.25783 shutdown 2024-04-02T14:45:15.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44c557f740 1 -- 172.21.15.62:0/2809947007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7cec0 msgr2=0x560469c832b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44c557f740 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7cec0 0x560469c832b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009900 tx=0x7f44ac02bc30 comp rx=0 tx=0).stop 2024-04-02T14:45:15.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44c557f740 1 -- 172.21.15.62:0/2809947007 shutdown_connections 2024-04-02T14:45:15.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44c557f740 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560469c963a0 0x560469c98810 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44c557f740 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c83d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44c557f740 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44c557f740 1 --2- 172.21.15.62:0/2809947007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7cec0 0x560469c832b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44c557f740 1 -- 172.21.15.62:0/2809947007 >> 172.21.15.62:0/2809947007 conn(0x560469c43a50 msgr2=0x560469c78dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44a5600640 0 client.25783 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:45:15.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44c557f740 1 -- 172.21.15.62:0/2809947007 shutdown_connections 2024-04-02T14:45:15.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44c557f740 1 -- 172.21.15.62:0/2809947007 wait complete. 2024-04-02T14:45:15.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.834+0000 7f44c557f740 20 client.25783 trim_cache size 0 max 16384 2024-04-02T14:45:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.862+0000 7f44c557f740 1 Processor -- start 2024-04-02T14:45:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.862+0000 7f44c557f740 1 -- start start 2024-04-02T14:45:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.863+0000 7f44c557f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7cec0 0x560469c859e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.863+0000 7f44c557f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7dba0 0x560469c91c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.863+0000 7f44c557f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c822b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.863+0000 7f44c557f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1af8e0 con 0x560469c7cec0 2024-04-02T14:45:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.863+0000 7f44c557f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1b1420 con 0x560469c86230 2024-04-02T14:45:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.863+0000 7f44c557f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1affb0 con 0x560469c7dba0 2024-04-02T14:45:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.863+0000 7f44bb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7cec0 0x560469c859e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.863+0000 7f44bb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7cec0 0x560469c859e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60062/0 (socket says 172.21.15.62:60062) 2024-04-02T14:45:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.863+0000 7f44bb400640 1 -- 172.21.15.62:0/662258885 learned_addr learned my addr 172.21.15.62:0/662258885 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.863+0000 7f44baa00640 1 --2- 172.21.15.62:0/662258885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7dba0 0x560469c91c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.863+0000 7f44bbe00640 1 --2- 172.21.15.62:0/662258885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c822b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.864+0000 7f44baa00640 1 -- 172.21.15.62:0/662258885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 msgr2=0x560469c822b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.864+0000 7f44baa00640 1 --2- 172.21.15.62:0/662258885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c822b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.864+0000 7f44baa00640 1 -- 172.21.15.62:0/662258885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7cec0 msgr2=0x560469c859e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.864+0000 7f44baa00640 1 --2- 172.21.15.62:0/662258885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7cec0 0x560469c859e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.864+0000 7f44baa00640 1 -- 172.21.15.62:0/662258885 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560469c77b20 con 0x560469c7dba0 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.864+0000 7f44bb400640 1 --2- 172.21.15.62:0/662258885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7cec0 0x560469c859e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.864+0000 7f44baa00640 1 --2- 172.21.15.62:0/662258885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7dba0 0x560469c91c40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f44a8013e70 tx=0x7f44a8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87364d465f3f7e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.864+0000 7f44a7e00640 1 -- 172.21.15.62:0/662258885 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a802e070 con 0x560469c7dba0 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.864+0000 7f44a7e00640 1 -- 172.21.15.62:0/662258885 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f44a80050e0 con 0x560469c7dba0 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.864+0000 7f44a7e00640 1 -- 172.21.15.62:0/662258885 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8003970 con 0x560469c7dba0 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.865+0000 7f44c557f740 1 -- 172.21.15.62:0/662258885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7dba0 msgr2=0x560469c91c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.865+0000 7f44c557f740 1 --2- 172.21.15.62:0/662258885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7dba0 0x560469c91c40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f44a8013e70 tx=0x7f44a8009c00 comp rx=0 tx=0).stop 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.865+0000 7f44c557f740 1 -- 172.21.15.62:0/662258885 shutdown_connections 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.865+0000 7f44c557f740 1 --2- 172.21.15.62:0/662258885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c86230 0x560469c822b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.865+0000 7f44c557f740 1 --2- 172.21.15.62:0/662258885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c7dba0 0x560469c91c40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.865+0000 7f44c557f740 1 --2- 172.21.15.62:0/662258885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7cec0 0x560469c859e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.865+0000 7f44c557f740 1 -- 172.21.15.62:0/662258885 >> 172.21.15.62:0/662258885 conn(0x560469c43a50 msgr2=0x560469c81ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.865+0000 7f44c557f740 1 -- 172.21.15.62:0/662258885 shutdown_connections 2024-04-02T14:45:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.866+0000 7f44c557f740 1 -- 172.21.15.62:0/662258885 wait complete. 2024-04-02T14:45:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.866+0000 7f44c557f740 1 -- start start 2024-04-02T14:45:15.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.866+0000 7f44c557f740 1 Processor -- start 2024-04-02T14:45:15.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44c557f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c7cec0 0x560469c8e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44c557f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c80550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44c557f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c86230 0x560469c80a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44c557f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1b0d50 con 0x560469c7dba0 2024-04-02T14:45:15.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44c557f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1b0680 con 0x560469c7cec0 2024-04-02T14:45:15.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44c557f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1ad8e0 con 0x560469c86230 2024-04-02T14:45:15.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44c557f740 10 client.0 _ll_register_callbacks cb 0x7f44ac02cd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c80550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c80550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60076/0 (socket says 172.21.15.62:60076) 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44bb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c7cec0 0x560469c8e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44baa00640 1 -- 172.21.15.62:0/3480846026 learned_addr learned my addr 172.21.15.62:0/3480846026 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.867+0000 7f44bbe00640 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c86230 0x560469c80a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.868+0000 7f44bb400640 1 -- 172.21.15.62:0/3480846026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c86230 msgr2=0x560469c80a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.868+0000 7f44bb400640 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c86230 0x560469c80a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.868+0000 7f44bb400640 1 -- 172.21.15.62:0/3480846026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 msgr2=0x560469c80550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.868+0000 7f44bb400640 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c80550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.868+0000 7f44bb400640 1 -- 172.21.15.62:0/3480846026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560469c812c0 con 0x560469c7cec0 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.868+0000 7f44bb400640 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c7cec0 0x560469c8e4f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009900 tx=0x7f44ac003ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989ceddfeb18cad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.868+0000 7f44a5600640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.868+0000 7f44a5600640 1 -- 172.21.15.62:0/3480846026 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac034b70 con 0x560469c7cec0 2024-04-02T14:45:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.868+0000 7f44c557f740 1 -- 172.21.15.62:0/3480846026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a80050e0 con 0x560469c7cec0 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.868+0000 7f44a5600640 1 -- 172.21.15.62:0/3480846026 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f44ac034d10 con 0x560469c7cec0 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.869+0000 7f44a5600640 1 -- 172.21.15.62:0/3480846026 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44ac0046c0 con 0x560469c7cec0 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.869+0000 7f44c557f740 10 client.25811 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.869+0000 7f44c557f740 1 -- 172.21.15.62:0/3480846026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560469cfc9b0 con 0x560469c7cec0 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.869+0000 7f44a5600640 1 -- 172.21.15.62:0/3480846026 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f44a80050e0 con 0x560469c7cec0 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.869+0000 7f44a5600640 1 -- 172.21.15.62:0/3480846026 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f44ac054040 con 0x560469c7cec0 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.869+0000 7f44c557f740 10 client.25811 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.869+0000 7f44c557f740 10 client.25811 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.869+0000 7f44c557f740 20 client.25811 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.869+0000 7f44c557f740 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560469cfd0c0 0x560469cff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.869+0000 7f44c557f740 4 client.25811 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:45:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.869+0000 7f44c557f740 1 -- 172.21.15.62:0/3480846026 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560469beec30 con 0x560469cfd0c0 2024-04-02T14:45:15.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:15 smithi113 ceph-mon[36135]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-02T14:45:15.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:15 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/4235722973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:45:15.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.870+0000 7f44baa00640 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560469cfd0c0 0x560469cff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:15.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.870+0000 7f44baa00640 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560469cfd0c0 0x560469cff4b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=579116f90eb36f65 server_cookie=d1d366c3d4b4e52 in_seq=0 out_seq=0 2024-04-02T14:45:15.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.870+0000 7f44a5600640 10 client.25811 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:45:15.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44a5600640 1 -- 172.21.15.62:0/3480846026 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560469beec30 con 0x560469cfd0c0 2024-04-02T14:45:15.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44a5600640 10 client.25811 handle_command_reply: tid=0 2024-04-02T14:45:15.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44c557f740 1 client.25811 shutdown 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44c557f740 1 -- 172.21.15.62:0/3480846026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c7cec0 msgr2=0x560469c8e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44c557f740 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c7cec0 0x560469c8e4f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009900 tx=0x7f44ac003ec0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44c557f740 1 -- 172.21.15.62:0/3480846026 shutdown_connections 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44c557f740 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560469cfd0c0 0x560469cff4b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44c557f740 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560469c86230 0x560469c80a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44c557f740 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560469c7dba0 0x560469c80550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44c557f740 1 --2- 172.21.15.62:0/3480846026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560469c7cec0 0x560469c8e4f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44a5600640 0 client.25811 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44c557f740 1 -- 172.21.15.62:0/3480846026 >> 172.21.15.62:0/3480846026 conn(0x560469c43a50 msgr2=0x560469c82430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44c557f740 1 -- 172.21.15.62:0/3480846026 shutdown_connections 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.871+0000 7f44c557f740 1 -- 172.21.15.62:0/3480846026 wait complete. 2024-04-02T14:45:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.872+0000 7f44c557f740 20 client.25811 trim_cache size 0 max 16384 2024-04-02T14:45:15.872 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:45:15.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.872+0000 7f44c1200640 1 -- 172.21.15.62:0/924617165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44a00a3f50 msgr2=0x7f44a00a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.872+0000 7f44c1200640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44a00a3f50 0x7f44a00a6410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f44ac006010 tx=0x7f44ac005b50 comp rx=0 tx=0).stop 2024-04-02T14:45:15.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.872+0000 7f44c1200640 1 -- 172.21.15.62:0/924617165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc19ee70 msgr2=0x7f44bc19f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:15.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.872+0000 7f44c1200640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc19ee70 0x7f44bc19f320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009fd0 tx=0x7f44b400b260 comp rx=0 tx=0).stop 2024-04-02T14:45:15.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.872+0000 7f44c1200640 1 -- 172.21.15.62:0/924617165 shutdown_connections 2024-04-02T14:45:15.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.872+0000 7f44c1200640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44a00a3f50 0x7f44a00a6410 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.872+0000 7f44c1200640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc19ee70 0x7f44bc19f320 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.872+0000 7f44c1200640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44bc108ec0 0x7f44bc19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.872+0000 7f44c1200640 1 --2- 172.21.15.62:0/924617165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44bc103df0 0x7f44bc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:15.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.872+0000 7f44c1200640 1 -- 172.21.15.62:0/924617165 >> 172.21.15.62:0/924617165 conn(0x7f44bc0ff610 msgr2=0x7f44bc100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:15.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.873+0000 7f44c1200640 1 -- 172.21.15.62:0/924617165 shutdown_connections 2024-04-02T14:45:15.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:15.873+0000 7f44c1200640 1 -- 172.21.15.62:0/924617165 wait complete. 2024-04-02T14:45:15.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:45:16.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.064+0000 7f6b82600640 1 -- 172.21.15.62:0/1021779656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7c1029c0 msgr2=0x7f6b7c06a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.064+0000 7f6b82600640 1 --2- 172.21.15.62:0/1021779656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7c1029c0 0x7f6b7c06a070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009900 tx=0x7f6b6c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:45:16.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.064+0000 7f6b82600640 1 -- 172.21.15.62:0/1021779656 shutdown_connections 2024-04-02T14:45:16.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.065+0000 7f6b82600640 1 --2- 172.21.15.62:0/1021779656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c10e350 0x7f6b7c110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.065+0000 7f6b82600640 1 --2- 172.21.15.62:0/1021779656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c06a5b0 0x7f6b7c06a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.065+0000 7f6b82600640 1 --2- 172.21.15.62:0/1021779656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7c1029c0 0x7f6b7c06a070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.065+0000 7f6b82600640 1 -- 172.21.15.62:0/1021779656 >> 172.21.15.62:0/1021779656 conn(0x7f6b7c0779e0 msgr2=0x7f6b7c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.065+0000 7f6b82600640 1 -- 172.21.15.62:0/1021779656 shutdown_connections 2024-04-02T14:45:16.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.065+0000 7f6b82600640 1 -- 172.21.15.62:0/1021779656 wait complete. 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.066+0000 7f6b82600640 1 Processor -- start 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.066+0000 7f6b82600640 1 -- start start 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.066+0000 7f6b82600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c06a5b0 0x7f6b7c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.066+0000 7f6b82600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7c1029c0 0x7f6b7c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.066+0000 7f6b82600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c10e350 0x7f6b7c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.066+0000 7f6b82600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c114150 con 0x7f6b7c10e350 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.066+0000 7f6b82600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c113fd0 con 0x7f6b7c06a5b0 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.066+0000 7f6b82600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1142d0 con 0x7f6b7c1029c0 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.067+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c06a5b0 0x7f6b7c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.067+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c06a5b0 0x7f6b7c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37146/0 (socket says 172.21.15.62:37146) 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.067+0000 7f6b7b400640 1 -- 172.21.15.62:0/951286601 learned_addr learned my addr 172.21.15.62:0/951286601 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.067+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7c1029c0 0x7f6b7c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.067+0000 7f6b7be00640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c10e350 0x7f6b7c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.067+0000 7f6b7b400640 1 -- 172.21.15.62:0/951286601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7c1029c0 msgr2=0x7f6b7c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.067+0000 7f6b7b400640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7c1029c0 0x7f6b7c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.067+0000 7f6b7b400640 1 -- 172.21.15.62:0/951286601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c10e350 msgr2=0x7f6b7c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.067+0000 7f6b7b400640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c10e350 0x7f6b7c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.067+0000 7f6b7b400640 1 -- 172.21.15.62:0/951286601 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c19f680 con 0x7f6b7c06a5b0 2024-04-02T14:45:16.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.067+0000 7f6b7b400640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c06a5b0 0x7f6b7c19a690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c005ab0 tx=0x7f6b6c02dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.068+0000 7f6b73e00640 1 -- 172.21.15.62:0/951286601 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c043020 con 0x7f6b7c06a5b0 2024-04-02T14:45:16.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.068+0000 7f6b73e00640 1 -- 172.21.15.62:0/951286601 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6b6c004250 con 0x7f6b7c06a5b0 2024-04-02T14:45:16.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.068+0000 7f6b73e00640 1 -- 172.21.15.62:0/951286601 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c003a10 con 0x7f6b7c06a5b0 2024-04-02T14:45:16.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.068+0000 7f6b82600640 1 -- 172.21.15.62:0/951286601 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c19f910 con 0x7f6b7c06a5b0 2024-04-02T14:45:16.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.068+0000 7f6b82600640 1 -- 172.21.15.62:0/951286601 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c1a7200 con 0x7f6b7c06a5b0 2024-04-02T14:45:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.071+0000 7f6b853eb740 1 Processor -- start 2024-04-02T14:45:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.071+0000 7f6b853eb740 1 -- start start 2024-04-02T14:45:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.072+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 0x5648b48ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.072+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48acd10 0x5648b48ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.072+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48b6080 0x5648b48b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.072+0000 7f6b853eb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5648b47ed400 con 0x5648b48ad9f0 2024-04-02T14:45:16.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.072+0000 7f6b853eb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5648b47d1f10 con 0x5648b48b6080 2024-04-02T14:45:16.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.072+0000 7f6b853eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648b48116f0 con 0x5648b48acd10 2024-04-02T14:45:16.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.072+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 0x5648b48ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.072+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 0x5648b48ac740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60108/0 (socket says 172.21.15.62:60108) 2024-04-02T14:45:16.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.072+0000 7f6b7be00640 1 -- 172.21.15.62:0/3960556798 learned_addr learned my addr 172.21.15.62:0/3960556798 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.072+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/3960556798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48b6080 0x5648b48b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.072+0000 7f6b73e00640 1 -- 172.21.15.62:0/951286601 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f6b6c046020 con 0x7f6b7c06a5b0 2024-04-02T14:45:16.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.073+0000 7f6b7be00640 1 -- 172.21.15.62:0/3960556798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48acd10 msgr2=0x5648b48ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.073+0000 7f6b7be00640 1 --2- 172.21.15.62:0/3960556798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48acd10 0x5648b48ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.073+0000 7f6b7be00640 1 -- 172.21.15.62:0/3960556798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48b6080 msgr2=0x5648b48b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.073+0000 7f6b7be00640 1 --2- 172.21.15.62:0/3960556798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48b6080 0x5648b48b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.073+0000 7f6b7be00640 1 -- 172.21.15.62:0/3960556798 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b45253f0 con 0x5648b48ad9f0 2024-04-02T14:45:16.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.073+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/3960556798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48b6080 0x5648b48b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:16.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.073+0000 7f6b7be00640 1 --2- 172.21.15.62:0/3960556798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 0x5648b48ac740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009f30 tx=0x7f6b740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795671324eecdec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.074+0000 7f6b70c00640 1 -- 172.21.15.62:0/3960556798 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74004d20 con 0x5648b48ad9f0 2024-04-02T14:45:16.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.074+0000 7f6b70c00640 1 -- 172.21.15.62:0/3960556798 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6b74004050 con 0x5648b48ad9f0 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.074+0000 7f6b70c00640 1 -- 172.21.15.62:0/3960556798 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74005280 con 0x5648b48ad9f0 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.074+0000 7f6b73e00640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f6b600a3ed0 0x7f6b600a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.074+0000 7f6b73e00640 1 -- 172.21.15.62:0/951286601 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f6b6c0f29a0 con 0x7f6b7c06a5b0 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.075+0000 7f6b853eb740 1 -- 172.21.15.62:0/3960556798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 msgr2=0x5648b48ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.075+0000 7f6b853eb740 1 --2- 172.21.15.62:0/3960556798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 0x5648b48ac740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009f30 tx=0x7f6b740118d0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.075+0000 7f6b7b400640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f6b600a3ed0 0x7f6b600a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.075+0000 7f6b853eb740 1 -- 172.21.15.62:0/3960556798 shutdown_connections 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.075+0000 7f6b853eb740 1 --2- 172.21.15.62:0/3960556798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48b6080 0x5648b48b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.075+0000 7f6b853eb740 1 --2- 172.21.15.62:0/3960556798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48acd10 0x5648b48ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.075+0000 7f6b853eb740 1 --2- 172.21.15.62:0/3960556798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 0x5648b48ac740 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.075+0000 7f6b853eb740 1 -- 172.21.15.62:0/3960556798 >> 172.21.15.62:0/3960556798 conn(0x5648b4873a50 msgr2=0x5648b48a7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.075+0000 7f6b853eb740 1 -- 172.21.15.62:0/3960556798 shutdown_connections 2024-04-02T14:45:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.076+0000 7f6b853eb740 1 -- 172.21.15.62:0/3960556798 wait complete. 2024-04-02T14:45:16.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.076+0000 7f6b7b400640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f6b600a3ed0 0x7f6b600a6390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0f0070 tx=0x7f6b6c005610 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.076+0000 7f6b853eb740 1 -- start start 2024-04-02T14:45:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.077+0000 7f6b853eb740 1 Processor -- start 2024-04-02T14:45:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.077+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48acd10 0x5648b48b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.077+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48ad9f0 0x5648b48b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.077+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 0x5648b48b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.077+0000 7f6b853eb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5648b480d6a0 con 0x5648b48b6080 2024-04-02T14:45:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.077+0000 7f6b853eb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5648b47b9980 con 0x5648b48acd10 2024-04-02T14:45:16.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.077+0000 7f6b853eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648b47cccb0 con 0x5648b48ad9f0 2024-04-02T14:45:16.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.077+0000 7f6b853eb740 10 client.0 _ll_register_callbacks cb 0x5648b47cc6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:16.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.078+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48acd10 0x5648b48b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.078+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 0x5648b48b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.078+0000 7f6b7aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48ad9f0 0x5648b48b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.078+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 0x5648b48b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60122/0 (socket says 172.21.15.62:60122) 2024-04-02T14:45:16.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.078+0000 7f6b7be00640 1 -- 172.21.15.62:0/3010982189 learned_addr learned my addr 172.21.15.62:0/3010982189 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.078+0000 7f6b7aa00640 1 -- 172.21.15.62:0/3010982189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48acd10 msgr2=0x5648b48b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.078+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/3010982189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48acd10 0x5648b48b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.078+0000 7f6b7aa00640 1 -- 172.21.15.62:0/3010982189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 msgr2=0x5648b48b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.078+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/3010982189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 0x5648b48b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.078+0000 7f6b7aa00640 1 -- 172.21.15.62:0/3010982189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b45ef7d0 con 0x5648b48ad9f0 2024-04-02T14:45:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.079+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/3010982189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48ad9f0 0x5648b48b35c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6b680126d0 tx=0x7f6b68012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f19f5ecde5989b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.079+0000 7f6b65600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:45:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.079+0000 7f6b65600640 1 -- 172.21.15.62:0/3010982189 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68004030 con 0x5648b48ad9f0 2024-04-02T14:45:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.079+0000 7f6b65600640 1 -- 172.21.15.62:0/3010982189 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f6b680041d0 con 0x5648b48ad9f0 2024-04-02T14:45:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.079+0000 7f6b853eb740 1 -- 172.21.15.62:0/3010982189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b74004050 con 0x5648b48ad9f0 2024-04-02T14:45:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.079+0000 7f6b65600640 1 -- 172.21.15.62:0/3010982189 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b680043d0 con 0x5648b48ad9f0 2024-04-02T14:45:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.079+0000 7f6b853eb740 10 client.25798 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.079+0000 7f6b853eb740 1 -- 172.21.15.62:0/3010982189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5648b48cd7a0 con 0x5648b48ad9f0 2024-04-02T14:45:16.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.080+0000 7f6b65600640 1 -- 172.21.15.62:0/3010982189 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6b74004050 con 0x5648b48ad9f0 2024-04-02T14:45:16.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.080+0000 7f6b65600640 1 -- 172.21.15.62:0/3010982189 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6b68003c70 con 0x5648b48ad9f0 2024-04-02T14:45:16.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.081+0000 7f6b853eb740 10 client.25798 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:16.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.081+0000 7f6b853eb740 10 client.25798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:45:16.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.081+0000 7f6b853eb740 20 client.25798 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:16.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.081+0000 7f6b853eb740 1 --2- 172.21.15.62:0/3010982189 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5648b48cec00 0x5648b48d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.081+0000 7f6b853eb740 4 client.25798 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:16.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.081+0000 7f6b853eb740 1 -- 172.21.15.62:0/3010982189 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648b47d1040 con 0x5648b48cec00 2024-04-02T14:45:16.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.081+0000 7f6b7be00640 1 --2- 172.21.15.62:0/3010982189 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5648b48cec00 0x5648b48d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.081+0000 7f6b65600640 10 client.25798 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:45:16.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.081+0000 7f6b7be00640 1 --2- 172.21.15.62:0/3010982189 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5648b48cec00 0x5648b48d1020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53afd847ac82779b server_cookie=7d052a1ac80bc54a in_seq=0 out_seq=0 2024-04-02T14:45:16.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.083+0000 7f6b65600640 1 -- 172.21.15.62:0/3010982189 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648b47d1040 con 0x5648b48cec00 2024-04-02T14:45:16.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.083+0000 7f6b65600640 10 client.25798 handle_command_reply: tid=0 2024-04-02T14:45:16.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.083+0000 7f6b853eb740 1 client.25798 shutdown 2024-04-02T14:45:16.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.083+0000 7f6b853eb740 1 -- 172.21.15.62:0/3010982189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48ad9f0 msgr2=0x5648b48b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.083+0000 7f6b853eb740 1 --2- 172.21.15.62:0/3010982189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48ad9f0 0x5648b48b35c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6b680126d0 tx=0x7f6b68012b90 comp rx=0 tx=0).stop 2024-04-02T14:45:16.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.083+0000 7f6b853eb740 1 -- 172.21.15.62:0/3010982189 shutdown_connections 2024-04-02T14:45:16.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.083+0000 7f6b853eb740 1 --2- 172.21.15.62:0/3010982189 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5648b48cec00 0x5648b48d1020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.083+0000 7f6b853eb740 1 --2- 172.21.15.62:0/3010982189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 0x5648b48b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.083+0000 7f6b853eb740 1 --2- 172.21.15.62:0/3010982189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48ad9f0 0x5648b48b35c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.083+0000 7f6b853eb740 1 --2- 172.21.15.62:0/3010982189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48acd10 0x5648b48b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.083+0000 7f6b853eb740 1 -- 172.21.15.62:0/3010982189 >> 172.21.15.62:0/3010982189 conn(0x5648b4873a50 msgr2=0x5648b48a94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.084+0000 7f6b65600640 0 client.25798 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:45:16.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.084+0000 7f6b853eb740 1 -- 172.21.15.62:0/3010982189 shutdown_connections 2024-04-02T14:45:16.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.084+0000 7f6b853eb740 1 -- 172.21.15.62:0/3010982189 wait complete. 2024-04-02T14:45:16.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.084+0000 7f6b853eb740 20 client.25798 trim_cache size 0 max 16384 2024-04-02T14:45:16.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.099+0000 7f6b853eb740 1 Processor -- start 2024-04-02T14:45:16.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.099+0000 7f6b853eb740 1 -- start start 2024-04-02T14:45:16.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.099+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48acd10 0x5648b48b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.099+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 0x5648b48ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.099+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48b6080 0x5648b48d6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.099+0000 7f6b853eb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1ab5f0 con 0x5648b48ad9f0 2024-04-02T14:45:16.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.099+0000 7f6b853eb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1ad130 con 0x5648b48acd10 2024-04-02T14:45:16.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.099+0000 7f6b853eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1abcc0 con 0x5648b48b6080 2024-04-02T14:45:16.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 0x5648b48ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48acd10 0x5648b48b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48acd10 0x5648b48b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37186/0 (socket says 172.21.15.62:37186) 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7be00640 1 -- 172.21.15.62:0/2191357050 learned_addr learned my addr 172.21.15.62:0/2191357050 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7be00640 1 --2- 172.21.15.62:0/2191357050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48b6080 0x5648b48d6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7aa00640 1 -- 172.21.15.62:0/2191357050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48b6080 msgr2=0x5648b48d6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/2191357050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48b6080 0x5648b48d6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7aa00640 1 -- 172.21.15.62:0/2191357050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48acd10 msgr2=0x5648b48b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/2191357050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48acd10 0x5648b48b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7aa00640 1 -- 172.21.15.62:0/2191357050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b48a7b20 con 0x5648b48ad9f0 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7be00640 1 --2- 172.21.15.62:0/2191357050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48b6080 0x5648b48d6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.100+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/2191357050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 0x5648b48ca380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6b680059f0 tx=0x7f6b68009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aae7307fb4cc285 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.101+0000 7f6b67e00640 1 -- 172.21.15.62:0/2191357050 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68003640 con 0x5648b48ad9f0 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.101+0000 7f6b67e00640 1 -- 172.21.15.62:0/2191357050 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6b68005030 con 0x5648b48ad9f0 2024-04-02T14:45:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.101+0000 7f6b67e00640 1 -- 172.21.15.62:0/2191357050 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68003d30 con 0x5648b48ad9f0 2024-04-02T14:45:16.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.101+0000 7f6b853eb740 1 -- 172.21.15.62:0/2191357050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 msgr2=0x5648b48ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.101+0000 7f6b853eb740 1 --2- 172.21.15.62:0/2191357050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 0x5648b48ca380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6b680059f0 tx=0x7f6b68009830 comp rx=0 tx=0).stop 2024-04-02T14:45:16.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.102+0000 7f6b853eb740 1 -- 172.21.15.62:0/2191357050 shutdown_connections 2024-04-02T14:45:16.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.102+0000 7f6b853eb740 1 --2- 172.21.15.62:0/2191357050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48b6080 0x5648b48d6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.102+0000 7f6b853eb740 1 --2- 172.21.15.62:0/2191357050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48ad9f0 0x5648b48ca380 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.102+0000 7f6b853eb740 1 --2- 172.21.15.62:0/2191357050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48acd10 0x5648b48b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.102+0000 7f6b853eb740 1 -- 172.21.15.62:0/2191357050 >> 172.21.15.62:0/2191357050 conn(0x5648b4873a50 msgr2=0x5648b48b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.102+0000 7f6b853eb740 1 -- 172.21.15.62:0/2191357050 shutdown_connections 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.102+0000 7f6b853eb740 1 -- 172.21.15.62:0/2191357050 wait complete. 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.102+0000 7f6b853eb740 1 -- start start 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.102+0000 7f6b853eb740 1 Processor -- start 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48acd10 0x5648b48c6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48ad9f0 0x5648b48c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b853eb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 0x5648b48b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b853eb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1aca60 con 0x5648b48b6080 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b853eb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1ac390 con 0x5648b48ad9f0 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b853eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1a9610 con 0x5648b48acd10 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b853eb740 10 client.0 _ll_register_callbacks cb 0x7f6b68009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b7aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48ad9f0 0x5648b48c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b7aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48ad9f0 0x5648b48c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37190/0 (socket says 172.21.15.62:37190) 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b7aa00640 1 -- 172.21.15.62:0/592653102 learned_addr learned my addr 172.21.15.62:0/592653102 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b7be00640 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48acd10 0x5648b48c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.103+0000 7f6b7be00640 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 0x5648b48b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.104+0000 7f6b7aa00640 1 -- 172.21.15.62:0/592653102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48acd10 msgr2=0x5648b48c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.104+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48acd10 0x5648b48c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.104+0000 7f6b7aa00640 1 -- 172.21.15.62:0/592653102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 msgr2=0x5648b48b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.104+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 0x5648b48b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.104+0000 7f6b7aa00640 1 -- 172.21.15.62:0/592653102 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b48b0bb0 con 0x5648b48ad9f0 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.104+0000 7f6b7be00640 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 0x5648b48b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.104+0000 7f6b7aa00640 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48ad9f0 0x5648b48c6dd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800a010 tx=0x7f6b680051e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc651daf336f3843 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.104+0000 7f6b65600640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.104+0000 7f6b65600640 1 -- 172.21.15.62:0/592653102 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6800f920 con 0x5648b48ad9f0 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.104+0000 7f6b853eb740 1 -- 172.21.15.62:0/592653102 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b68005030 con 0x5648b48ad9f0 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.104+0000 7f6b65600640 1 -- 172.21.15.62:0/592653102 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f6b68018070 con 0x5648b48ad9f0 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.105+0000 7f6b65600640 1 -- 172.21.15.62:0/592653102 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b6800fac0 con 0x5648b48ad9f0 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.105+0000 7f6b853eb740 10 client.25829 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.105+0000 7f6b853eb740 1 -- 172.21.15.62:0/592653102 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5648b48b7100 con 0x5648b48ad9f0 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.105+0000 7f6b65600640 1 -- 172.21.15.62:0/592653102 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6b68005030 con 0x5648b48ad9f0 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.105+0000 7f6b65600640 1 -- 172.21.15.62:0/592653102 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6b68004b30 con 0x5648b48ad9f0 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.105+0000 7f6b853eb740 10 client.25829 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.105+0000 7f6b853eb740 10 client.25829 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.105+0000 7f6b853eb740 20 client.25829 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.106+0000 7f6b853eb740 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5648b4935410 0x5648b4937830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.106+0000 7f6b853eb740 4 client.25829 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:45:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.106+0000 7f6b853eb740 1 -- 172.21.15.62:0/592653102 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648b481ec30 con 0x5648b4935410 2024-04-02T14:45:16.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.106+0000 7f6b7be00640 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5648b4935410 0x5648b4937830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.106+0000 7f6b7be00640 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5648b4935410 0x5648b4937830 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9b830402ff3ac4 server_cookie=d5fb470af0009629 in_seq=0 out_seq=0 2024-04-02T14:45:16.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.106+0000 7f6b65600640 10 client.25829 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:45:16.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b65600640 1 -- 172.21.15.62:0/592653102 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648b481ec30 con 0x5648b4935410 2024-04-02T14:45:16.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b65600640 10 client.25829 handle_command_reply: tid=0 2024-04-02T14:45:16.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b853eb740 1 client.25829 shutdown 2024-04-02T14:45:16.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b853eb740 1 -- 172.21.15.62:0/592653102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48ad9f0 msgr2=0x5648b48c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b853eb740 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48ad9f0 0x5648b48c6dd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800a010 tx=0x7f6b680051e0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b853eb740 1 -- 172.21.15.62:0/592653102 shutdown_connections 2024-04-02T14:45:16.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b853eb740 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5648b4935410 0x5648b4937830 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b853eb740 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648b48b6080 0x5648b48b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b65600640 0 client.25829 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:45:16.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b853eb740 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648b48ad9f0 0x5648b48c6dd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b853eb740 1 --2- 172.21.15.62:0/592653102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648b48acd10 0x5648b48c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b853eb740 1 -- 172.21.15.62:0/592653102 >> 172.21.15.62:0/592653102 conn(0x5648b4873a50 msgr2=0x5648b48b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.107+0000 7f6b853eb740 1 -- 172.21.15.62:0/592653102 shutdown_connections 2024-04-02T14:45:16.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b853eb740 1 -- 172.21.15.62:0/592653102 wait complete. 2024-04-02T14:45:16.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b853eb740 20 client.25829 trim_cache size 0 max 16384 2024-04-02T14:45:16.108 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:45:16.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b82600640 1 -- 172.21.15.62:0/951286601 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f6b600a3ed0 msgr2=0x7f6b600a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b82600640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f6b600a3ed0 0x7f6b600a6390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0f0070 tx=0x7f6b6c005610 comp rx=0 tx=0).stop 2024-04-02T14:45:16.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b82600640 1 -- 172.21.15.62:0/951286601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c06a5b0 msgr2=0x7f6b7c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b82600640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c06a5b0 0x7f6b7c19a690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c005ab0 tx=0x7f6b6c02dc90 comp rx=0 tx=0).stop 2024-04-02T14:45:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b82600640 1 -- 172.21.15.62:0/951286601 shutdown_connections 2024-04-02T14:45:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b82600640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f6b600a3ed0 0x7f6b600a6390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b82600640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b7c10e350 0x7f6b7c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b82600640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7c1029c0 0x7f6b7c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b82600640 1 --2- 172.21.15.62:0/951286601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c06a5b0 0x7f6b7c19a690 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.108+0000 7f6b82600640 1 -- 172.21.15.62:0/951286601 >> 172.21.15.62:0/951286601 conn(0x7f6b7c0779e0 msgr2=0x7f6b7c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.109+0000 7f6b82600640 1 -- 172.21.15.62:0/951286601 shutdown_connections 2024-04-02T14:45:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.109+0000 7f6b82600640 1 -- 172.21.15.62:0/951286601 wait complete. 2024-04-02T14:45:16.122 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:45:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.338+0000 7f709f200640 1 -- 172.21.15.62:0/3365425905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a0073ed0 msgr2=0x7f70a00742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.338+0000 7f709f200640 1 --2- 172.21.15.62:0/3365425905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a0073ed0 0x7f70a00742b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7090009900 tx=0x7f709002bcd0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.338+0000 7f709f200640 1 -- 172.21.15.62:0/3365425905 shutdown_connections 2024-04-02T14:45:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.338+0000 7f709f200640 1 --2- 172.21.15.62:0/3365425905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a010e210 0x7f70a0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.338+0000 7f709f200640 1 --2- 172.21.15.62:0/3365425905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a00747f0 0x7f70a010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.338+0000 7f709f200640 1 --2- 172.21.15.62:0/3365425905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a0073ed0 0x7f70a00742b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.338+0000 7f709f200640 1 -- 172.21.15.62:0/3365425905 >> 172.21.15.62:0/3365425905 conn(0x7f70a00fd4a0 msgr2=0x7f70a00ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.339+0000 7f709f200640 1 -- 172.21.15.62:0/3365425905 shutdown_connections 2024-04-02T14:45:16.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.339+0000 7f709f200640 1 -- 172.21.15.62:0/3365425905 wait complete. 2024-04-02T14:45:16.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.340+0000 7f709f200640 1 Processor -- start 2024-04-02T14:45:16.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.340+0000 7f709f200640 1 -- start start 2024-04-02T14:45:16.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.341+0000 7f709f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a0073ed0 0x7f70a019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.341+0000 7f709f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a00747f0 0x7f70a019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.341+0000 7f709f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010e210 0x7f70a019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.341+0000 7f709f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f70a01140d0 con 0x7f70a010e210 2024-04-02T14:45:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.341+0000 7f709f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f70a0113f50 con 0x7f70a0073ed0 2024-04-02T14:45:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.341+0000 7f709f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f70a0114250 con 0x7f70a00747f0 2024-04-02T14:45:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.341+0000 7f709d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a00747f0 0x7f70a019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.341+0000 7f709de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a0073ed0 0x7f70a019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.341+0000 7f709d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a00747f0 0x7f70a019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55528/0 (socket says 172.21.15.62:55528) 2024-04-02T14:45:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.341+0000 7f709d400640 1 -- 172.21.15.62:0/1583563793 learned_addr learned my addr 172.21.15.62:0/1583563793 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.341+0000 7f709e800640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010e210 0x7f70a019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.342+0000 7f709d400640 1 -- 172.21.15.62:0/1583563793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a0073ed0 msgr2=0x7f70a019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.342+0000 7f709d400640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a0073ed0 0x7f70a019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.342+0000 7f709d400640 1 -- 172.21.15.62:0/1583563793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010e210 msgr2=0x7f70a019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.342+0000 7f709d400640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010e210 0x7f70a019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.342+0000 7f709d400640 1 -- 172.21.15.62:0/1583563793 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a019f600 con 0x7f70a00747f0 2024-04-02T14:45:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.342+0000 7f709e800640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010e210 0x7f70a019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:45:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.342+0000 7f709d400640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a00747f0 0x7f70a019ab50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f708c00d950 tx=0x7f708c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.342+0000 7f7096a00640 1 -- 172.21.15.62:0/1583563793 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c017030 con 0x7f70a00747f0 2024-04-02T14:45:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.342+0000 7f7096a00640 1 -- 172.21.15.62:0/1583563793 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f708c004020 con 0x7f70a00747f0 2024-04-02T14:45:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.342+0000 7f709f200640 1 -- 172.21.15.62:0/1583563793 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a019f8f0 con 0x7f70a00747f0 2024-04-02T14:45:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.343+0000 7f7096a00640 1 -- 172.21.15.62:0/1583563793 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c004a00 con 0x7f70a00747f0 2024-04-02T14:45:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.343+0000 7f709f200640 1 -- 172.21.15.62:0/1583563793 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a01a7180 con 0x7f70a00747f0 2024-04-02T14:45:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.347+0000 7f70a7fb5740 1 Processor -- start 2024-04-02T14:45:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.348+0000 7f70a7fb5740 1 -- start start 2024-04-02T14:45:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.348+0000 7f7096a00640 1 -- 172.21.15.62:0/1583563793 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f708c0041d0 con 0x7f70a00747f0 2024-04-02T14:45:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.348+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.348+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897c0ec0 0x559a897c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.348+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897ca230 0x559a897cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.348+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559a89701400 con 0x559a897c1ba0 2024-04-02T14:45:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.348+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559a896e5f10 con 0x559a897c0ec0 2024-04-02T14:45:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.348+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559a897256f0 con 0x559a897ca230 2024-04-02T14:45:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.349+0000 7f709e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.349+0000 7f709e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60170/0 (socket says 172.21.15.62:60170) 2024-04-02T14:45:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.349+0000 7f709e800640 1 -- 172.21.15.62:0/2933165289 learned_addr learned my addr 172.21.15.62:0/2933165289 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.349+0000 7f709de00640 1 --2- 172.21.15.62:0/2933165289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897c0ec0 0x559a897c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.349+0000 7f709d400640 1 --2- 172.21.15.62:0/2933165289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897ca230 0x559a897cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.349+0000 7f709e800640 1 -- 172.21.15.62:0/2933165289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897ca230 msgr2=0x559a897cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.349+0000 7f709e800640 1 --2- 172.21.15.62:0/2933165289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897ca230 0x559a897cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.349+0000 7f709e800640 1 -- 172.21.15.62:0/2933165289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897c0ec0 msgr2=0x559a897c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.349+0000 7f709e800640 1 --2- 172.21.15.62:0/2933165289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897c0ec0 0x559a897c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.349+0000 7f709e800640 1 -- 172.21.15.62:0/2933165289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a894393f0 con 0x559a897c1ba0 2024-04-02T14:45:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.350+0000 7f709e800640 1 --2- 172.21.15.62:0/2933165289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c08f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7098009eb0 tx=0x7f709800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a72a26fae4355b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.350+0000 7f7096a00640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f70840a3fd0 0x7f70840a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.350+0000 7f708b400640 1 -- 172.21.15.62:0/2933165289 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098003b70 con 0x559a897c1ba0 2024-04-02T14:45:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.350+0000 7f708b400640 1 -- 172.21.15.62:0/2933165289 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7098003d10 con 0x559a897c1ba0 2024-04-02T14:45:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.351+0000 7f708b400640 1 -- 172.21.15.62:0/2933165289 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098011a60 con 0x559a897c1ba0 2024-04-02T14:45:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.351+0000 7f7096a00640 1 -- 172.21.15.62:0/1583563793 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f708c0d0410 con 0x7f70a00747f0 2024-04-02T14:45:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.351+0000 7f709de00640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f70840a3fd0 0x7f70840a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.351+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2933165289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 msgr2=0x559a897c08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.351+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/2933165289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c08f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7098009eb0 tx=0x7f709800f730 comp rx=0 tx=0).stop 2024-04-02T14:45:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.352+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2933165289 shutdown_connections 2024-04-02T14:45:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.352+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/2933165289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897ca230 0x559a897cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.352+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/2933165289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897c0ec0 0x559a897c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.352+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/2933165289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c08f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.352+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2933165289 >> 172.21.15.62:0/2933165289 conn(0x559a89787a50 msgr2=0x559a897bb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.352+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2933165289 shutdown_connections 2024-04-02T14:45:16.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.353+0000 7f709de00640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f70840a3fd0 0x7f70840a6490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7090005ab0 tx=0x7f7090005fd0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.353+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2933165289 wait complete. 2024-04-02T14:45:16.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.353+0000 7f70a7fb5740 1 -- start start 2024-04-02T14:45:16.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.354+0000 7f70a7fb5740 1 Processor -- start 2024-04-02T14:45:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.354+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.354+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.354+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 0x559a897c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.354+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559a897216a0 con 0x559a897c1ba0 2024-04-02T14:45:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.354+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559a896cd980 con 0x559a897ca230 2024-04-02T14:45:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.354+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559a896e0cb0 con 0x559a897c0ec0 2024-04-02T14:45:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.354+0000 7f70a7fb5740 10 client.0 _ll_register_callbacks cb 0x559a896e06a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.355+0000 7f709de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.355+0000 7f709de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55536/0 (socket says 172.21.15.62:55536) 2024-04-02T14:45:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.355+0000 7f709de00640 1 -- 172.21.15.62:0/2531601662 learned_addr learned my addr 172.21.15.62:0/2531601662 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.355+0000 7f709e800640 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 0x559a897c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.355+0000 7f709d400640 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.355+0000 7f709e800640 1 -- 172.21.15.62:0/2531601662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 msgr2=0x559a897c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.356+0000 7f709e800640 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.356+0000 7f709e800640 1 -- 172.21.15.62:0/2531601662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 msgr2=0x559a897c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.356+0000 7f709e800640 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.356+0000 7f709e800640 1 -- 172.21.15.62:0/2531601662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a895037d0 con 0x559a897ca230 2024-04-02T14:45:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.356+0000 7f709d400640 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.356+0000 7f709e800640 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 0x559a897c7c90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f709800fc30 tx=0x7f7098003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d09a87816151ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.356+0000 7f707fe00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.356+0000 7f707fe00640 1 -- 172.21.15.62:0/2531601662 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098009b00 con 0x559a897ca230 2024-04-02T14:45:16.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.356+0000 7f707fe00640 1 -- 172.21.15.62:0/2531601662 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f7098004c50 con 0x559a897ca230 2024-04-02T14:45:16.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.357+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2531601662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7098003d10 con 0x559a897ca230 2024-04-02T14:45:16.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.357+0000 7f707fe00640 1 -- 172.21.15.62:0/2531601662 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7098004e50 con 0x559a897ca230 2024-04-02T14:45:16.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.357+0000 7f70a7fb5740 10 client.25841 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:16.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.357+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2531601662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a897e18e0 con 0x559a897ca230 2024-04-02T14:45:16.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.357+0000 7f707fe00640 1 -- 172.21.15.62:0/2531601662 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7098003d10 con 0x559a897ca230 2024-04-02T14:45:16.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.358+0000 7f707fe00640 1 -- 172.21.15.62:0/2531601662 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f70980045b0 con 0x559a897ca230 2024-04-02T14:45:16.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.358+0000 7f70a7fb5740 10 client.25841 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:16.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.358+0000 7f70a7fb5740 10 client.25841 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:45:16.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.358+0000 7f70a7fb5740 20 client.25841 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:16.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.358+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559a897e2c30 0x559a897e5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.358+0000 7f70a7fb5740 4 client.25841 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:16.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.358+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2531601662 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a896e5040 con 0x559a897e2c30 2024-04-02T14:45:16.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.358+0000 7f709de00640 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559a897e2c30 0x559a897e5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.359+0000 7f709de00640 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559a897e2c30 0x559a897e5020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9aff29d160533a1 server_cookie=52457263d19d97e5 in_seq=0 out_seq=0 2024-04-02T14:45:16.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.359+0000 7f707fe00640 10 client.25841 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:45:16.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.360+0000 7f707fe00640 1 -- 172.21.15.62:0/2531601662 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a896e5040 con 0x559a897e2c30 2024-04-02T14:45:16.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.360+0000 7f707fe00640 10 client.25841 handle_command_reply: tid=0 2024-04-02T14:45:16.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.361+0000 7f70a7fb5740 1 client.25841 shutdown 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2531601662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 msgr2=0x559a897c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 0x559a897c7c90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f709800fc30 tx=0x7f7098003ba0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2531601662 shutdown_connections 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559a897e2c30 0x559a897e5020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 0x559a897c7c90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f707fe00640 0 client.25841 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/2531601662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2531601662 >> 172.21.15.62:0/2531601662 conn(0x559a89787a50 msgr2=0x559a897bd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2531601662 shutdown_connections 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f70a7fb5740 1 -- 172.21.15.62:0/2531601662 wait complete. 2024-04-02T14:45:16.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.363+0000 7f70a7fb5740 20 client.25841 trim_cache size 0 max 16384 2024-04-02T14:45:16.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.390+0000 7f70a7fb5740 1 Processor -- start 2024-04-02T14:45:16.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.390+0000 7f70a7fb5740 1 -- start start 2024-04-02T14:45:16.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897c9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 0x559a897bd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f70a01ab410 con 0x559a897c1ba0 2024-04-02T14:45:16.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f70a01acf50 con 0x559a897ca230 2024-04-02T14:45:16.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f70a01abae0 con 0x559a897c0ec0 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f709d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f709e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 0x559a897bd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f709e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 0x559a897bd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37218/0 (socket says 172.21.15.62:37218) 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f709d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60188/0 (socket says 172.21.15.62:60188) 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f709de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.391+0000 7f709e800640 1 -- 172.21.15.62:0/897063500 learned_addr learned my addr 172.21.15.62:0/897063500 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.392+0000 7f709e800640 1 -- 172.21.15.62:0/897063500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 msgr2=0x559a897c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.392+0000 7f709e800640 1 --2- 172.21.15.62:0/897063500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.392+0000 7f709e800640 1 -- 172.21.15.62:0/897063500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 msgr2=0x559a897c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.392+0000 7f709e800640 1 --2- 172.21.15.62:0/897063500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.392+0000 7f709e800640 1 -- 172.21.15.62:0/897063500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a897bfef0 con 0x559a897ca230 2024-04-02T14:45:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.392+0000 7f709d400640 1 --2- 172.21.15.62:0/897063500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:16.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.392+0000 7f709e800640 1 --2- 172.21.15.62:0/897063500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 0x559a897bd400 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f709800f770 tx=0x7f7098004e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77844cbd3f84ea1e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.392+0000 7f708aa00640 1 -- 172.21.15.62:0/897063500 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098032070 con 0x559a897ca230 2024-04-02T14:45:16.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.392+0000 7f708aa00640 1 -- 172.21.15.62:0/897063500 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7098005200 con 0x559a897ca230 2024-04-02T14:45:16.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.392+0000 7f708aa00640 1 -- 172.21.15.62:0/897063500 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098011cb0 con 0x559a897ca230 2024-04-02T14:45:16.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.393+0000 7f70a7fb5740 1 -- 172.21.15.62:0/897063500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 msgr2=0x559a897bd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.393+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/897063500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 0x559a897bd400 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f709800f770 tx=0x7f7098004e60 comp rx=0 tx=0).stop 2024-04-02T14:45:16.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.393+0000 7f70a7fb5740 1 -- 172.21.15.62:0/897063500 shutdown_connections 2024-04-02T14:45:16.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.393+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/897063500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897ca230 0x559a897bd400 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.393+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/897063500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897c1ba0 0x559a897c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.393+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/897063500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.393+0000 7f70a7fb5740 1 -- 172.21.15.62:0/897063500 >> 172.21.15.62:0/897063500 conn(0x559a89787a50 msgr2=0x559a897bccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.394+0000 7f70a7fb5740 1 -- 172.21.15.62:0/897063500 shutdown_connections 2024-04-02T14:45:16.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.394+0000 7f70a7fb5740 1 -- 172.21.15.62:0/897063500 wait complete. 2024-04-02T14:45:16.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.394+0000 7f70a7fb5740 1 -- start start 2024-04-02T14:45:16.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.394+0000 7f70a7fb5740 1 Processor -- start 2024-04-02T14:45:16.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.395+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897e3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.395+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897c1ba0 0x559a897e3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.395+0000 7f70a7fb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897ca230 0x559a89846df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.395+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f70a01ac880 con 0x559a897ca230 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.395+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f70a01ac1b0 con 0x559a897c1ba0 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.395+0000 7f70a7fb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f70a01a93f0 con 0x559a897c0ec0 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.395+0000 7f70a7fb5740 10 client.0 _ll_register_callbacks cb 0x7f708c0d3d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.395+0000 7f709e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897ca230 0x559a89846df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.395+0000 7f709d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897c1ba0 0x559a897e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.395+0000 7f709e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897ca230 0x559a89846df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60204/0 (socket says 172.21.15.62:60204) 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.395+0000 7f709e800640 1 -- 172.21.15.62:0/839346075 learned_addr learned my addr 172.21.15.62:0/839346075 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.396+0000 7f709de00640 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897e3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.396+0000 7f709e800640 1 -- 172.21.15.62:0/839346075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 msgr2=0x559a897e3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.396+0000 7f709e800640 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897e3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.396+0000 7f709e800640 1 -- 172.21.15.62:0/839346075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897c1ba0 msgr2=0x559a897e3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.396+0000 7f709e800640 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897c1ba0 0x559a897e3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.396+0000 7f709e800640 1 -- 172.21.15.62:0/839346075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a898473e0 con 0x559a897ca230 2024-04-02T14:45:16.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.396+0000 7f709d400640 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897c1ba0 0x559a897e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:16.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.396+0000 7f709e800640 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897ca230 0x559a89846df0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7098014040 tx=0x7f7098011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfeb3d26da510b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.396+0000 7f707fe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.397+0000 7f707fe00640 1 -- 172.21.15.62:0/839346075 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709803b060 con 0x559a897ca230 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.397+0000 7f707fe00640 1 -- 172.21.15.62:0/839346075 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f7098005650 con 0x559a897ca230 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.397+0000 7f70a7fb5740 1 -- 172.21.15.62:0/839346075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7098005200 con 0x559a897ca230 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.397+0000 7f707fe00640 1 -- 172.21.15.62:0/839346075 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7098032040 con 0x559a897ca230 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.397+0000 7f70a7fb5740 10 client.15723 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.397+0000 7f70a7fb5740 1 -- 172.21.15.62:0/839346075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a898486c0 con 0x559a897ca230 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.397+0000 7f707fe00640 1 -- 172.21.15.62:0/839346075 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7098005200 con 0x559a897ca230 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.398+0000 7f707fe00640 1 -- 172.21.15.62:0/839346075 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f709802eab0 con 0x559a897ca230 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.398+0000 7f70a7fb5740 10 client.15723 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.398+0000 7f70a7fb5740 10 client.15723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.398+0000 7f70a7fb5740 20 client.15723 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.398+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559a89848d20 0x559a8984b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.398+0000 7f70a7fb5740 4 client.15723 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.398+0000 7f70a7fb5740 1 -- 172.21.15.62:0/839346075 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a89732c30 con 0x559a89848d20 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.398+0000 7f709de00640 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559a89848d20 0x559a8984b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f709de00640 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559a89848d20 0x559a8984b140 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8eafa8d1a22c5ef6 server_cookie=9565db164e903ccb in_seq=0 out_seq=0 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f707fe00640 10 client.15723 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f707fe00640 1 -- 172.21.15.62:0/839346075 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a89732c30 con 0x559a89848d20 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f707fe00640 10 client.15723 handle_command_reply: tid=0 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f70a7fb5740 1 client.15723 shutdown 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f70a7fb5740 1 -- 172.21.15.62:0/839346075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897ca230 msgr2=0x559a89846df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897ca230 0x559a89846df0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7098014040 tx=0x7f7098011d50 comp rx=0 tx=0).stop 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f70a7fb5740 1 -- 172.21.15.62:0/839346075 shutdown_connections 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559a89848d20 0x559a8984b140 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a897ca230 0x559a89846df0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a897c1ba0 0x559a897e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.399+0000 7f70a7fb5740 1 --2- 172.21.15.62:0/839346075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a897c0ec0 0x559a897e3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.400+0000 7f70a7fb5740 1 -- 172.21.15.62:0/839346075 >> 172.21.15.62:0/839346075 conn(0x559a89787a50 msgr2=0x559a897bccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.400+0000 7f707fe00640 0 client.15723 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:45:16.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.400+0000 7f70a7fb5740 1 -- 172.21.15.62:0/839346075 shutdown_connections 2024-04-02T14:45:16.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.400+0000 7f70a7fb5740 1 -- 172.21.15.62:0/839346075 wait complete. 2024-04-02T14:45:16.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.400+0000 7f70a7fb5740 20 client.15723 trim_cache size 0 max 16384 2024-04-02T14:45:16.399 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 -- 172.21.15.62:0/1583563793 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f70840a3fd0 msgr2=0x7f70840a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f70840a3fd0 0x7f70840a6490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7090005ab0 tx=0x7f7090005fd0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 -- 172.21.15.62:0/1583563793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a00747f0 msgr2=0x7f70a019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a00747f0 0x7f70a019ab50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f708c00d950 tx=0x7f708c00de10 comp rx=0 tx=0).stop 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 -- 172.21.15.62:0/1583563793 shutdown_connections 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f70840a3fd0 0x7f70840a6490 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010e210 0x7f70a019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a00747f0 0x7f70a019ab50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 --2- 172.21.15.62:0/1583563793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a0073ed0 0x7f70a019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 -- 172.21.15.62:0/1583563793 >> 172.21.15.62:0/1583563793 conn(0x7f70a00fd4a0 msgr2=0x7f70a00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 -- 172.21.15.62:0/1583563793 shutdown_connections 2024-04-02T14:45:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.401+0000 7f709f200640 1 -- 172.21.15.62:0/1583563793 wait complete. 2024-04-02T14:45:16.422 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:45:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.610+0000 7f941cc00640 1 -- 172.21.15.62:0/3952518693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f941810e4c0 msgr2=0x7f94181108b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.610+0000 7f941cc00640 1 --2- 172.21.15.62:0/3952518693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f941810e4c0 0x7f94181108b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9410009ca0 tx=0x7f941002c0f0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.610+0000 7f941cc00640 1 -- 172.21.15.62:0/3952518693 shutdown_connections 2024-04-02T14:45:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.610+0000 7f941cc00640 1 --2- 172.21.15.62:0/3952518693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f941810e4c0 0x7f94181108b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.610+0000 7f941cc00640 1 --2- 172.21.15.62:0/3952518693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418101230 0x7f941810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.610+0000 7f941cc00640 1 --2- 172.21.15.62:0/3952518693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9418100910 0x7f9418100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.610+0000 7f941cc00640 1 -- 172.21.15.62:0/3952518693 >> 172.21.15.62:0/3952518693 conn(0x7f9418077b60 msgr2=0x7f9418077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.610+0000 7f941cc00640 1 -- 172.21.15.62:0/3952518693 shutdown_connections 2024-04-02T14:45:16.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.610+0000 7f941cc00640 1 -- 172.21.15.62:0/3952518693 wait complete. 2024-04-02T14:45:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.611+0000 7f941cc00640 1 Processor -- start 2024-04-02T14:45:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.612+0000 7f941cc00640 1 -- start start 2024-04-02T14:45:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.612+0000 7f941cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9418100910 0x7f941819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.612+0000 7f941cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418101230 0x7f941819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.612+0000 7f941cc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f941810e4c0 0x7f941819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.612+0000 7f941cc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f94181141f0 con 0x7f9418101230 2024-04-02T14:45:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.612+0000 7f941cc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9418114070 con 0x7f9418100910 2024-04-02T14:45:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.612+0000 7f941cc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9418114370 con 0x7f941810e4c0 2024-04-02T14:45:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.612+0000 7f9415e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9418100910 0x7f941819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.612+0000 7f9415e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9418100910 0x7f941819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37234/0 (socket says 172.21.15.62:37234) 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.612+0000 7f9415e00640 1 -- 172.21.15.62:0/942515022 learned_addr learned my addr 172.21.15.62:0/942515022 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f9415400640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418101230 0x7f941819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f9416800640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f941810e4c0 0x7f941819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f9415e00640 1 -- 172.21.15.62:0/942515022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f941810e4c0 msgr2=0x7f941819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f9415e00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f941810e4c0 0x7f941819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f9415e00640 1 -- 172.21.15.62:0/942515022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418101230 msgr2=0x7f941819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f9415e00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418101230 0x7f941819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f9415e00640 1 -- 172.21.15.62:0/942515022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941819f660 con 0x7f9418100910 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f9415400640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418101230 0x7f941819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f9415e00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9418100910 0x7f941819a670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9f0 tx=0x7f940800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f940ea00640 1 -- 172.21.15.62:0/942515022 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940800cd80 con 0x7f9418100910 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f940ea00640 1 -- 172.21.15.62:0/942515022 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9408005520 con 0x7f9418100910 2024-04-02T14:45:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.613+0000 7f940ea00640 1 -- 172.21.15.62:0/942515022 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408010690 con 0x7f9418100910 2024-04-02T14:45:16.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.614+0000 7f941cc00640 1 -- 172.21.15.62:0/942515022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941819f950 con 0x7f9418100910 2024-04-02T14:45:16.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.614+0000 7f941cc00640 1 -- 172.21.15.62:0/942515022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94181a71e0 con 0x7f9418100910 2024-04-02T14:45:16.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.618+0000 7f941f802740 1 Processor -- start 2024-04-02T14:45:16.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.618+0000 7f940ea00640 1 -- 172.21.15.62:0/942515022 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f94080046c0 con 0x7f9418100910 2024-04-02T14:45:16.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f941f802740 1 -- start start 2024-04-02T14:45:16.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f941f802740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef801079f0 0x55ef80106740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f941f802740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80106d10 0x55ef80107190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f941f802740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef80112470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f941f802740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ef80047400 con 0x55ef80106d10 2024-04-02T14:45:16.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f941f802740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ef8002bf10 con 0x55ef80110080 2024-04-02T14:45:16.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f941f802740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ef8006b6f0 con 0x55ef801079f0 2024-04-02T14:45:16.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f9416800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef801079f0 0x55ef80106740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f9416800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef801079f0 0x55ef80106740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55570/0 (socket says 172.21.15.62:55570) 2024-04-02T14:45:16.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f9416800640 1 -- 172.21.15.62:0/2627765241 learned_addr learned my addr 172.21.15.62:0/2627765241 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f9415400640 1 --2- 172.21.15.62:0/2627765241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef80112470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.619+0000 7f9415e00640 1 --2- 172.21.15.62:0/2627765241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80106d10 0x55ef80107190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.620+0000 7f9416800640 1 -- 172.21.15.62:0/2627765241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 msgr2=0x55ef80112470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.620+0000 7f9416800640 1 --2- 172.21.15.62:0/2627765241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef80112470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.620+0000 7f9416800640 1 -- 172.21.15.62:0/2627765241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80106d10 msgr2=0x55ef80107190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.620+0000 7f9416800640 1 --2- 172.21.15.62:0/2627765241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80106d10 0x55ef80107190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.620+0000 7f9416800640 1 -- 172.21.15.62:0/2627765241 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef7fd7f3f0 con 0x55ef801079f0 2024-04-02T14:45:16.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.620+0000 7f9415e00640 1 --2- 172.21.15.62:0/2627765241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80106d10 0x55ef80107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:16.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.620+0000 7f9416800640 1 --2- 172.21.15.62:0/2627765241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef801079f0 0x55ef80106740 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9410005e30 tx=0x7f9410009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b128b7430b8f88a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.621+0000 7f940ea00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f93fc0a3f50 0x7f93fc0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.621+0000 7f9403000640 1 -- 172.21.15.62:0/2627765241 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410043070 con 0x55ef801079f0 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.621+0000 7f9403000640 1 -- 172.21.15.62:0/2627765241 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9410032340 con 0x55ef801079f0 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.621+0000 7f9403000640 1 -- 172.21.15.62:0/2627765241 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410039040 con 0x55ef801079f0 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.621+0000 7f940ea00640 1 -- 172.21.15.62:0/942515022 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f94080d0ed0 con 0x7f9418100910 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.621+0000 7f9415e00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f93fc0a3f50 0x7f93fc0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.622+0000 7f941f802740 1 -- 172.21.15.62:0/2627765241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef801079f0 msgr2=0x55ef80106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.622+0000 7f941f802740 1 --2- 172.21.15.62:0/2627765241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef801079f0 0x55ef80106740 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9410005e30 tx=0x7f9410009880 comp rx=0 tx=0).stop 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.622+0000 7f9415e00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f93fc0a3f50 0x7f93fc0a6410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f94080ce490 tx=0x7f9408004c10 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.622+0000 7f941f802740 1 -- 172.21.15.62:0/2627765241 shutdown_connections 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.622+0000 7f941f802740 1 --2- 172.21.15.62:0/2627765241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef80112470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.622+0000 7f941f802740 1 --2- 172.21.15.62:0/2627765241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80106d10 0x55ef80107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.622+0000 7f941f802740 1 --2- 172.21.15.62:0/2627765241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef801079f0 0x55ef80106740 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.622+0000 7f941f802740 1 -- 172.21.15.62:0/2627765241 >> 172.21.15.62:0/2627765241 conn(0x55ef800cda50 msgr2=0x55ef80101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.622+0000 7f941f802740 1 -- 172.21.15.62:0/2627765241 shutdown_connections 2024-04-02T14:45:16.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.622+0000 7f941f802740 1 -- 172.21.15.62:0/2627765241 wait complete. 2024-04-02T14:45:16.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.623+0000 7f941f802740 1 -- start start 2024-04-02T14:45:16.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.623+0000 7f941f802740 1 Processor -- start 2024-04-02T14:45:16.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.624+0000 7f941f802740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef8010d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.624+0000 7f941f802740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 0x55ef8010d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.624+0000 7f941f802740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef8010daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.624+0000 7f941f802740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ef800676a0 con 0x55ef801079f0 2024-04-02T14:45:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.624+0000 7f941f802740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ef80013980 con 0x55ef80110080 2024-04-02T14:45:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.624+0000 7f941f802740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ef80026cb0 con 0x55ef80106d10 2024-04-02T14:45:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.624+0000 7f941f802740 10 client.0 _ll_register_callbacks cb 0x55ef800266a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.624+0000 7f9416800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef8010d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.624+0000 7f9416800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef8010d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55582/0 (socket says 172.21.15.62:55582) 2024-04-02T14:45:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.624+0000 7f9416800640 1 -- 172.21.15.62:0/2769130531 learned_addr learned my addr 172.21.15.62:0/2769130531 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.624+0000 7f9416800640 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef8010daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.625+0000 7f9416800640 1 -- 172.21.15.62:0/2769130531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 msgr2=0x55ef8010daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.625+0000 7f9416800640 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef8010daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.625+0000 7f9416800640 1 -- 172.21.15.62:0/2769130531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 msgr2=0x55ef8010d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.625+0000 7f9415400640 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 0x55ef8010d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.625+0000 7f9416800640 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 0x55ef8010d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.625+0000 7f9416800640 1 -- 172.21.15.62:0/2769130531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef7fe497d0 con 0x55ef80106d10 2024-04-02T14:45:16.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.625+0000 7f9415400640 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 0x55ef8010d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:45:16.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.625+0000 7f9416800640 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef8010d020 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9410004af0 tx=0x7f941002e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e29206eab7c3b9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.625+0000 7f93f7400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:45:16.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.625+0000 7f93f7400640 1 -- 172.21.15.62:0/2769130531 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941004c020 con 0x55ef80106d10 2024-04-02T14:45:16.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.625+0000 7f941f802740 1 -- 172.21.15.62:0/2769130531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9410032340 con 0x55ef80106d10 2024-04-02T14:45:16.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f93f7400640 1 -- 172.21.15.62:0/2769130531 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f9410032990 con 0x55ef80106d10 2024-04-02T14:45:16.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f93f7400640 1 -- 172.21.15.62:0/2769130531 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9410039070 con 0x55ef80106d10 2024-04-02T14:45:16.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f941f802740 10 client.25834 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:16.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f941f802740 1 -- 172.21.15.62:0/2769130531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef8012fee0 con 0x55ef80106d10 2024-04-02T14:45:16.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f93f7400640 1 -- 172.21.15.62:0/2769130531 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9410032340 con 0x55ef80106d10 2024-04-02T14:45:16.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f93f7400640 1 -- 172.21.15.62:0/2769130531 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9410048750 con 0x55ef80106d10 2024-04-02T14:45:16.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f941f802740 10 client.25834 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:16.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f941f802740 10 client.25834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:16.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f941f802740 20 client.25834 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:16.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f941f802740 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ef80131230 0x55ef80133620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f941f802740 4 client.25834 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.626+0000 7f941f802740 1 -- 172.21.15.62:0/2769130531 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef8002b040 con 0x55ef80131230 2024-04-02T14:45:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.627+0000 7f9415400640 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ef80131230 0x55ef80133620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.627+0000 7f9415400640 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ef80131230 0x55ef80133620 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8ac43fac8cfea6 server_cookie=ef0052b7c06f96d5 in_seq=0 out_seq=0 2024-04-02T14:45:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.627+0000 7f93f7400640 10 client.25834 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.628+0000 7f93f7400640 1 -- 172.21.15.62:0/2769130531 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef8002b040 con 0x55ef80131230 2024-04-02T14:45:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.628+0000 7f93f7400640 10 client.25834 handle_command_reply: tid=0 2024-04-02T14:45:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.628+0000 7f941f802740 1 client.25834 shutdown 2024-04-02T14:45:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.628+0000 7f941f802740 1 -- 172.21.15.62:0/2769130531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 msgr2=0x55ef8010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.629+0000 7f941f802740 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef8010d020 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9410004af0 tx=0x7f941002e040 comp rx=0 tx=0).stop 2024-04-02T14:45:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.629+0000 7f941f802740 1 -- 172.21.15.62:0/2769130531 shutdown_connections 2024-04-02T14:45:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.629+0000 7f941f802740 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ef80131230 0x55ef80133620 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.629+0000 7f941f802740 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef8010daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.629+0000 7f941f802740 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 0x55ef8010d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.629+0000 7f941f802740 1 --2- 172.21.15.62:0/2769130531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef8010d020 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.629+0000 7f941f802740 1 -- 172.21.15.62:0/2769130531 >> 172.21.15.62:0/2769130531 conn(0x55ef800cda50 msgr2=0x55ef801034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.629+0000 7f93f7400640 0 client.25834 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.629+0000 7f941f802740 1 -- 172.21.15.62:0/2769130531 shutdown_connections 2024-04-02T14:45:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.629+0000 7f941f802740 1 -- 172.21.15.62:0/2769130531 wait complete. 2024-04-02T14:45:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.629+0000 7f941f802740 20 client.25834 trim_cache size 0 max 16384 2024-04-02T14:45:16.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.658+0000 7f941f802740 1 Processor -- start 2024-04-02T14:45:16.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.658+0000 7f941f802740 1 -- start start 2024-04-02T14:45:16.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.658+0000 7f941f802740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef8010f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.658+0000 7f941f802740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 0x55ef80101290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.658+0000 7f941f802740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef8010c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.658+0000 7f941f802740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9418073b50 con 0x55ef801079f0 2024-04-02T14:45:16.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.658+0000 7f941f802740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94181a8d20 con 0x55ef80110080 2024-04-02T14:45:16.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.658+0000 7f941f802740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f94181a78b0 con 0x55ef80106d10 2024-04-02T14:45:16.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.659+0000 7f9415400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 0x55ef80101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.659+0000 7f9416800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef8010f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.659+0000 7f9415400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 0x55ef80101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60238/0 (socket says 172.21.15.62:60238) 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.659+0000 7f9415400640 1 -- 172.21.15.62:0/3384051756 learned_addr learned my addr 172.21.15.62:0/3384051756 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.659+0000 7f9416800640 1 --2- 172.21.15.62:0/3384051756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef8010c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.659+0000 7f9416800640 1 -- 172.21.15.62:0/3384051756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 msgr2=0x55ef8010c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.659+0000 7f9416800640 1 --2- 172.21.15.62:0/3384051756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef8010c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.659+0000 7f9416800640 1 -- 172.21.15.62:0/3384051756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 msgr2=0x55ef80101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.659+0000 7f9416800640 1 --2- 172.21.15.62:0/3384051756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 0x55ef80101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.659+0000 7f9416800640 1 -- 172.21.15.62:0/3384051756 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef80101970 con 0x55ef80106d10 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.660+0000 7f9416800640 1 --2- 172.21.15.62:0/3384051756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef8010f6e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9410048b20 tx=0x7f9410005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c38deb5426755b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.660+0000 7f9402600640 1 -- 172.21.15.62:0/3384051756 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941004c070 con 0x55ef80106d10 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.660+0000 7f9402600640 1 -- 172.21.15.62:0/3384051756 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9410048950 con 0x55ef80106d10 2024-04-02T14:45:16.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.660+0000 7f9402600640 1 -- 172.21.15.62:0/3384051756 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410043040 con 0x55ef80106d10 2024-04-02T14:45:16.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.661+0000 7f941f802740 1 -- 172.21.15.62:0/3384051756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 msgr2=0x55ef8010f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.661+0000 7f941f802740 1 --2- 172.21.15.62:0/3384051756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef8010f6e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9410048b20 tx=0x7f9410005d80 comp rx=0 tx=0).stop 2024-04-02T14:45:16.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.661+0000 7f941f802740 1 -- 172.21.15.62:0/3384051756 shutdown_connections 2024-04-02T14:45:16.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.661+0000 7f941f802740 1 --2- 172.21.15.62:0/3384051756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef80110080 0x55ef8010c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.661+0000 7f941f802740 1 --2- 172.21.15.62:0/3384051756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef801079f0 0x55ef80101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.661+0000 7f941f802740 1 --2- 172.21.15.62:0/3384051756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef8010f6e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.661+0000 7f941f802740 1 -- 172.21.15.62:0/3384051756 >> 172.21.15.62:0/3384051756 conn(0x55ef800cda50 msgr2=0x55ef80102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.661+0000 7f941f802740 1 -- 172.21.15.62:0/3384051756 shutdown_connections 2024-04-02T14:45:16.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.661+0000 7f941f802740 1 -- 172.21.15.62:0/3384051756 wait complete. 2024-04-02T14:45:16.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.661+0000 7f941f802740 1 -- start start 2024-04-02T14:45:16.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.662+0000 7f941f802740 1 Processor -- start 2024-04-02T14:45:16.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.662+0000 7f941f802740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef80109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.662+0000 7f941f802740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef801079f0 0x55ef801099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.662+0000 7f941f802740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80110080 0x55ef80109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.662+0000 7f941f802740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f94181a8650 con 0x55ef80110080 2024-04-02T14:45:16.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.662+0000 7f941f802740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94181a7f80 con 0x55ef801079f0 2024-04-02T14:45:16.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.662+0000 7f941f802740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9418071b30 con 0x55ef80106d10 2024-04-02T14:45:16.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.662+0000 7f941f802740 10 client.0 _ll_register_callbacks cb 0x7f9410009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:16.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.663+0000 7f9416800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80110080 0x55ef80109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.663+0000 7f9416800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80110080 0x55ef80109ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60240/0 (socket says 172.21.15.62:60240) 2024-04-02T14:45:16.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.663+0000 7f9415400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef801079f0 0x55ef801099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.663+0000 7f9416800640 1 -- 172.21.15.62:0/1981696598 learned_addr learned my addr 172.21.15.62:0/1981696598 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:16.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.663+0000 7f9416800640 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef80109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.663+0000 7f9415400640 1 -- 172.21.15.62:0/1981696598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 msgr2=0x55ef80109470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.663+0000 7f9415400640 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef80109470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.663+0000 7f9415400640 1 -- 172.21.15.62:0/1981696598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80110080 msgr2=0x55ef80109ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:16.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.663+0000 7f9415400640 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80110080 0x55ef80109ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.663+0000 7f9415400640 1 -- 172.21.15.62:0/1981696598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef80128280 con 0x55ef801079f0 2024-04-02T14:45:16.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.663+0000 7f9416800640 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80110080 0x55ef80109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:16.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.664+0000 7f9415400640 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef801079f0 0x55ef801099b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9404009460 tx=0x7f9404004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cf48f2303f0a878 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.664+0000 7f93f7400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.664+0000 7f93f7400640 1 -- 172.21.15.62:0/1981696598 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940401e380 con 0x55ef801079f0 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.664+0000 7f941f802740 1 -- 172.21.15.62:0/1981696598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9410048950 con 0x55ef801079f0 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.664+0000 7f93f7400640 1 -- 172.21.15.62:0/1981696598 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f940401e520 con 0x55ef801079f0 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.665+0000 7f93f7400640 1 -- 172.21.15.62:0/1981696598 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f940402d050 con 0x55ef801079f0 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.665+0000 7f941f802740 10 client.25865 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.665+0000 7f941f802740 1 -- 172.21.15.62:0/1981696598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef801295e0 con 0x55ef801079f0 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.665+0000 7f93f7400640 1 -- 172.21.15.62:0/1981696598 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9410048950 con 0x55ef801079f0 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.665+0000 7f93f7400640 1 -- 172.21.15.62:0/1981696598 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9404011440 con 0x55ef801079f0 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.665+0000 7f941f802740 10 client.25865 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.665+0000 7f941f802740 10 client.25865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.665+0000 7f941f802740 20 client.25865 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.665+0000 7f941f802740 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ef801976e0 0x55ef80199b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.666+0000 7f941f802740 4 client.25865 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:45:16.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.666+0000 7f941f802740 1 -- 172.21.15.62:0/1981696598 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef80078c30 con 0x55ef801976e0 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.666+0000 7f9416800640 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ef801976e0 0x55ef80199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.666+0000 7f9416800640 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ef801976e0 0x55ef80199b00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada56e7059583d54 server_cookie=c88f5171d01dfd85 in_seq=0 out_seq=0 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.666+0000 7f93f7400640 10 client.25865 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f93f7400640 1 -- 172.21.15.62:0/1981696598 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef80078c30 con 0x55ef801976e0 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f93f7400640 10 client.25865 handle_command_reply: tid=0 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f941f802740 1 client.25865 shutdown 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f941f802740 1 -- 172.21.15.62:0/1981696598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef801079f0 msgr2=0x55ef801099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f941f802740 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef801079f0 0x55ef801099b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9404009460 tx=0x7f9404004410 comp rx=0 tx=0).stop 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f941f802740 1 -- 172.21.15.62:0/1981696598 shutdown_connections 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f941f802740 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ef801976e0 0x55ef80199b00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f941f802740 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef80110080 0x55ef80109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f941f802740 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef801079f0 0x55ef801099b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f93f7400640 0 client.25865 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:16.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f941f802740 1 --2- 172.21.15.62:0/1981696598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef80106d10 0x55ef80109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f941f802740 1 -- 172.21.15.62:0/1981696598 >> 172.21.15.62:0/1981696598 conn(0x55ef800cda50 msgr2=0x55ef8010d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f941f802740 1 -- 172.21.15.62:0/1981696598 shutdown_connections 2024-04-02T14:45:16.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.667+0000 7f941f802740 1 -- 172.21.15.62:0/1981696598 wait complete. 2024-04-02T14:45:16.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.668+0000 7f941f802740 20 client.25865 trim_cache size 0 max 16384 2024-04-02T14:45:16.670 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 -- 172.21.15.62:0/942515022 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f93fc0a3f50 msgr2=0x7f93fc0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f93fc0a3f50 0x7f93fc0a6410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f94080ce490 tx=0x7f9408004c10 comp rx=0 tx=0).stop 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 -- 172.21.15.62:0/942515022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9418100910 msgr2=0x7f941819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9418100910 0x7f941819a670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9f0 tx=0x7f940800edf0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 -- 172.21.15.62:0/942515022 shutdown_connections 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f93fc0a3f50 0x7f93fc0a6410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f941810e4c0 0x7f941819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418101230 0x7f941819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 --2- 172.21.15.62:0/942515022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9418100910 0x7f941819a670 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 -- 172.21.15.62:0/942515022 >> 172.21.15.62:0/942515022 conn(0x7f9418077b60 msgr2=0x7f941810f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 -- 172.21.15.62:0/942515022 shutdown_connections 2024-04-02T14:45:16.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:16.669+0000 7f941cc00640 1 -- 172.21.15.62:0/942515022 wait complete. 2024-04-02T14:45:17.690 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:45:17.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:45:17.690 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:45:17.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:17 smithi204 ceph-mon[35948]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-02T14:45:17.789 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:17 smithi062 ceph-mon[32116]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-02T14:45:17.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:17 smithi113 ceph-mon[36135]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-02T14:45:17.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.988+0000 7f05dbe00640 1 -- 172.21.15.62:0/1955179295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05dc102b30 msgr2=0x7f05dc102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:17.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.988+0000 7f05dbe00640 1 --2- 172.21.15.62:0/1955179295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05dc102b30 0x7f05dc102f10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009900 tx=0x7f05cc02bcd0 comp rx=0 tx=0).stop 2024-04-02T14:45:17.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.988+0000 7f05dbe00640 1 -- 172.21.15.62:0/1955179295 shutdown_connections 2024-04-02T14:45:17.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.988+0000 7f05dbe00640 1 --2- 172.21.15.62:0/1955179295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc10bf50 0x7f05dc10e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:17.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.988+0000 7f05dbe00640 1 --2- 172.21.15.62:0/1955179295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05dc103450 0x7f05dc10b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:17.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.988+0000 7f05dbe00640 1 --2- 172.21.15.62:0/1955179295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05dc102b30 0x7f05dc102f10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:17.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.988+0000 7f05dbe00640 1 -- 172.21.15.62:0/1955179295 >> 172.21.15.62:0/1955179295 conn(0x7f05dc0fc350 msgr2=0x7f05dc0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:17.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.988+0000 7f05dbe00640 1 -- 172.21.15.62:0/1955179295 shutdown_connections 2024-04-02T14:45:17.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.988+0000 7f05dbe00640 1 -- 172.21.15.62:0/1955179295 wait complete. 2024-04-02T14:45:17.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.989+0000 7f05dbe00640 1 Processor -- start 2024-04-02T14:45:17.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.990+0000 7f05dbe00640 1 -- start start 2024-04-02T14:45:17.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.990+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05dc102b30 0x7f05dc1039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:17.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.990+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc103450 0x7f05dc103ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:17.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.990+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05dc10bf50 0x7f05dc104420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:17.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.990+0000 7f05dbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc111ec0 con 0x7f05dc103450 2024-04-02T14:45:17.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.990+0000 7f05dbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc111d40 con 0x7f05dc102b30 2024-04-02T14:45:17.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.990+0000 7f05dbe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc112040 con 0x7f05dc10bf50 2024-04-02T14:45:17.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.991+0000 7f05daa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05dc102b30 0x7f05dc1039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:17.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.991+0000 7f05daa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05dc102b30 0x7f05dc1039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37282/0 (socket says 172.21.15.62:37282) 2024-04-02T14:45:17.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.991+0000 7f05daa00640 1 -- 172.21.15.62:0/4108677349 learned_addr learned my addr 172.21.15.62:0/4108677349 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:17.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.991+0000 7f05da000640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc103450 0x7f05dc103ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:17.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.991+0000 7f05db400640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05dc10bf50 0x7f05dc104420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:17.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.991+0000 7f05daa00640 1 -- 172.21.15.62:0/4108677349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05dc10bf50 msgr2=0x7f05dc104420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:17.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.991+0000 7f05daa00640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05dc10bf50 0x7f05dc104420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:17.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.991+0000 7f05daa00640 1 -- 172.21.15.62:0/4108677349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc103450 msgr2=0x7f05dc103ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:17.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.991+0000 7f05daa00640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc103450 0x7f05dc103ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:17.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.991+0000 7f05daa00640 1 -- 172.21.15.62:0/4108677349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc1aabb0 con 0x7f05dc102b30 2024-04-02T14:45:17.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.992+0000 7f05daa00640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05dc102b30 0x7f05dc1039a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f05cc02c2e0 tx=0x7f05cc004950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:17.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.992+0000 7f05d3400640 1 -- 172.21.15.62:0/4108677349 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc043020 con 0x7f05dc102b30 2024-04-02T14:45:17.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.992+0000 7f05d3400640 1 -- 172.21.15.62:0/4108677349 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f05cc004250 con 0x7f05dc102b30 2024-04-02T14:45:17.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.992+0000 7f05d3400640 1 -- 172.21.15.62:0/4108677349 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc003ab0 con 0x7f05dc102b30 2024-04-02T14:45:17.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.993+0000 7f05dbe00640 1 -- 172.21.15.62:0/4108677349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05dc1aae40 con 0x7f05dc102b30 2024-04-02T14:45:17.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.993+0000 7f05dbe00640 1 -- 172.21.15.62:0/4108677349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05dc1ab2b0 con 0x7f05dc102b30 2024-04-02T14:45:17.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.997+0000 7f05e4424740 1 Processor -- start 2024-04-02T14:45:17.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.998+0000 7f05e4424740 1 -- start start 2024-04-02T14:45:17.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.998+0000 7f05e4424740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x55863052cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.998+0000 7f05e4424740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55863052d510 0x55863052d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.998+0000 7f05e4424740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558630536880 0x558630538c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.998+0000 7f05e4424740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558630491310 con 0x55863052d510 2024-04-02T14:45:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.998+0000 7f05e4424740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558630439690 con 0x558630536880 2024-04-02T14:45:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.998+0000 7f05e4424740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55863048d2c0 con 0x55863052e1f0 2024-04-02T14:45:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.998+0000 7f05d3400640 1 -- 172.21.15.62:0/4108677349 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f05cc046020 con 0x7f05dc102b30 2024-04-02T14:45:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.999+0000 7f05db400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x55863052cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.999+0000 7f05db400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x55863052cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55606/0 (socket says 172.21.15.62:55606) 2024-04-02T14:45:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.999+0000 7f05db400640 1 -- 172.21.15.62:0/3333062870 learned_addr learned my addr 172.21.15.62:0/3333062870 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.999+0000 7f05daa00640 1 --2- 172.21.15.62:0/3333062870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55863052d510 0x55863052d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.999+0000 7f05db400640 1 -- 172.21.15.62:0/3333062870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558630536880 msgr2=0x558630538c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T14:45:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.999+0000 7f05db400640 1 --2- 172.21.15.62:0/3333062870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558630536880 0x558630538c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.999+0000 7f05db400640 1 -- 172.21.15.62:0/3333062870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55863052d510 msgr2=0x55863052d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.999+0000 7f05db400640 1 --2- 172.21.15.62:0/3333062870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55863052d510 0x55863052d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:17.999+0000 7f05db400640 1 -- 172.21.15.62:0/3333062870 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863026d7e0 con 0x55863052e1f0 2024-04-02T14:45:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.000+0000 7f05db400640 1 --2- 172.21.15.62:0/3333062870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x55863052cf40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009f40 tx=0x7f05d4010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dec754e2661748a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:17.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.000+0000 7f05c7e00640 1 -- 172.21.15.62:0/3333062870 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4003b70 con 0x55863052e1f0 2024-04-02T14:45:17.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.000+0000 7f05c7e00640 1 -- 172.21.15.62:0/3333062870 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f05d4003d10 con 0x55863052e1f0 2024-04-02T14:45:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.000+0000 7f05c7e00640 1 -- 172.21.15.62:0/3333062870 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4011b00 con 0x55863052e1f0 2024-04-02T14:45:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.001+0000 7f05d3400640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f05c00a3fd0 0x7f05c00a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.001+0000 7f05e4424740 1 -- 172.21.15.62:0/3333062870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 msgr2=0x55863052cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.001+0000 7f05e4424740 1 --2- 172.21.15.62:0/3333062870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x55863052cf40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009f40 tx=0x7f05d4010860 comp rx=0 tx=0).stop 2024-04-02T14:45:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.001+0000 7f05d3400640 1 -- 172.21.15.62:0/4108677349 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f05cc0f1ed0 con 0x7f05dc102b30 2024-04-02T14:45:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.001+0000 7f05e4424740 1 -- 172.21.15.62:0/3333062870 shutdown_connections 2024-04-02T14:45:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.001+0000 7f05e4424740 1 --2- 172.21.15.62:0/3333062870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558630536880 0x558630538c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.001+0000 7f05e4424740 1 --2- 172.21.15.62:0/3333062870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55863052d510 0x55863052d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.001+0000 7f05e4424740 1 --2- 172.21.15.62:0/3333062870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x55863052cf40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.001+0000 7f05e4424740 1 -- 172.21.15.62:0/3333062870 >> 172.21.15.62:0/3333062870 conn(0x5586304c1270 msgr2=0x558630527f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.001+0000 7f05daa00640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f05c00a3fd0 0x7f05c00a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:17.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.001+0000 7f05e4424740 1 -- 172.21.15.62:0/3333062870 shutdown_connections 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.002+0000 7f05e4424740 1 -- 172.21.15.62:0/3333062870 wait complete. 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.002+0000 7f05e4424740 1 -- start start 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.002+0000 7f05e4424740 1 Processor -- start 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.002+0000 7f05daa00640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f05c00a3fd0 0x7f05c00a6490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0ef5a0 tx=0x7f05cc0de040 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05e4424740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052d510 0x558630533820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05e4424740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x558630533d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05e4424740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x5586305342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05e4424740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5586305396f0 con 0x558630536880 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05e4424740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55863044ca60 con 0x55863052d510 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05e4424740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558630539870 con 0x55863052e1f0 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05e4424740 10 client.0 _ll_register_callbacks cb 0x558630452030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05db400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x5586305342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05da000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x558630533d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05db400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052d510 0x558630533820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05da000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x558630533d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55620/0 (socket says 172.21.15.62:55620) 2024-04-02T14:45:18.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.003+0000 7f05da000640 1 -- 172.21.15.62:0/1011451072 learned_addr learned my addr 172.21.15.62:0/1011451072 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.004+0000 7f05db400640 1 -- 172.21.15.62:0/1011451072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 msgr2=0x558630533d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.004+0000 7f05db400640 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x558630533d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.004+0000 7f05db400640 1 -- 172.21.15.62:0/1011451072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 msgr2=0x5586305342a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.004+0000 7f05db400640 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x5586305342a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.004+0000 7f05db400640 1 -- 172.21.15.62:0/1011451072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586302cdf70 con 0x55863052d510 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.004+0000 7f05da000640 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x558630533d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.004+0000 7f05db400640 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052d510 0x558630533820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f05d40049b0 tx=0x7f05d4011740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9505c19905f6d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.004+0000 7f05c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.004+0000 7f05c4c00640 1 -- 172.21.15.62:0/1011451072 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4024040 con 0x55863052d510 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.004+0000 7f05e4424740 1 -- 172.21.15.62:0/1011451072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05d4003d10 con 0x55863052d510 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.004+0000 7f05c4c00640 1 -- 172.21.15.62:0/1011451072 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f05d4004c40 con 0x55863052d510 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.005+0000 7f05c4c00640 1 -- 172.21.15.62:0/1011451072 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05d4004e40 con 0x55863052d510 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.005+0000 7f05e4424740 10 client.25874 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.005+0000 7f05e4424740 1 -- 172.21.15.62:0/1011451072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558630165170 con 0x55863052d510 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.005+0000 7f05c4c00640 1 -- 172.21.15.62:0/1011451072 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f05d4003d10 con 0x55863052d510 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.006+0000 7f05c4c00640 1 -- 172.21.15.62:0/1011451072 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f05d4005510 con 0x55863052d510 2024-04-02T14:45:18.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.006+0000 7f05e4424740 10 client.25874 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:18.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.006+0000 7f05e4424740 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:18.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.006+0000 7f05e4424740 20 client.25874 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:18.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.006+0000 7f05e4424740 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558630546a00 0x558630548df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.006+0000 7f05e4424740 4 client.25874 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:18.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.006+0000 7f05e4424740 1 -- 172.21.15.62:0/1011451072 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55863049e850 con 0x558630546a00 2024-04-02T14:45:18.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.006+0000 7f05da000640 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558630546a00 0x558630548df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.007+0000 7f05c4c00640 10 client.25874 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.007+0000 7f05da000640 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558630546a00 0x558630548df0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d39d874ac7ce0dd server_cookie=8f7874431c524313 in_seq=0 out_seq=0 2024-04-02T14:45:18.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05c4c00640 1 -- 172.21.15.62:0/1011451072 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55863049e850 con 0x558630546a00 2024-04-02T14:45:18.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05c4c00640 10 client.25874 handle_command_reply: tid=0 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 1 client.25874 shutdown 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 1 -- 172.21.15.62:0/1011451072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052d510 msgr2=0x558630533820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052d510 0x558630533820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f05d40049b0 tx=0x7f05d4011740 comp rx=0 tx=0).stop 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 1 -- 172.21.15.62:0/1011451072 shutdown_connections 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558630546a00 0x558630548df0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x5586305342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052e1f0 0x558630533d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05c4c00640 0 client.25874 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 1 --2- 172.21.15.62:0/1011451072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052d510 0x558630533820 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 1 -- 172.21.15.62:0/1011451072 >> 172.21.15.62:0/1011451072 conn(0x5586304c1270 msgr2=0x558630529cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 1 -- 172.21.15.62:0/1011451072 shutdown_connections 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 1 -- 172.21.15.62:0/1011451072 wait complete. 2024-04-02T14:45:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.008+0000 7f05e4424740 20 client.25874 trim_cache size 0 max 16384 2024-04-02T14:45:18.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.046+0000 7f05e4424740 1 Processor -- start 2024-04-02T14:45:18.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.046+0000 7f05e4424740 1 -- start start 2024-04-02T14:45:18.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.046+0000 7f05e4424740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 0x558630535f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.046+0000 7f05e4424740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 0x5586305425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.046+0000 7f05e4424740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x558630526d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.046+0000 7f05e4424740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1b3c60 con 0x558630536880 2024-04-02T14:45:18.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.046+0000 7f05e4424740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1b57a0 con 0x55863052e1f0 2024-04-02T14:45:18.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.046+0000 7f05e4424740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1b4330 con 0x55863052d510 2024-04-02T14:45:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.047+0000 7f05db400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 0x558630535f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.047+0000 7f05db400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 0x558630535f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55624/0 (socket says 172.21.15.62:55624) 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.047+0000 7f05db400640 1 -- 172.21.15.62:0/3408096344 learned_addr learned my addr 172.21.15.62:0/3408096344 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.047+0000 7f05db400640 1 --2- 172.21.15.62:0/3408096344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x558630526d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.047+0000 7f05da000640 1 --2- 172.21.15.62:0/3408096344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 0x5586305425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.047+0000 7f05db400640 1 -- 172.21.15.62:0/3408096344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 msgr2=0x5586305425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.047+0000 7f05db400640 1 --2- 172.21.15.62:0/3408096344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 0x5586305425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.047+0000 7f05db400640 1 -- 172.21.15.62:0/3408096344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 msgr2=0x558630526d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.047+0000 7f05db400640 1 --2- 172.21.15.62:0/3408096344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x558630526d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.047+0000 7f05db400640 1 -- 172.21.15.62:0/3408096344 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586305395d0 con 0x55863052d510 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.048+0000 7f05db400640 1 --2- 172.21.15.62:0/3408096344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 0x558630535f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f05d4006010 tx=0x7f05d4010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc0fb73fa77cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.048+0000 7f05c7400640 1 -- 172.21.15.62:0/3408096344 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4035520 con 0x55863052d510 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.048+0000 7f05c7400640 1 -- 172.21.15.62:0/3408096344 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f05d4003ca0 con 0x55863052d510 2024-04-02T14:45:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.048+0000 7f05c7400640 1 -- 172.21.15.62:0/3408096344 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d40040c0 con 0x55863052d510 2024-04-02T14:45:18.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.049+0000 7f05e4424740 1 -- 172.21.15.62:0/3408096344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 msgr2=0x558630535f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.049+0000 7f05e4424740 1 --2- 172.21.15.62:0/3408096344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 0x558630535f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f05d4006010 tx=0x7f05d4010860 comp rx=0 tx=0).stop 2024-04-02T14:45:18.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.049+0000 7f05e4424740 1 -- 172.21.15.62:0/3408096344 shutdown_connections 2024-04-02T14:45:18.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.049+0000 7f05e4424740 1 --2- 172.21.15.62:0/3408096344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x558630526d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.049+0000 7f05e4424740 1 --2- 172.21.15.62:0/3408096344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 0x5586305425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.049+0000 7f05e4424740 1 --2- 172.21.15.62:0/3408096344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 0x558630535f50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.049+0000 7f05e4424740 1 -- 172.21.15.62:0/3408096344 >> 172.21.15.62:0/3408096344 conn(0x5586304c1270 msgr2=0x558630526670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.049+0000 7f05e4424740 1 -- 172.21.15.62:0/3408096344 shutdown_connections 2024-04-02T14:45:18.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.049+0000 7f05e4424740 1 -- 172.21.15.62:0/3408096344 wait complete. 2024-04-02T14:45:18.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.050+0000 7f05e4424740 1 -- start start 2024-04-02T14:45:18.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.050+0000 7f05e4424740 1 Processor -- start 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05e4424740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 0x55863053efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05e4424740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 0x558630530c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05e4424740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x5586305311e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05e4424740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1b50d0 con 0x558630536880 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05e4424740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1b4a00 con 0x55863052e1f0 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05e4424740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1b1c40 con 0x55863052d510 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05e4424740 10 client.0 _ll_register_callbacks cb 0x7f05d4009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05db400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 0x55863053efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05da000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 0x558630530c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05db400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x5586305311e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05da000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 0x558630530c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37322/0 (socket says 172.21.15.62:37322) 2024-04-02T14:45:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.051+0000 7f05da000640 1 -- 172.21.15.62:0/1172961726 learned_addr learned my addr 172.21.15.62:0/1172961726 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.052+0000 7f05da000640 1 -- 172.21.15.62:0/1172961726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 msgr2=0x55863053efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.052+0000 7f05da000640 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 0x55863053efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.052+0000 7f05da000640 1 -- 172.21.15.62:0/1172961726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 msgr2=0x5586305311e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.052+0000 7f05da000640 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x5586305311e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.052+0000 7f05da000640 1 -- 172.21.15.62:0/1172961726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558630537410 con 0x55863052e1f0 2024-04-02T14:45:18.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.052+0000 7f05db400640 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x5586305311e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:18.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.053+0000 7f05da000640 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 0x558630530c60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x558630549480 tx=0x7f05c80138e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c1eed14b4440ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.053+0000 7f05c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:18.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.054+0000 7f05c4c00640 1 -- 172.21.15.62:0/1172961726 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c801f590 con 0x55863052e1f0 2024-04-02T14:45:18.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.054+0000 7f05c4c00640 1 -- 172.21.15.62:0/1172961726 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f05c801f730 con 0x55863052e1f0 2024-04-02T14:45:18.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.054+0000 7f05e4424740 1 -- 172.21.15.62:0/1172961726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05d4003ca0 con 0x55863052e1f0 2024-04-02T14:45:18.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.054+0000 7f05c4c00640 1 -- 172.21.15.62:0/1172961726 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05c801f930 con 0x55863052e1f0 2024-04-02T14:45:18.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.054+0000 7f05e4424740 10 client.25880 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:18.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.054+0000 7f05e4424740 1 -- 172.21.15.62:0/1172961726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586305ad0d0 con 0x55863052e1f0 2024-04-02T14:45:18.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.054+0000 7f05c4c00640 1 -- 172.21.15.62:0/1172961726 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f05d4003ca0 con 0x55863052e1f0 2024-04-02T14:45:18.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.055+0000 7f05c4c00640 1 -- 172.21.15.62:0/1172961726 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f05c80327b0 con 0x55863052e1f0 2024-04-02T14:45:18.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.055+0000 7f05e4424740 10 client.25880 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:18.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.055+0000 7f05e4424740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:18.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.055+0000 7f05e4424740 20 client.25880 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:18.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.055+0000 7f05e4424740 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5586305ad7e0 0x5586305afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.055+0000 7f05e4424740 4 client.25880 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:45:18.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.055+0000 7f05e4424740 1 -- 172.21.15.62:0/1172961726 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558630498ba0 con 0x5586305ad7e0 2024-04-02T14:45:18.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.055+0000 7f05db400640 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5586305ad7e0 0x5586305afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.056+0000 7f05db400640 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5586305ad7e0 0x5586305afc00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e379bd968d388c79 server_cookie=607a4eb279a34c4 in_seq=0 out_seq=0 2024-04-02T14:45:18.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.056+0000 7f05c4c00640 10 client.25880 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.057+0000 7f05c4c00640 1 -- 172.21.15.62:0/1172961726 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558630498ba0 con 0x5586305ad7e0 2024-04-02T14:45:18.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.057+0000 7f05c4c00640 10 client.25880 handle_command_reply: tid=0 2024-04-02T14:45:18.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.057+0000 7f05e4424740 1 client.25880 shutdown 2024-04-02T14:45:18.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.057+0000 7f05e4424740 1 -- 172.21.15.62:0/1172961726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 msgr2=0x558630530c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.057+0000 7f05e4424740 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 0x558630530c60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x558630549480 tx=0x7f05c80138e0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.057+0000 7f05e4424740 1 -- 172.21.15.62:0/1172961726 shutdown_connections 2024-04-02T14:45:18.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.057+0000 7f05e4424740 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5586305ad7e0 0x5586305afc00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.057+0000 7f05e4424740 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558630536880 0x5586305311e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.057+0000 7f05e4424740 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55863052e1f0 0x558630530c60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.057+0000 7f05e4424740 1 --2- 172.21.15.62:0/1172961726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863052d510 0x55863053efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.057+0000 7f05e4424740 1 -- 172.21.15.62:0/1172961726 >> 172.21.15.62:0/1172961726 conn(0x5586304c1270 msgr2=0x558630526f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.058+0000 7f05c4c00640 0 client.25880 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.058+0000 7f05e4424740 1 -- 172.21.15.62:0/1172961726 shutdown_connections 2024-04-02T14:45:18.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.058+0000 7f05e4424740 1 -- 172.21.15.62:0/1172961726 wait complete. 2024-04-02T14:45:18.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.058+0000 7f05e4424740 20 client.25880 trim_cache size 0 max 16384 2024-04-02T14:45:18.056 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:45:18.056 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-02T14:45:18.056 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "e01455b5-8fc0-4edb-ad56-88346f11d3a8", 2024-04-02T14:45:18.056 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-02T14:45:18.056 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:45:18.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.059+0000 7f05dbe00640 1 -- 172.21.15.62:0/4108677349 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f05c00a3fd0 msgr2=0x7f05c00a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.059+0000 7f05dbe00640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f05c00a3fd0 0x7f05c00a6490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0ef5a0 tx=0x7f05cc0de040 comp rx=0 tx=0).stop 2024-04-02T14:45:18.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.059+0000 7f05dbe00640 1 -- 172.21.15.62:0/4108677349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05dc102b30 msgr2=0x7f05dc1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.059+0000 7f05dbe00640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05dc102b30 0x7f05dc1039a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f05cc02c2e0 tx=0x7f05cc004950 comp rx=0 tx=0).stop 2024-04-02T14:45:18.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.059+0000 7f05dbe00640 1 -- 172.21.15.62:0/4108677349 shutdown_connections 2024-04-02T14:45:18.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.059+0000 7f05dbe00640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f05c00a3fd0 0x7f05c00a6490 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.059+0000 7f05dbe00640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05dc10bf50 0x7f05dc104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.059+0000 7f05dbe00640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc103450 0x7f05dc103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.059+0000 7f05dbe00640 1 --2- 172.21.15.62:0/4108677349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05dc102b30 0x7f05dc1039a0 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f05cc02c2e0 tx=0x7f05cc004950 comp rx=0 tx=0).stop 2024-04-02T14:45:18.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.059+0000 7f05dbe00640 1 -- 172.21.15.62:0/4108677349 >> 172.21.15.62:0/4108677349 conn(0x7f05dc0fc350 msgr2=0x7f05dc10e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.060+0000 7f05dbe00640 1 -- 172.21.15.62:0/4108677349 shutdown_connections 2024-04-02T14:45:18.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.060+0000 7f05dbe00640 1 -- 172.21.15.62:0/4108677349 wait complete. 2024-04-02T14:45:18.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:45:18.073 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:45:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.254+0000 7f0965e00640 1 -- 172.21.15.62:0/1703129673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0960107980 msgr2=0x7f0960107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.254+0000 7f0965e00640 1 --2- 172.21.15.62:0/1703129673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0960107980 0x7f0960107d60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0954009930 tx=0x7f095402bc30 comp rx=0 tx=0).stop 2024-04-02T14:45:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.254+0000 7f0965e00640 1 -- 172.21.15.62:0/1703129673 shutdown_connections 2024-04-02T14:45:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.254+0000 7f0965e00640 1 --2- 172.21.15.62:0/1703129673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0960102330 0x7f096010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.254+0000 7f0965e00640 1 --2- 172.21.15.62:0/1703129673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0960101970 0x7f0960101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.254+0000 7f0965e00640 1 --2- 172.21.15.62:0/1703129673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0960107980 0x7f0960107d60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.254+0000 7f0965e00640 1 -- 172.21.15.62:0/1703129673 >> 172.21.15.62:0/1703129673 conn(0x7f09600fd4a0 msgr2=0x7f09600ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.254+0000 7f0965e00640 1 -- 172.21.15.62:0/1703129673 shutdown_connections 2024-04-02T14:45:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.254+0000 7f0965e00640 1 -- 172.21.15.62:0/1703129673 wait complete. 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.255+0000 7f0965e00640 1 Processor -- start 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.255+0000 7f0965e00640 1 -- start start 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.256+0000 7f0965e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0960101970 0x7f096019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.256+0000 7f0965e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0960102330 0x7f096019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.256+0000 7f0965e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0960107980 0x7f096019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.256+0000 7f0965e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0960111e60 con 0x7f0960101970 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.256+0000 7f0965e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0960111ce0 con 0x7f0960107980 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.256+0000 7f0965e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0960111fe0 con 0x7f0960102330 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.256+0000 7f095fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0960102330 0x7f096019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.256+0000 7f0964a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0960101970 0x7f096019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.256+0000 7f095fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0960102330 0x7f096019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55670/0 (socket says 172.21.15.62:55670) 2024-04-02T14:45:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.256+0000 7f095fe00640 1 -- 172.21.15.62:0/2185814799 learned_addr learned my addr 172.21.15.62:0/2185814799 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.256+0000 7f0965400640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0960107980 0x7f096019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.257+0000 7f0965400640 1 -- 172.21.15.62:0/2185814799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0960102330 msgr2=0x7f096019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.257+0000 7f0965400640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0960102330 0x7f096019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.257+0000 7f0965400640 1 -- 172.21.15.62:0/2185814799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0960101970 msgr2=0x7f096019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.257+0000 7f0965400640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0960101970 0x7f096019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.257+0000 7f0965400640 1 -- 172.21.15.62:0/2185814799 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096019f540 con 0x7f0960107980 2024-04-02T14:45:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.257+0000 7f0964a00640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0960101970 0x7f096019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.257+0000 7f0965400640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0960107980 0x7f096019ee10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0958009fd0 tx=0x7f095800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.257+0000 7f095d600640 1 -- 172.21.15.62:0/2185814799 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095800ce20 con 0x7f0960107980 2024-04-02T14:45:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.257+0000 7f095d600640 1 -- 172.21.15.62:0/2185814799 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0958004b20 con 0x7f0960107980 2024-04-02T14:45:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.257+0000 7f095d600640 1 -- 172.21.15.62:0/2185814799 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958010790 con 0x7f0960107980 2024-04-02T14:45:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.258+0000 7f0965e00640 1 -- 172.21.15.62:0/2185814799 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096019f830 con 0x7f0960107980 2024-04-02T14:45:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.258+0000 7f0965e00640 1 -- 172.21.15.62:0/2185814799 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09601a70c0 con 0x7f0960107980 2024-04-02T14:45:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.262+0000 7f095d600640 1 -- 172.21.15.62:0/2185814799 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f0958005020 con 0x7f0960107980 2024-04-02T14:45:18.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.263+0000 7f0969feb740 1 Processor -- start 2024-04-02T14:45:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.263+0000 7f0969feb740 1 -- start start 2024-04-02T14:45:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.263+0000 7f095d600640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09480a3ed0 0x7f09480a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.263+0000 7f0969feb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e74960 0x5570d4e74d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.263+0000 7f0969feb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e7d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.263+0000 7f095d600640 1 -- 172.21.15.62:0/2185814799 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f09580d1120 con 0x7f0960107980 2024-04-02T14:45:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.263+0000 7f0969feb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e7de60 0x5570d4e80250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.263+0000 7f0969feb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5570d4dd8770 con 0x5570d4e74960 2024-04-02T14:45:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.263+0000 7f0969feb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5570d4d80700 con 0x5570d4e75280 2024-04-02T14:45:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.263+0000 7f0969feb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5570d4dd4720 con 0x5570d4e7de60 2024-04-02T14:45:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f095fe00640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09480a3ed0 0x7f09480a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f0964a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e74960 0x5570d4e74d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f095fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e7d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f0965400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e7de60 0x5570d4e80250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f095fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e7d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37344/0 (socket says 172.21.15.62:37344) 2024-04-02T14:45:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f0964a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e74960 0x5570d4e74d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60338/0 (socket says 172.21.15.62:60338) 2024-04-02T14:45:18.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f095fe00640 1 -- 172.21.15.62:0/1412272784 learned_addr learned my addr 172.21.15.62:0/1412272784 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f0965400640 1 -- 172.21.15.62:0/1412272784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 msgr2=0x5570d4e7d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f0965400640 1 --2- 172.21.15.62:0/1412272784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e7d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f0965400640 1 -- 172.21.15.62:0/1412272784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e74960 msgr2=0x5570d4e74d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f0965400640 1 --2- 172.21.15.62:0/1412272784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e74960 0x5570d4e74d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.264+0000 7f0965400640 1 -- 172.21.15.62:0/1412272784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d4bb67d0 con 0x5570d4e7de60 2024-04-02T14:45:18.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.265+0000 7f095fe00640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09480a3ed0 0x7f09480a6390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f094c011a70 tx=0x7f094c009ca0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.265+0000 7f0965400640 1 --2- 172.21.15.62:0/1412272784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e7de60 0x5570d4e80250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f09580ce680 tx=0x7f095809ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff2bb5a320d5056 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.265+0000 7f0951c00640 1 -- 172.21.15.62:0/1412272784 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09580b5020 con 0x5570d4e7de60 2024-04-02T14:45:18.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.265+0000 7f0951c00640 1 -- 172.21.15.62:0/1412272784 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0958005310 con 0x5570d4e7de60 2024-04-02T14:45:18.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.265+0000 7f0951c00640 1 -- 172.21.15.62:0/1412272784 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09580b6220 con 0x5570d4e7de60 2024-04-02T14:45:18.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.266+0000 7f0969feb740 1 -- 172.21.15.62:0/1412272784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e7de60 msgr2=0x5570d4e80250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.266+0000 7f0969feb740 1 --2- 172.21.15.62:0/1412272784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e7de60 0x5570d4e80250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f09580ce680 tx=0x7f095809ced0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.266+0000 7f0969feb740 1 -- 172.21.15.62:0/1412272784 shutdown_connections 2024-04-02T14:45:18.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.266+0000 7f0969feb740 1 --2- 172.21.15.62:0/1412272784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e7de60 0x5570d4e80250 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.266+0000 7f0969feb740 1 --2- 172.21.15.62:0/1412272784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e7d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.266+0000 7f0969feb740 1 --2- 172.21.15.62:0/1412272784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e74960 0x5570d4e74d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.266+0000 7f0969feb740 1 -- 172.21.15.62:0/1412272784 >> 172.21.15.62:0/1412272784 conn(0x5570d4e6f090 msgr2=0x5570d4e6f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.266+0000 7f0969feb740 1 -- 172.21.15.62:0/1412272784 shutdown_connections 2024-04-02T14:45:18.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.267+0000 7f0969feb740 1 -- 172.21.15.62:0/1412272784 wait complete. 2024-04-02T14:45:18.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.267+0000 7f0969feb740 1 -- start start 2024-04-02T14:45:18.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.267+0000 7f0969feb740 1 Processor -- start 2024-04-02T14:45:18.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.268+0000 7f0969feb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e74960 0x5570d4e7af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.268+0000 7f0969feb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e7b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.268+0000 7f0969feb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e7de60 0x5570d4e7b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.268+0000 7f0969feb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5570d4e80c50 con 0x5570d4e7de60 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.268+0000 7f0969feb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5570d4d93ac0 con 0x5570d4e75280 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.268+0000 7f0969feb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5570d4e80dd0 con 0x5570d4e74960 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.268+0000 7f0969feb740 10 client.0 _ll_register_callbacks cb 0x5570d4d96180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.268+0000 7f095fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e7b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.268+0000 7f095fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e7b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37358/0 (socket says 172.21.15.62:37358) 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.268+0000 7f095fe00640 1 -- 172.21.15.62:0/2488348106 learned_addr learned my addr 172.21.15.62:0/2488348106 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.269+0000 7f0964a00640 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e74960 0x5570d4e7af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.269+0000 7f0965400640 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e7de60 0x5570d4e7b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.269+0000 7f095fe00640 1 -- 172.21.15.62:0/2488348106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e74960 msgr2=0x5570d4e7af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.269+0000 7f095fe00640 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e74960 0x5570d4e7af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.269+0000 7f095fe00640 1 -- 172.21.15.62:0/2488348106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e7de60 msgr2=0x5570d4e7b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.269+0000 7f095fe00640 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e7de60 0x5570d4e7b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.269+0000 7f095fe00640 1 -- 172.21.15.62:0/2488348106 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d4aec3f0 con 0x5570d4e75280 2024-04-02T14:45:18.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.269+0000 7f0965400640 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e7de60 0x5570d4e7b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:18.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.269+0000 7f095fe00640 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e7b470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f094c0097f0 tx=0x7f094c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14db5e52c8ae140a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.269+0000 7f0946000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.270+0000 7f0946000640 1 -- 172.21.15.62:0/2488348106 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0044b0 con 0x5570d4e75280 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.270+0000 7f0969feb740 1 -- 172.21.15.62:0/2488348106 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0958005310 con 0x5570d4e75280 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.270+0000 7f0946000640 1 -- 172.21.15.62:0/2488348106 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f094c005020 con 0x5570d4e75280 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.270+0000 7f0946000640 1 -- 172.21.15.62:0/2488348106 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f094c005220 con 0x5570d4e75280 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.270+0000 7f0969feb740 10 client.25892 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.270+0000 7f0969feb740 1 -- 172.21.15.62:0/2488348106 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570d4aac0f0 con 0x5570d4e75280 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.270+0000 7f0946000640 1 -- 172.21.15.62:0/2488348106 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0958005310 con 0x5570d4e75280 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.271+0000 7f0946000640 1 -- 172.21.15.62:0/2488348106 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f094c0048e0 con 0x5570d4e75280 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.271+0000 7f0969feb740 10 client.25892 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.271+0000 7f0969feb740 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.271+0000 7f0969feb740 20 client.25892 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:18.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.271+0000 7f0969feb740 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5570d4e8ded0 0x5570d4e90330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.271+0000 7f0969feb740 4 client.25892 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:18.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.271+0000 7f0969feb740 1 -- 172.21.15.62:0/2488348106 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570d4d79630 con 0x5570d4e8ded0 2024-04-02T14:45:18.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.271+0000 7f0964a00640 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5570d4e8ded0 0x5570d4e90330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.272+0000 7f0964a00640 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5570d4e8ded0 0x5570d4e90330 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4caade641932ca25 server_cookie=690fff430e40c661 in_seq=0 out_seq=0 2024-04-02T14:45:18.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.272+0000 7f0946000640 10 client.25892 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0946000640 1 -- 172.21.15.62:0/2488348106 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570d4d79630 con 0x5570d4e8ded0 2024-04-02T14:45:18.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0946000640 10 client.25892 handle_command_reply: tid=0 2024-04-02T14:45:18.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0969feb740 1 client.25892 shutdown 2024-04-02T14:45:18.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0969feb740 1 -- 172.21.15.62:0/2488348106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 msgr2=0x5570d4e7b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0969feb740 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e7b470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f094c0097f0 tx=0x7f094c005f90 comp rx=0 tx=0).stop 2024-04-02T14:45:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0969feb740 1 -- 172.21.15.62:0/2488348106 shutdown_connections 2024-04-02T14:45:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0969feb740 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5570d4e8ded0 0x5570d4e90330 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0946000640 0 client.25892 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0969feb740 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e7de60 0x5570d4e7b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0969feb740 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e7b470 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0969feb740 1 --2- 172.21.15.62:0/2488348106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e74960 0x5570d4e7af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0969feb740 1 -- 172.21.15.62:0/2488348106 >> 172.21.15.62:0/2488348106 conn(0x5570d4e6f090 msgr2=0x5570d4e6e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0969feb740 1 -- 172.21.15.62:0/2488348106 shutdown_connections 2024-04-02T14:45:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.274+0000 7f0969feb740 1 -- 172.21.15.62:0/2488348106 wait complete. 2024-04-02T14:45:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.275+0000 7f0969feb740 20 client.25892 trim_cache size 0 max 16384 2024-04-02T14:45:18.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.312+0000 7f0969feb740 1 Processor -- start 2024-04-02T14:45:18.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.312+0000 7f0969feb740 1 -- start start 2024-04-02T14:45:18.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.312+0000 7f0969feb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e6f090 0x5570d4e7d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.312+0000 7f0969feb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e74960 0x5570d4e89b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.312+0000 7f0969feb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e95c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.312+0000 7f0969feb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09601af7d0 con 0x5570d4e6f090 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.312+0000 7f0969feb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09601b1310 con 0x5570d4e75280 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.312+0000 7f0969feb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09601afea0 con 0x5570d4e74960 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.313+0000 7f0964a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e6f090 0x5570d4e7d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.313+0000 7f0964a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e6f090 0x5570d4e7d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60360/0 (socket says 172.21.15.62:60360) 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.313+0000 7f0964a00640 1 -- 172.21.15.62:0/1742634611 learned_addr learned my addr 172.21.15.62:0/1742634611 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.313+0000 7f095fe00640 1 --2- 172.21.15.62:0/1742634611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e74960 0x5570d4e89b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.313+0000 7f0965400640 1 --2- 172.21.15.62:0/1742634611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e95c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.313+0000 7f0964a00640 1 -- 172.21.15.62:0/1742634611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e74960 msgr2=0x5570d4e89b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.313+0000 7f0964a00640 1 --2- 172.21.15.62:0/1742634611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e74960 0x5570d4e89b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.313+0000 7f0964a00640 1 -- 172.21.15.62:0/1742634611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 msgr2=0x5570d4e95c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.313+0000 7f0964a00640 1 --2- 172.21.15.62:0/1742634611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e95c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.313+0000 7f0964a00640 1 -- 172.21.15.62:0/1742634611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d4e74330 con 0x5570d4e6f090 2024-04-02T14:45:18.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.313+0000 7f0965400640 1 --2- 172.21.15.62:0/1742634611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e95c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.314+0000 7f0964a00640 1 --2- 172.21.15.62:0/1742634611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e6f090 0x5570d4e7d660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f095402c970 tx=0x7f095402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25df94f0dbd8c92a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.314+0000 7f0951200640 1 -- 172.21.15.62:0/1742634611 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954048c10 con 0x5570d4e6f090 2024-04-02T14:45:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.314+0000 7f0951200640 1 -- 172.21.15.62:0/1742634611 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0954048db0 con 0x5570d4e6f090 2024-04-02T14:45:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.314+0000 7f0951200640 1 -- 172.21.15.62:0/1742634611 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954048450 con 0x5570d4e6f090 2024-04-02T14:45:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.315+0000 7f0969feb740 1 -- 172.21.15.62:0/1742634611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e6f090 msgr2=0x5570d4e7d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.315+0000 7f0969feb740 1 --2- 172.21.15.62:0/1742634611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e6f090 0x5570d4e7d660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f095402c970 tx=0x7f095402e040 comp rx=0 tx=0).stop 2024-04-02T14:45:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.315+0000 7f0969feb740 1 -- 172.21.15.62:0/1742634611 shutdown_connections 2024-04-02T14:45:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.316+0000 7f0969feb740 1 --2- 172.21.15.62:0/1742634611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e95c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.316+0000 7f0969feb740 1 --2- 172.21.15.62:0/1742634611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e74960 0x5570d4e89b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.316+0000 7f0969feb740 1 --2- 172.21.15.62:0/1742634611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e6f090 0x5570d4e7d660 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.316+0000 7f0969feb740 1 -- 172.21.15.62:0/1742634611 >> 172.21.15.62:0/1742634611 conn(0x5570d4e79b90 msgr2=0x5570d4e6e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.316+0000 7f0969feb740 1 -- 172.21.15.62:0/1742634611 shutdown_connections 2024-04-02T14:45:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.316+0000 7f0969feb740 1 -- 172.21.15.62:0/1742634611 wait complete. 2024-04-02T14:45:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.317+0000 7f0969feb740 1 -- start start 2024-04-02T14:45:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.317+0000 7f0969feb740 1 Processor -- start 2024-04-02T14:45:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.318+0000 7f0969feb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e6f090 0x5570d4e86210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.318+0000 7f0969feb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e74960 0x5570d4e82ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.318+0000 7f0969feb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e83530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.318+0000 7f0969feb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09601b0c40 con 0x5570d4e74960 2024-04-02T14:45:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.318+0000 7f0969feb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09601b0570 con 0x5570d4e75280 2024-04-02T14:45:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.318+0000 7f0969feb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09601ad7b0 con 0x5570d4e6f090 2024-04-02T14:45:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.318+0000 7f0969feb740 10 client.0 _ll_register_callbacks cb 0x7f094c011430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.318+0000 7f0964a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e6f090 0x5570d4e86210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.318+0000 7f0964a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e6f090 0x5570d4e86210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55708/0 (socket says 172.21.15.62:55708) 2024-04-02T14:45:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.318+0000 7f0964a00640 1 -- 172.21.15.62:0/3566670870 learned_addr learned my addr 172.21.15.62:0/3566670870 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.318+0000 7f095fe00640 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e74960 0x5570d4e82ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.319+0000 7f0965400640 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e83530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.319+0000 7f0964a00640 1 -- 172.21.15.62:0/3566670870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 msgr2=0x5570d4e83530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.319+0000 7f0964a00640 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e83530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.319+0000 7f0964a00640 1 -- 172.21.15.62:0/3566670870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e74960 msgr2=0x5570d4e82ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.319+0000 7f0964a00640 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e74960 0x5570d4e82ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.319+0000 7f0964a00640 1 -- 172.21.15.62:0/3566670870 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d4e83af0 con 0x5570d4e6f090 2024-04-02T14:45:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.319+0000 7f0964a00640 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e6f090 0x5570d4e86210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f095402c170 tx=0x7f0954005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b75f3766baeb3a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.319+0000 7f0946000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:45:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.320+0000 7f0946000640 1 -- 172.21.15.62:0/3566670870 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954004280 con 0x5570d4e6f090 2024-04-02T14:45:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.320+0000 7f0946000640 1 -- 172.21.15.62:0/3566670870 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f095402c9d0 con 0x5570d4e6f090 2024-04-02T14:45:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.320+0000 7f0969feb740 1 -- 172.21.15.62:0/3566670870 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0954048db0 con 0x5570d4e6f090 2024-04-02T14:45:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.320+0000 7f0946000640 1 -- 172.21.15.62:0/3566670870 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f095405f050 con 0x5570d4e6f090 2024-04-02T14:45:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.320+0000 7f0969feb740 10 client.25876 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.320+0000 7f0969feb740 1 -- 172.21.15.62:0/3566670870 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570d4ef5240 con 0x5570d4e6f090 2024-04-02T14:45:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.321+0000 7f0946000640 1 -- 172.21.15.62:0/3566670870 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0954048db0 con 0x5570d4e6f090 2024-04-02T14:45:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.321+0000 7f0946000640 1 -- 172.21.15.62:0/3566670870 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f09540037b0 con 0x5570d4e6f090 2024-04-02T14:45:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.321+0000 7f0969feb740 10 client.25876 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.321+0000 7f0969feb740 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.321+0000 7f0969feb740 20 client.25876 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.321+0000 7f0969feb740 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5570d4e7de60 0x5570d4ef7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.321+0000 7f0969feb740 4 client.25876 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:45:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.321+0000 7f0969feb740 1 -- 172.21.15.62:0/3566670870 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5570d4de5cb0 con 0x5570d4e7de60 2024-04-02T14:45:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.322+0000 7f095fe00640 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5570d4e7de60 0x5570d4ef7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.322+0000 7f095fe00640 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5570d4e7de60 0x5570d4ef7970 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a02887de094091a5 server_cookie=68235619af47a2ef in_seq=0 out_seq=0 2024-04-02T14:45:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.322+0000 7f0946000640 10 client.25876 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0946000640 1 -- 172.21.15.62:0/3566670870 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5570d4de5cb0 con 0x5570d4e7de60 2024-04-02T14:45:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0946000640 10 client.25876 handle_command_reply: tid=0 2024-04-02T14:45:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 1 client.25876 shutdown 2024-04-02T14:45:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 1 -- 172.21.15.62:0/3566670870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e6f090 msgr2=0x5570d4e86210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e6f090 0x5570d4e86210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f095402c170 tx=0x7f0954005be0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 1 -- 172.21.15.62:0/3566670870 shutdown_connections 2024-04-02T14:45:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5570d4e7de60 0x5570d4ef7970 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570d4e75280 0x5570d4e83530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570d4e74960 0x5570d4e82ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 1 --2- 172.21.15.62:0/3566670870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570d4e6f090 0x5570d4e86210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0946000640 0 client.25876 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 1 -- 172.21.15.62:0/3566670870 >> 172.21.15.62:0/3566670870 conn(0x5570d4e79b90 msgr2=0x5570d4e704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 1 -- 172.21.15.62:0/3566670870 shutdown_connections 2024-04-02T14:45:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 1 -- 172.21.15.62:0/3566670870 wait complete. 2024-04-02T14:45:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.323+0000 7f0969feb740 20 client.25876 trim_cache size 0 max 16384 2024-04-02T14:45:18.321 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:45:18.322 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T14:45:18.322 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-02T14:45:18.322 INFO:teuthology.orchestra.run.smithi062.stdout: "e01455b5-8fc0-4edb-ad56-88346f11d3a8": { 2024-04-02T14:45:18.322 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-04-02T14:45:18.322 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "e01455b5-8fc0-4edb-ad56-88346f11d3a8", 2024-04-02T14:45:18.322 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-04-02T14:45:18.322 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:45:18.322 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:45:18.322 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.324+0000 7f0965e00640 1 -- 172.21.15.62:0/2185814799 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09480a3ed0 msgr2=0x7f09480a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.324+0000 7f0965e00640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09480a3ed0 0x7f09480a6390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f094c011a70 tx=0x7f094c009ca0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.324+0000 7f0965e00640 1 -- 172.21.15.62:0/2185814799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0960107980 msgr2=0x7f096019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.324+0000 7f0965e00640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0960107980 0x7f096019ee10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0958009fd0 tx=0x7f095800ef10 comp rx=0 tx=0).stop 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.325+0000 7f0965e00640 1 -- 172.21.15.62:0/2185814799 shutdown_connections 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.325+0000 7f0965e00640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09480a3ed0 0x7f09480a6390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.325+0000 7f0965e00640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0960107980 0x7f096019ee10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.325+0000 7f0965e00640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0960102330 0x7f096019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.325+0000 7f0965e00640 1 --2- 172.21.15.62:0/2185814799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0960101970 0x7f096019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.325+0000 7f0965e00640 1 -- 172.21.15.62:0/2185814799 >> 172.21.15.62:0/2185814799 conn(0x7f09600fd4a0 msgr2=0x7f096010cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.325+0000 7f0965e00640 1 -- 172.21.15.62:0/2185814799 shutdown_connections 2024-04-02T14:45:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:18.325+0000 7f0965e00640 1 -- 172.21.15.62:0/2185814799 wait complete. 2024-04-02T14:45:18.346 INFO:tasks.cephfs.filesystem:scrub status for tag:e01455b5-8fc0-4edb-ad56-88346f11d3a8 - {'path': '/', 'tag': 'e01455b5-8fc0-4edb-ad56-88346f11d3a8', 'options': 'recursive,force'} 2024-04-02T14:45:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.651+0000 7f2734a00640 1 -- 172.21.15.62:0/1170770812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27301022c0 msgr2=0x7f273010e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.651+0000 7f2734a00640 1 --2- 172.21.15.62:0/1170770812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27301022c0 0x7f273010e7d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2728009ca0 tx=0x7f272802c100 comp rx=0 tx=0).stop 2024-04-02T14:45:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.651+0000 7f2734a00640 1 -- 172.21.15.62:0/1170770812 shutdown_connections 2024-04-02T14:45:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.651+0000 7f2734a00640 1 --2- 172.21.15.62:0/1170770812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27301022c0 0x7f273010e7d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.651+0000 7f2734a00640 1 --2- 172.21.15.62:0/1170770812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2730101900 0x7f2730101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.651+0000 7f2734a00640 1 --2- 172.21.15.62:0/1170770812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2730107910 0x7f2730107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.651+0000 7f2734a00640 1 -- 172.21.15.62:0/1170770812 >> 172.21.15.62:0/1170770812 conn(0x7f27300fd410 msgr2=0x7f27300ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.651+0000 7f2734a00640 1 -- 172.21.15.62:0/1170770812 shutdown_connections 2024-04-02T14:45:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.652+0000 7f2734a00640 1 -- 172.21.15.62:0/1170770812 wait complete. 2024-04-02T14:45:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.652+0000 7f2734a00640 1 Processor -- start 2024-04-02T14:45:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.653+0000 7f2734a00640 1 -- start start 2024-04-02T14:45:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.653+0000 7f2734a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2730101900 0x7f2730073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.653+0000 7f2734a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27301022c0 0x7f2730074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.653+0000 7f2734a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2730107910 0x7f2730074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.653+0000 7f2734a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2730111e10 con 0x7f27301022c0 2024-04-02T14:45:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.653+0000 7f2734a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2730111c90 con 0x7f2730107910 2024-04-02T14:45:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.653+0000 7f2734a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2730111f90 con 0x7f2730101900 2024-04-02T14:45:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.653+0000 7f272f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2730101900 0x7f2730073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.653+0000 7f272f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2730101900 0x7f2730073e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55726/0 (socket says 172.21.15.62:55726) 2024-04-02T14:45:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.654+0000 7f272f400640 1 -- 172.21.15.62:0/1707102370 learned_addr learned my addr 172.21.15.62:0/1707102370 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.654+0000 7f272fe00640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2730107910 0x7f2730074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.654+0000 7f272ea00640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27301022c0 0x7f2730074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.654+0000 7f272f400640 1 -- 172.21.15.62:0/1707102370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2730107910 msgr2=0x7f2730074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.654+0000 7f272f400640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2730107910 0x7f2730074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.654+0000 7f272f400640 1 -- 172.21.15.62:0/1707102370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27301022c0 msgr2=0x7f2730074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.654+0000 7f272f400640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27301022c0 0x7f2730074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.654+0000 7f272f400640 1 -- 172.21.15.62:0/1707102370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27301a6770 con 0x7f2730101900 2024-04-02T14:45:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.654+0000 7f272f400640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2730101900 0x7f2730073e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f272000e9f0 tx=0x7f272000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.655+0000 7f2727e00640 1 -- 172.21.15.62:0/1707102370 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272000cd80 con 0x7f2730101900 2024-04-02T14:45:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.655+0000 7f2734a00640 1 -- 172.21.15.62:0/1707102370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27301a6a60 con 0x7f2730101900 2024-04-02T14:45:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.655+0000 7f2727e00640 1 -- 172.21.15.62:0/1707102370 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2720010930 con 0x7f2730101900 2024-04-02T14:45:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.655+0000 7f2727e00640 1 -- 172.21.15.62:0/1707102370 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720018b00 con 0x7f2730101900 2024-04-02T14:45:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.655+0000 7f2734a00640 1 -- 172.21.15.62:0/1707102370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27301a7000 con 0x7f2730101900 2024-04-02T14:45:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.658+0000 7f2727e00640 1 -- 172.21.15.62:0/1707102370 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f2720018ca0 con 0x7f2730101900 2024-04-02T14:45:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.659+0000 7f2738db2740 1 Processor -- start 2024-04-02T14:45:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.659+0000 7f2727e00640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f27140a3ec0 0x7f27140a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.660+0000 7f2738db2740 1 -- start start 2024-04-02T14:45:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.660+0000 7f2727e00640 1 -- 172.21.15.62:0/1707102370 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f27200d0da0 con 0x7f2730101900 2024-04-02T14:45:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.660+0000 7f272fe00640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f27140a3ec0 0x7f27140a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.660+0000 7f2738db2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17db0e0 0x5654f17db4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.660+0000 7f2738db2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 0x5654f17e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.660+0000 7f2738db2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17e45e0 0x5654f17e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.660+0000 7f2738db2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5654f173fd70 con 0x5654f17db0e0 2024-04-02T14:45:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.660+0000 7f2738db2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5654f16e82a0 con 0x5654f17dba00 2024-04-02T14:45:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.660+0000 7f2738db2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5654f173bd20 con 0x5654f17e45e0 2024-04-02T14:45:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.660+0000 7f272f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17db0e0 0x5654f17db4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.660+0000 7f272ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 0x5654f17e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.661+0000 7f272f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17db0e0 0x5654f17db4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60396/0 (socket says 172.21.15.62:60396) 2024-04-02T14:45:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.661+0000 7f272f400640 1 -- 172.21.15.62:0/1089797782 learned_addr learned my addr 172.21.15.62:0/1089797782 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.661+0000 7f272ea00640 1 -- 172.21.15.62:0/1089797782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17e45e0 msgr2=0x5654f17e69d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T14:45:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.661+0000 7f272ea00640 1 --2- 172.21.15.62:0/1089797782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17e45e0 0x5654f17e69d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.661+0000 7f272ea00640 1 -- 172.21.15.62:0/1089797782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17db0e0 msgr2=0x5654f17db4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.661+0000 7f272ea00640 1 --2- 172.21.15.62:0/1089797782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17db0e0 0x5654f17db4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.661+0000 7f272ea00640 1 -- 172.21.15.62:0/1089797782 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f16fc840 con 0x5654f17dba00 2024-04-02T14:45:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.661+0000 7f272fe00640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f27140a3ec0 0x7f27140a6380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2728005e30 tx=0x7f2728005d80 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.661+0000 7f272ea00640 1 --2- 172.21.15.62:0/1089797782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 0x5654f17e40a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f271c009bf0 tx=0x7f271c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77d4b1367a61baa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.662+0000 7f271be00640 1 -- 172.21.15.62:0/1089797782 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c009da0 con 0x5654f17dba00 2024-04-02T14:45:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.662+0000 7f271be00640 1 -- 172.21.15.62:0/1089797782 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f271c005020 con 0x5654f17dba00 2024-04-02T14:45:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.662+0000 7f271be00640 1 -- 172.21.15.62:0/1089797782 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c004c60 con 0x5654f17dba00 2024-04-02T14:45:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.663+0000 7f2738db2740 1 -- 172.21.15.62:0/1089797782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 msgr2=0x5654f17e40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.663+0000 7f2738db2740 1 --2- 172.21.15.62:0/1089797782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 0x5654f17e40a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f271c009bf0 tx=0x7f271c011810 comp rx=0 tx=0).stop 2024-04-02T14:45:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.663+0000 7f2738db2740 1 -- 172.21.15.62:0/1089797782 shutdown_connections 2024-04-02T14:45:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.663+0000 7f2738db2740 1 --2- 172.21.15.62:0/1089797782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17e45e0 0x5654f17e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.663+0000 7f2738db2740 1 --2- 172.21.15.62:0/1089797782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 0x5654f17e40a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.663+0000 7f2738db2740 1 --2- 172.21.15.62:0/1089797782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17db0e0 0x5654f17db4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.663+0000 7f2738db2740 1 -- 172.21.15.62:0/1089797782 >> 172.21.15.62:0/1089797782 conn(0x5654f17d5810 msgr2=0x5654f17d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.663+0000 7f2738db2740 1 -- 172.21.15.62:0/1089797782 shutdown_connections 2024-04-02T14:45:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.663+0000 7f2738db2740 1 -- 172.21.15.62:0/1089797782 wait complete. 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.663+0000 7f2738db2740 1 -- start start 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.664+0000 7f2738db2740 1 Processor -- start 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.664+0000 7f2738db2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 0x5654f17e1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.664+0000 7f2738db2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17dba00 0x5654f17e1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.664+0000 7f2738db2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17e45e0 0x5654f17e20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.664+0000 7f2738db2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5654f17e73d0 con 0x5654f17dba00 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.664+0000 7f2738db2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5654f16fb660 con 0x5654f17e45e0 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.664+0000 7f2738db2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5654f17e7550 con 0x5654f17db0e0 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.664+0000 7f2738db2740 10 client.0 _ll_register_callbacks cb 0x5654f16fb050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.664+0000 7f272f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 0x5654f17e1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.664+0000 7f272f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 0x5654f17e1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55738/0 (socket says 172.21.15.62:55738) 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.665+0000 7f272f400640 1 -- 172.21.15.62:0/1469591656 learned_addr learned my addr 172.21.15.62:0/1469591656 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.665+0000 7f272fe00640 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17e45e0 0x5654f17e20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.665+0000 7f272ea00640 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17dba00 0x5654f17e1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.665+0000 7f272f400640 1 -- 172.21.15.62:0/1469591656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17e45e0 msgr2=0x5654f17e20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.665+0000 7f272f400640 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17e45e0 0x5654f17e20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.665+0000 7f272f400640 1 -- 172.21.15.62:0/1469591656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17dba00 msgr2=0x5654f17e1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.665+0000 7f272f400640 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17dba00 0x5654f17e1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.665+0000 7f272f400640 1 -- 172.21.15.62:0/1469591656 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f170ae80 con 0x5654f17db0e0 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.665+0000 7f272ea00640 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17dba00 0x5654f17e1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.665+0000 7f272f400640 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 0x5654f17e1650 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f272000c6a0 tx=0x7f272009c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c11631362bbf95b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.665+0000 7f2718c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.666+0000 7f2718c00640 1 -- 172.21.15.62:0/1469591656 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27200b6060 con 0x5654f17db0e0 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.666+0000 7f2738db2740 1 -- 172.21.15.62:0/1469591656 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f271c005020 con 0x5654f17db0e0 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.666+0000 7f2718c00640 1 -- 172.21.15.62:0/1469591656 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f272009c050 con 0x5654f17db0e0 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.666+0000 7f2718c00640 1 -- 172.21.15.62:0/1469591656 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2720010ce0 con 0x5654f17db0e0 2024-04-02T14:45:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.666+0000 7f2738db2740 10 client.25885 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.666+0000 7f2738db2740 1 -- 172.21.15.62:0/1469591656 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654f1454210 con 0x5654f17db0e0 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.666+0000 7f2718c00640 1 -- 172.21.15.62:0/1469591656 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f271c005020 con 0x5654f17db0e0 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.667+0000 7f2718c00640 1 -- 172.21.15.62:0/1469591656 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2720010430 con 0x5654f17db0e0 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.668+0000 7f2738db2740 10 client.25885 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.668+0000 7f2738db2740 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.668+0000 7f2738db2740 20 client.25885 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.668+0000 7f2738db2740 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5654f17f4a40 0x5654f17f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.668+0000 7f2738db2740 4 client.25885 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.668+0000 7f2738db2740 1 -- 172.21.15.62:0/1469591656 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654f16efcd0 con 0x5654f17f4a40 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.668+0000 7f272ea00640 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5654f17f4a40 0x5654f17f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.669+0000 7f272ea00640 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5654f17f4a40 0x5654f17f6e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8ffce302077da99 server_cookie=6082bd73ccbe8920 in_seq=0 out_seq=0 2024-04-02T14:45:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.669+0000 7f2718c00640 10 client.25885 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.669+0000 7f2718c00640 1 -- 172.21.15.62:0/1469591656 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654f16efcd0 con 0x5654f17f4a40 2024-04-02T14:45:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.669+0000 7f2718c00640 10 client.25885 handle_command_reply: tid=0 2024-04-02T14:45:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.669+0000 7f2738db2740 1 client.25885 shutdown 2024-04-02T14:45:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2738db2740 1 -- 172.21.15.62:0/1469591656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 msgr2=0x5654f17e1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2738db2740 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 0x5654f17e1650 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f272000c6a0 tx=0x7f272009c620 comp rx=0 tx=0).stop 2024-04-02T14:45:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2738db2740 1 -- 172.21.15.62:0/1469591656 shutdown_connections 2024-04-02T14:45:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2738db2740 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5654f17f4a40 0x5654f17f6e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2738db2740 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17e45e0 0x5654f17e20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2738db2740 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17dba00 0x5654f17e1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2718c00640 0 client.25885 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2738db2740 1 --2- 172.21.15.62:0/1469591656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 0x5654f17e1650 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2738db2740 1 -- 172.21.15.62:0/1469591656 >> 172.21.15.62:0/1469591656 conn(0x5654f17d5810 msgr2=0x5654f17d7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2738db2740 1 -- 172.21.15.62:0/1469591656 shutdown_connections 2024-04-02T14:45:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2738db2740 1 -- 172.21.15.62:0/1469591656 wait complete. 2024-04-02T14:45:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.670+0000 7f2738db2740 20 client.25885 trim_cache size 0 max 16384 2024-04-02T14:45:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.686+0000 7f2738db2740 1 Processor -- start 2024-04-02T14:45:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.686+0000 7f2738db2740 1 -- start start 2024-04-02T14:45:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.686+0000 7f2738db2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17db0e0 0x5654f17e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.686+0000 7f2738db2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17dba00 0x5654f17f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.687+0000 7f2738db2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17e45e0 0x5654f17d7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.687+0000 7f2738db2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27301af850 con 0x5654f17e45e0 2024-04-02T14:45:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.687+0000 7f2738db2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f27301b1390 con 0x5654f17db0e0 2024-04-02T14:45:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.687+0000 7f2738db2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27301aff20 con 0x5654f17dba00 2024-04-02T14:45:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.687+0000 7f272f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17db0e0 0x5654f17e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.687+0000 7f272fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17e45e0 0x5654f17d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.687+0000 7f272f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17db0e0 0x5654f17e3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37404/0 (socket says 172.21.15.62:37404) 2024-04-02T14:45:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.687+0000 7f272f400640 1 -- 172.21.15.62:0/367132120 learned_addr learned my addr 172.21.15.62:0/367132120 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.687+0000 7f272ea00640 1 --2- 172.21.15.62:0/367132120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17dba00 0x5654f17f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.687+0000 7f272f400640 1 -- 172.21.15.62:0/367132120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17dba00 msgr2=0x5654f17f02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.688+0000 7f272f400640 1 --2- 172.21.15.62:0/367132120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17dba00 0x5654f17f02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.688+0000 7f272f400640 1 -- 172.21.15.62:0/367132120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17e45e0 msgr2=0x5654f17d7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.688+0000 7f272f400640 1 --2- 172.21.15.62:0/367132120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17e45e0 0x5654f17d7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.688+0000 7f272f400640 1 -- 172.21.15.62:0/367132120 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f17f71a0 con 0x5654f17db0e0 2024-04-02T14:45:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.688+0000 7f272ea00640 1 --2- 172.21.15.62:0/367132120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17dba00 0x5654f17f02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.688+0000 7f272f400640 1 --2- 172.21.15.62:0/367132120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17db0e0 0x5654f17e3d80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f272009cb90 tx=0x7f2720018520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f12c4dd6e2a74eab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.688+0000 7f271b400640 1 -- 172.21.15.62:0/367132120 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720010430 con 0x5654f17db0e0 2024-04-02T14:45:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.688+0000 7f271b400640 1 -- 172.21.15.62:0/367132120 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f272009c050 con 0x5654f17db0e0 2024-04-02T14:45:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.688+0000 7f271b400640 1 -- 172.21.15.62:0/367132120 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272009bac0 con 0x5654f17db0e0 2024-04-02T14:45:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.689+0000 7f2738db2740 1 -- 172.21.15.62:0/367132120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17db0e0 msgr2=0x5654f17e3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.689+0000 7f2738db2740 1 --2- 172.21.15.62:0/367132120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17db0e0 0x5654f17e3d80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f272009cb90 tx=0x7f2720018520 comp rx=0 tx=0).stop 2024-04-02T14:45:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.689+0000 7f2738db2740 1 -- 172.21.15.62:0/367132120 shutdown_connections 2024-04-02T14:45:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.689+0000 7f2738db2740 1 --2- 172.21.15.62:0/367132120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17e45e0 0x5654f17d7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.689+0000 7f2738db2740 1 --2- 172.21.15.62:0/367132120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17dba00 0x5654f17f02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.689+0000 7f2738db2740 1 --2- 172.21.15.62:0/367132120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17db0e0 0x5654f17e3d80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.689+0000 7f2738db2740 1 -- 172.21.15.62:0/367132120 >> 172.21.15.62:0/367132120 conn(0x5654f17d5810 msgr2=0x5654f17d6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.689+0000 7f2738db2740 1 -- 172.21.15.62:0/367132120 shutdown_connections 2024-04-02T14:45:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.689+0000 7f2738db2740 1 -- 172.21.15.62:0/367132120 wait complete. 2024-04-02T14:45:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.690+0000 7f2738db2740 1 -- start start 2024-04-02T14:45:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.690+0000 7f2738db2740 1 Processor -- start 2024-04-02T14:45:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.690+0000 7f2738db2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 0x5654f17de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.690+0000 7f2738db2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 0x5654f17deca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.690+0000 7f2738db2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17e45e0 0x5654f1859f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.690+0000 7f2738db2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27301b0cc0 con 0x5654f17e45e0 2024-04-02T14:45:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.690+0000 7f2738db2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f27301b05f0 con 0x5654f17dba00 2024-04-02T14:45:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.690+0000 7f2738db2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27301ad850 con 0x5654f17db0e0 2024-04-02T14:45:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.691+0000 7f2738db2740 10 client.0 _ll_register_callbacks cb 0x7f272009c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.691+0000 7f272ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 0x5654f17deca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.691+0000 7f272ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 0x5654f17deca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37420/0 (socket says 172.21.15.62:37420) 2024-04-02T14:45:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.691+0000 7f272ea00640 1 -- 172.21.15.62:0/3594537193 learned_addr learned my addr 172.21.15.62:0/3594537193 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.691+0000 7f272f400640 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 0x5654f17de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.691+0000 7f272fe00640 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17e45e0 0x5654f1859f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.691+0000 7f272ea00640 1 -- 172.21.15.62:0/3594537193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 msgr2=0x5654f17de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.691+0000 7f272ea00640 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 0x5654f17de760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.691+0000 7f272ea00640 1 -- 172.21.15.62:0/3594537193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17e45e0 msgr2=0x5654f1859f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.691+0000 7f272ea00640 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17e45e0 0x5654f1859f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.691+0000 7f272ea00640 1 -- 172.21.15.62:0/3594537193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f1520620 con 0x5654f17dba00 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.692+0000 7f272ea00640 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 0x5654f17deca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f271c01b040 tx=0x7f271c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aff6ecbe389be7f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.692+0000 7f2718c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.692+0000 7f2718c00640 1 -- 172.21.15.62:0/3594537193 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c029d60 con 0x5654f17dba00 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.692+0000 7f2718c00640 1 -- 172.21.15.62:0/3594537193 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f271c0052d0 con 0x5654f17dba00 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.692+0000 7f2738db2740 1 -- 172.21.15.62:0/3594537193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f272009c050 con 0x5654f17dba00 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.692+0000 7f2718c00640 1 -- 172.21.15.62:0/3594537193 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f271c038050 con 0x5654f17dba00 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.692+0000 7f2738db2740 10 client.25916 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.692+0000 7f2738db2740 1 -- 172.21.15.62:0/3594537193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654f185b4f0 con 0x5654f17dba00 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.693+0000 7f2718c00640 1 -- 172.21.15.62:0/3594537193 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f272009c050 con 0x5654f17dba00 2024-04-02T14:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.693+0000 7f2718c00640 1 -- 172.21.15.62:0/3594537193 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f271c0294b0 con 0x5654f17dba00 2024-04-02T14:45:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.693+0000 7f2738db2740 10 client.25916 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.693+0000 7f2738db2740 10 client.25916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.693+0000 7f2738db2740 20 client.25916 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.693+0000 7f2738db2740 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5654f185bad0 0x5654f185def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.693+0000 7f2738db2740 4 client.25916 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:45:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.693+0000 7f2738db2740 1 -- 172.21.15.62:0/3594537193 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654f174d2b0 con 0x5654f185bad0 2024-04-02T14:45:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.694+0000 7f272f400640 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5654f185bad0 0x5654f185def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.694+0000 7f272f400640 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5654f185bad0 0x5654f185def0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b8f350a3f9d0c8 server_cookie=1e646c5c2c7b0911 in_seq=0 out_seq=0 2024-04-02T14:45:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.694+0000 7f2718c00640 10 client.25916 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2718c00640 1 -- 172.21.15.62:0/3594537193 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654f174d2b0 con 0x5654f185bad0 2024-04-02T14:45:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2718c00640 10 client.25916 handle_command_reply: tid=0 2024-04-02T14:45:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2738db2740 1 client.25916 shutdown 2024-04-02T14:45:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2738db2740 1 -- 172.21.15.62:0/3594537193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 msgr2=0x5654f17deca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2738db2740 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 0x5654f17deca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f271c01b040 tx=0x7f271c005d30 comp rx=0 tx=0).stop 2024-04-02T14:45:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2738db2740 1 -- 172.21.15.62:0/3594537193 shutdown_connections 2024-04-02T14:45:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2738db2740 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5654f185bad0 0x5654f185def0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2738db2740 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654f17e45e0 0x5654f1859f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2738db2740 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5654f17dba00 0x5654f17deca0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2738db2740 1 --2- 172.21.15.62:0/3594537193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f17db0e0 0x5654f17de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2718c00640 0 client.25916 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.700+0000 7f2738db2740 1 -- 172.21.15.62:0/3594537193 >> 172.21.15.62:0/3594537193 conn(0x5654f17d5810 msgr2=0x5654f17d74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.701+0000 7f2738db2740 1 -- 172.21.15.62:0/3594537193 shutdown_connections 2024-04-02T14:45:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.701+0000 7f2738db2740 1 -- 172.21.15.62:0/3594537193 wait complete. 2024-04-02T14:45:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.701+0000 7f2738db2740 20 client.25916 trim_cache size 0 max 16384 2024-04-02T14:45:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:45:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:45:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:45:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:45:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.701+0000 7f2734a00640 1 -- 172.21.15.62:0/1707102370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f27140a3ec0 msgr2=0x7f27140a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.701+0000 7f2734a00640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f27140a3ec0 0x7f27140a6380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2728005e30 tx=0x7f2728005d80 comp rx=0 tx=0).stop 2024-04-02T14:45:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.701+0000 7f2734a00640 1 -- 172.21.15.62:0/1707102370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2730101900 msgr2=0x7f2730073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.701+0000 7f2734a00640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2730101900 0x7f2730073e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f272000e9f0 tx=0x7f272000eeb0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.702+0000 7f2734a00640 1 -- 172.21.15.62:0/1707102370 shutdown_connections 2024-04-02T14:45:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.702+0000 7f2734a00640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f27140a3ec0 0x7f27140a6380 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.702+0000 7f2734a00640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2730107910 0x7f2730074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.702+0000 7f2734a00640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27301022c0 0x7f2730074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.702+0000 7f2734a00640 1 --2- 172.21.15.62:0/1707102370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2730101900 0x7f2730073e00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.702+0000 7f2734a00640 1 -- 172.21.15.62:0/1707102370 >> 172.21.15.62:0/1707102370 conn(0x7f27300fd410 msgr2=0x7f27300fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.702+0000 7f2734a00640 1 -- 172.21.15.62:0/1707102370 shutdown_connections 2024-04-02T14:45:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:18.702+0000 7f2734a00640 1 -- 172.21.15.62:0/1707102370 wait complete. 2024-04-02T14:45:18.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:18 smithi204 ceph-mon[35948]: scrub queued for path: / 2024-04-02T14:45:18.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:18 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:18.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:18 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:45:18.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:18 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:18.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:18 smithi062 ceph-mon[32116]: scrub queued for path: / 2024-04-02T14:45:18.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:18 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:18.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:18 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:45:18.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:18 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:18.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:18 smithi113 ceph-mon[36135]: scrub queued for path: / 2024-04-02T14:45:18.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:18 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:18.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:18 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:45:18.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:18 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:19.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:19 smithi204 ceph-mon[35948]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-04-02T14:45:19.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:19 smithi062 ceph-mon[32116]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-04-02T14:45:19.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:19 smithi113 ceph-mon[36135]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-04-02T14:45:20.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:20 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:45:20.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:20 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:45:20.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:20 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:45:21.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:21 smithi204 ceph-mon[35948]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:45:21.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:21 smithi062 ceph-mon[32116]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:45:21.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:21 smithi113 ceph-mon[36135]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:45:23.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:23 smithi204 ceph-mon[35948]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-02T14:45:23.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:23 smithi062 ceph-mon[32116]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-02T14:45:23.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:23 smithi113 ceph-mon[36135]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-02T14:45:25.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:25 smithi204 ceph-mon[35948]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T14:45:25.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:25 smithi062 ceph-mon[32116]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T14:45:25.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:25 smithi113 ceph-mon[36135]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T14:45:27.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:27 smithi204 ceph-mon[35948]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-02T14:45:27.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:27 smithi062 ceph-mon[32116]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-02T14:45:27.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:27 smithi113 ceph-mon[36135]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-02T14:45:29.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:29 smithi204 ceph-mon[35948]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-02T14:45:29.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:29 smithi062 ceph-mon[32116]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-02T14:45:29.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:29 smithi113 ceph-mon[36135]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-02T14:45:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:31 smithi204 ceph-mon[35948]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T14:45:31.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:31 smithi062 ceph-mon[32116]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T14:45:31.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:31 smithi113 ceph-mon[36135]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T14:45:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.888+0000 7fb78ac00640 1 -- 172.21.15.62:0/2746977206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb784102ab0 msgr2=0x7fb784102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.888+0000 7fb78ac00640 1 --2- 172.21.15.62:0/2746977206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb784102ab0 0x7fb784102e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb778009860 tx=0x7fb77802bc00 comp rx=0 tx=0).stop 2024-04-02T14:45:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.889+0000 7fb78ac00640 1 -- 172.21.15.62:0/2746977206 shutdown_connections 2024-04-02T14:45:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.889+0000 7fb78ac00640 1 --2- 172.21.15.62:0/2746977206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb78410bed0 0x7fb78410e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.889+0000 7fb78ac00640 1 --2- 172.21.15.62:0/2746977206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7841033d0 0x7fb78410b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.889+0000 7fb78ac00640 1 --2- 172.21.15.62:0/2746977206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb784102ab0 0x7fb784102e90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.889+0000 7fb78ac00640 1 -- 172.21.15.62:0/2746977206 >> 172.21.15.62:0/2746977206 conn(0x7fb7840fc310 msgr2=0x7fb7840fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.889+0000 7fb78ac00640 1 -- 172.21.15.62:0/2746977206 shutdown_connections 2024-04-02T14:45:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.889+0000 7fb78ac00640 1 -- 172.21.15.62:0/2746977206 wait complete. 2024-04-02T14:45:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.890+0000 7fb78ac00640 1 Processor -- start 2024-04-02T14:45:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.890+0000 7fb78ac00640 1 -- start start 2024-04-02T14:45:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.891+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb784102ab0 0x7fb784100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.891+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7841033d0 0x7fb784100c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.891+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb78410bed0 0x7fb784101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.891+0000 7fb78ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb784111df0 con 0x7fb78410bed0 2024-04-02T14:45:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.891+0000 7fb78ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb784111c70 con 0x7fb784102ab0 2024-04-02T14:45:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.891+0000 7fb78ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb784111f70 con 0x7fb7841033d0 2024-04-02T14:45:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.891+0000 7fb78a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb78410bed0 0x7fb784101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.891+0000 7fb789800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb784102ab0 0x7fb784100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.892+0000 7fb78a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb78410bed0 0x7fb784101180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37790/0 (socket says 172.21.15.62:37790) 2024-04-02T14:45:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.892+0000 7fb78a200640 1 -- 172.21.15.62:0/1004941453 learned_addr learned my addr 172.21.15.62:0/1004941453 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.892+0000 7fb788e00640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7841033d0 0x7fb784100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.892+0000 7fb789800640 1 -- 172.21.15.62:0/1004941453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7841033d0 msgr2=0x7fb784100c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.892+0000 7fb789800640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7841033d0 0x7fb784100c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.892+0000 7fb789800640 1 -- 172.21.15.62:0/1004941453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb78410bed0 msgr2=0x7fb784101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.892+0000 7fb789800640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb78410bed0 0x7fb784101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.892+0000 7fb789800640 1 -- 172.21.15.62:0/1004941453 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7841a6790 con 0x7fb784102ab0 2024-04-02T14:45:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.892+0000 7fb789800640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb784102ab0 0x7fb784100700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb778037040 tx=0x7fb77802c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.893+0000 7fb782000640 1 -- 172.21.15.62:0/1004941453 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778041020 con 0x7fb784102ab0 2024-04-02T14:45:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.893+0000 7fb782000640 1 -- 172.21.15.62:0/1004941453 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb77802ce50 con 0x7fb784102ab0 2024-04-02T14:45:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.893+0000 7fb78ac00640 1 -- 172.21.15.62:0/1004941453 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7841a6a20 con 0x7fb784102ab0 2024-04-02T14:45:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.893+0000 7fb782000640 1 -- 172.21.15.62:0/1004941453 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778039040 con 0x7fb784102ab0 2024-04-02T14:45:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.893+0000 7fb78ac00640 1 -- 172.21.15.62:0/1004941453 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7841a6f10 con 0x7fb784102ab0 2024-04-02T14:45:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.897+0000 7fb78ef90740 1 Processor -- start 2024-04-02T14:45:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.898+0000 7fb78ef90740 1 -- start start 2024-04-02T14:45:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.898+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.898+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c7c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.898+0000 7fb782000640 1 -- 172.21.15.62:0/1004941453 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7fb778044020 con 0x7fb784102ab0 2024-04-02T14:45:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.898+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c7e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.898+0000 7fb78ef90740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565430bd7d70 con 0x565430c730e0 2024-04-02T14:45:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.898+0000 7fb78ef90740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x565430b802a0 con 0x565430c7c5e0 2024-04-02T14:45:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.898+0000 7fb78ef90740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x565430bd3d20 con 0x565430c73a00 2024-04-02T14:45:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.899+0000 7fb78a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.899+0000 7fb78a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37802/0 (socket says 172.21.15.62:37802) 2024-04-02T14:45:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.899+0000 7fb78a200640 1 -- 172.21.15.62:0/3657195812 learned_addr learned my addr 172.21.15.62:0/3657195812 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.899+0000 7fb788e00640 1 --2- 172.21.15.62:0/3657195812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c7e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.899+0000 7fb789800640 1 --2- 172.21.15.62:0/3657195812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c7c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.899+0000 7fb788e00640 1 -- 172.21.15.62:0/3657195812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 msgr2=0x565430c7c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.899+0000 7fb788e00640 1 --2- 172.21.15.62:0/3657195812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c7c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.899+0000 7fb788e00640 1 -- 172.21.15.62:0/3657195812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 msgr2=0x565430c734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.899+0000 7fb788e00640 1 --2- 172.21.15.62:0/3657195812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.899+0000 7fb788e00640 1 -- 172.21.15.62:0/3657195812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565430b94840 con 0x565430c7c5e0 2024-04-02T14:45:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.900+0000 7fb788e00640 1 --2- 172.21.15.62:0/3657195812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c7e9d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb770009bc0 tx=0x7fb770011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f8ba51113d7142a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.900+0000 7fb776a00640 1 -- 172.21.15.62:0/3657195812 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770004230 con 0x565430c7c5e0 2024-04-02T14:45:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.900+0000 7fb776a00640 1 -- 172.21.15.62:0/3657195812 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb770009e30 con 0x565430c7c5e0 2024-04-02T14:45:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.900+0000 7fb776a00640 1 -- 172.21.15.62:0/3657195812 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770004e50 con 0x565430c7c5e0 2024-04-02T14:45:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.900+0000 7fb782000640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb76c0a3ef0 0x7fb76c0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.901+0000 7fb782000640 1 -- 172.21.15.62:0/1004941453 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fb7780f1df0 con 0x7fb784102ab0 2024-04-02T14:45:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.901+0000 7fb78a200640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb76c0a3ef0 0x7fb76c0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.901+0000 7fb78ef90740 1 -- 172.21.15.62:0/3657195812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 msgr2=0x565430c7e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.901+0000 7fb78ef90740 1 --2- 172.21.15.62:0/3657195812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c7e9d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb770009bc0 tx=0x7fb770011810 comp rx=0 tx=0).stop 2024-04-02T14:45:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.902+0000 7fb78ef90740 1 -- 172.21.15.62:0/3657195812 shutdown_connections 2024-04-02T14:45:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.902+0000 7fb78ef90740 1 --2- 172.21.15.62:0/3657195812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c7e9d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.902+0000 7fb78ef90740 1 --2- 172.21.15.62:0/3657195812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c7c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.902+0000 7fb78ef90740 1 --2- 172.21.15.62:0/3657195812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.902+0000 7fb78ef90740 1 -- 172.21.15.62:0/3657195812 >> 172.21.15.62:0/3657195812 conn(0x565430c6d810 msgr2=0x565430c6dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.902+0000 7fb78ef90740 1 -- 172.21.15.62:0/3657195812 shutdown_connections 2024-04-02T14:45:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.902+0000 7fb78ef90740 1 -- 172.21.15.62:0/3657195812 wait complete. 2024-04-02T14:45:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.902+0000 7fb78a200640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb76c0a3ef0 0x7fb76c0a63b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009590 tx=0x7fb77c010040 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.903+0000 7fb78ef90740 1 -- start start 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.903+0000 7fb78ef90740 1 Processor -- start 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c79b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c7a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb78ef90740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565430c7f3d0 con 0x565430c730e0 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb78ef90740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x565430b93660 con 0x565430c7c5e0 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb78ef90740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x565430c7f550 con 0x565430c73a00 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb78ef90740 10 client.0 _ll_register_callbacks cb 0x565430b93050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb789800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb78a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c7a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb78a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c7a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:42112/0 (socket says 172.21.15.62:42112) 2024-04-02T14:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb78a200640 1 -- 172.21.15.62:0/2889171934 learned_addr learned my addr 172.21.15.62:0/2889171934 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.904+0000 7fb788e00640 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c79b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.905+0000 7fb788e00640 1 -- 172.21.15.62:0/2889171934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 msgr2=0x565430c7a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.905+0000 7fb788e00640 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c7a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.905+0000 7fb788e00640 1 -- 172.21.15.62:0/2889171934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 msgr2=0x565430c795c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.905+0000 7fb788e00640 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c795c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.905+0000 7fb788e00640 1 -- 172.21.15.62:0/2889171934 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565430ba2e80 con 0x565430c73a00 2024-04-02T14:45:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.905+0000 7fb789800640 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.905+0000 7fb788e00640 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c79b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb770009bc0 tx=0x7fb7700048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=650c3c2ff857a5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.905+0000 7fb76b400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:45:31.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.906+0000 7fb76b400640 1 -- 172.21.15.62:0/2889171934 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770027020 con 0x565430c73a00 2024-04-02T14:45:31.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.906+0000 7fb78ef90740 1 -- 172.21.15.62:0/2889171934 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb770009e30 con 0x565430c73a00 2024-04-02T14:45:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.906+0000 7fb76b400640 1 -- 172.21.15.62:0/2889171934 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fb7700037f0 con 0x565430c73a00 2024-04-02T14:45:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.906+0000 7fb76b400640 1 -- 172.21.15.62:0/2889171934 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb770032050 con 0x565430c73a00 2024-04-02T14:45:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.906+0000 7fb78ef90740 10 client.25903 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.906+0000 7fb78ef90740 1 -- 172.21.15.62:0/2889171934 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654308ec210 con 0x565430c73a00 2024-04-02T14:45:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.907+0000 7fb76b400640 1 -- 172.21.15.62:0/2889171934 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb770009e30 con 0x565430c73a00 2024-04-02T14:45:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.907+0000 7fb76b400640 1 -- 172.21.15.62:0/2889171934 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb770018070 con 0x565430c73a00 2024-04-02T14:45:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.907+0000 7fb78ef90740 10 client.25903 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.907+0000 7fb78ef90740 10 client.25903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.907+0000 7fb78ef90740 20 client.25903 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.908+0000 7fb78ef90740 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x565430c950c0 0x565430c974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.908+0000 7fb78ef90740 4 client.25903 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.908+0000 7fb78ef90740 1 -- 172.21.15.62:0/2889171934 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565430b87cd0 con 0x565430c950c0 2024-04-02T14:45:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.908+0000 7fb789800640 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x565430c950c0 0x565430c974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.909+0000 7fb789800640 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x565430c950c0 0x565430c974b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996514e610ec255f server_cookie=296044b738fe2fa3 in_seq=0 out_seq=0 2024-04-02T14:45:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.909+0000 7fb76b400640 10 client.25903 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.910+0000 7fb76b400640 1 -- 172.21.15.62:0/2889171934 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565430b87cd0 con 0x565430c950c0 2024-04-02T14:45:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.910+0000 7fb76b400640 10 client.25903 handle_command_reply: tid=0 2024-04-02T14:45:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.910+0000 7fb78ef90740 1 client.25903 shutdown 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.911+0000 7fb78ef90740 1 -- 172.21.15.62:0/2889171934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 msgr2=0x565430c79b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.911+0000 7fb78ef90740 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c79b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb770009bc0 tx=0x7fb7700048f0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.911+0000 7fb78ef90740 1 -- 172.21.15.62:0/2889171934 shutdown_connections 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.911+0000 7fb78ef90740 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x565430c950c0 0x565430c974b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.911+0000 7fb78ef90740 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c7a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.911+0000 7fb78ef90740 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c79b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.911+0000 7fb78ef90740 1 --2- 172.21.15.62:0/2889171934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.911+0000 7fb78ef90740 1 -- 172.21.15.62:0/2889171934 >> 172.21.15.62:0/2889171934 conn(0x565430c6d810 msgr2=0x565430c6f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.911+0000 7fb76b400640 0 client.25903 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.911+0000 7fb78ef90740 1 -- 172.21.15.62:0/2889171934 shutdown_connections 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.912+0000 7fb78ef90740 1 -- 172.21.15.62:0/2889171934 wait complete. 2024-04-02T14:45:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.912+0000 7fb78ef90740 20 client.25903 trim_cache size 0 max 16384 2024-04-02T14:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.943+0000 7fb78ef90740 1 Processor -- start 2024-04-02T14:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.943+0000 7fb78ef90740 1 -- start start 2024-04-02T14:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.943+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c7bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.944+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c90b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.944+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c9cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.944+0000 7fb78ef90740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841ab3a0 con 0x565430c730e0 2024-04-02T14:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.944+0000 7fb78ef90740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841acee0 con 0x565430c7c5e0 2024-04-02T14:45:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.944+0000 7fb78ef90740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841aba70 con 0x565430c73a00 2024-04-02T14:45:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.944+0000 7fb789800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c7bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.944+0000 7fb789800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c7bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37826/0 (socket says 172.21.15.62:37826) 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.944+0000 7fb789800640 1 -- 172.21.15.62:0/1718204476 learned_addr learned my addr 172.21.15.62:0/1718204476 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.944+0000 7fb788e00640 1 --2- 172.21.15.62:0/1718204476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c90b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.944+0000 7fb78a200640 1 --2- 172.21.15.62:0/1718204476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c9cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.945+0000 7fb788e00640 1 -- 172.21.15.62:0/1718204476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 msgr2=0x565430c9cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.945+0000 7fb788e00640 1 --2- 172.21.15.62:0/1718204476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c9cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.945+0000 7fb788e00640 1 -- 172.21.15.62:0/1718204476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 msgr2=0x565430c7bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.945+0000 7fb788e00640 1 --2- 172.21.15.62:0/1718204476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c7bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.945+0000 7fb788e00640 1 -- 172.21.15.62:0/1718204476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565430c78570 con 0x565430c73a00 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.945+0000 7fb789800640 1 --2- 172.21.15.62:0/1718204476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c7bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.945+0000 7fb788e00640 1 --2- 172.21.15.62:0/1718204476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c90b20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb770023cc0 tx=0x7fb770004960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50708ce0d8d5023 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.945+0000 7fb776000640 1 -- 172.21.15.62:0/1718204476 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770027070 con 0x565430c73a00 2024-04-02T14:45:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.945+0000 7fb776000640 1 -- 172.21.15.62:0/1718204476 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb770009770 con 0x565430c73a00 2024-04-02T14:45:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.945+0000 7fb776000640 1 -- 172.21.15.62:0/1718204476 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77001e040 con 0x565430c73a00 2024-04-02T14:45:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.946+0000 7fb78ef90740 1 -- 172.21.15.62:0/1718204476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 msgr2=0x565430c90b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.946+0000 7fb78ef90740 1 --2- 172.21.15.62:0/1718204476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c90b20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb770023cc0 tx=0x7fb770004960 comp rx=0 tx=0).stop 2024-04-02T14:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.946+0000 7fb78ef90740 1 -- 172.21.15.62:0/1718204476 shutdown_connections 2024-04-02T14:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.946+0000 7fb78ef90740 1 --2- 172.21.15.62:0/1718204476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c9cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.946+0000 7fb78ef90740 1 --2- 172.21.15.62:0/1718204476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c73a00 0x565430c90b20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.946+0000 7fb78ef90740 1 --2- 172.21.15.62:0/1718204476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c730e0 0x565430c7bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.946+0000 7fb78ef90740 1 -- 172.21.15.62:0/1718204476 >> 172.21.15.62:0/1718204476 conn(0x565430c6d810 msgr2=0x565430c6dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.946+0000 7fb78ef90740 1 -- 172.21.15.62:0/1718204476 shutdown_connections 2024-04-02T14:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.946+0000 7fb78ef90740 1 -- 172.21.15.62:0/1718204476 wait complete. 2024-04-02T14:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.947+0000 7fb78ef90740 1 -- start start 2024-04-02T14:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.947+0000 7fb78ef90740 1 Processor -- start 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c730e0 0x565430c8d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c76040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb78ef90740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c766e0 0x565430cfa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb78ef90740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841ac810 con 0x565430c766e0 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb78ef90740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841ac140 con 0x565430c7c5e0 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb78ef90740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841a93a0 con 0x565430c730e0 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb78ef90740 10 client.0 _ll_register_callbacks cb 0x7fb7700043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb789800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c730e0 0x565430c8d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb789800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c730e0 0x565430c8d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:51250/0 (socket says 172.21.15.62:51250) 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb789800640 1 -- 172.21.15.62:0/4127913121 learned_addr learned my addr 172.21.15.62:0/4127913121 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb78a200640 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c766e0 0x565430cfa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.948+0000 7fb788e00640 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c76040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.949+0000 7fb789800640 1 -- 172.21.15.62:0/4127913121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 msgr2=0x565430c76040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.949+0000 7fb789800640 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c76040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.949+0000 7fb789800640 1 -- 172.21.15.62:0/4127913121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c766e0 msgr2=0x565430cfa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.949+0000 7fb789800640 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c766e0 0x565430cfa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.949+0000 7fb789800640 1 -- 172.21.15.62:0/4127913121 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565430c743b0 con 0x565430c730e0 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.949+0000 7fb788e00640 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c76040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.949+0000 7fb78a200640 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c766e0 0x565430cfa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.949+0000 7fb789800640 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c730e0 0x565430c8d5b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb77800b230 tx=0x7fb7780038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60a99af2bb5416f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.949+0000 7fb76b400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.950+0000 7fb76b400640 1 -- 172.21.15.62:0/4127913121 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7780c2060 con 0x565430c730e0 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.950+0000 7fb76b400640 1 -- 172.21.15.62:0/4127913121 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fb7780bbd50 con 0x565430c730e0 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.950+0000 7fb78ef90740 1 -- 172.21.15.62:0/4127913121 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb770009770 con 0x565430c730e0 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.950+0000 7fb76b400640 1 -- 172.21.15.62:0/4127913121 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7780d6030 con 0x565430c730e0 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.950+0000 7fb78ef90740 10 client.25909 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.950+0000 7fb78ef90740 1 -- 172.21.15.62:0/4127913121 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654309b8620 con 0x565430c730e0 2024-04-02T14:45:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.950+0000 7fb76b400640 1 -- 172.21.15.62:0/4127913121 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb770009770 con 0x565430c730e0 2024-04-02T14:45:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.951+0000 7fb76b400640 1 -- 172.21.15.62:0/4127913121 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb7780efce0 con 0x565430c730e0 2024-04-02T14:45:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.951+0000 7fb78ef90740 10 client.25909 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.951+0000 7fb78ef90740 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.951+0000 7fb78ef90740 20 client.25909 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.951+0000 7fb78ef90740 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x565430cfbdb0 0x565430cfe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.951+0000 7fb78ef90740 4 client.25909 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:45:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.951+0000 7fb78ef90740 1 -- 172.21.15.62:0/4127913121 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565430be52b0 con 0x565430cfbdb0 2024-04-02T14:45:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.951+0000 7fb788e00640 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x565430cfbdb0 0x565430cfe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.952+0000 7fb788e00640 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x565430cfbdb0 0x565430cfe1d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ca743fa4026386 server_cookie=b145a418611cbb9c in_seq=0 out_seq=0 2024-04-02T14:45:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.952+0000 7fb76b400640 10 client.25909 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb76b400640 1 -- 172.21.15.62:0/4127913121 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565430be52b0 con 0x565430cfbdb0 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb76b400640 10 client.25909 handle_command_reply: tid=0 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb78ef90740 1 client.25909 shutdown 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb78ef90740 1 -- 172.21.15.62:0/4127913121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c730e0 msgr2=0x565430c8d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb78ef90740 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c730e0 0x565430c8d5b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb77800b230 tx=0x7fb7780038b0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb78ef90740 1 -- 172.21.15.62:0/4127913121 shutdown_connections 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb78ef90740 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x565430cfbdb0 0x565430cfe1d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb76b400640 0 client.25909 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb78ef90740 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565430c766e0 0x565430cfa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb78ef90740 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565430c7c5e0 0x565430c76040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb78ef90740 1 --2- 172.21.15.62:0/4127913121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565430c730e0 0x565430c8d5b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.958+0000 7fb78ef90740 1 -- 172.21.15.62:0/4127913121 >> 172.21.15.62:0/4127913121 conn(0x565430c6d810 msgr2=0x565430c6ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.959+0000 7fb78ef90740 1 -- 172.21.15.62:0/4127913121 shutdown_connections 2024-04-02T14:45:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.959+0000 7fb78ef90740 1 -- 172.21.15.62:0/4127913121 wait complete. 2024-04-02T14:45:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.959+0000 7fb78ef90740 20 client.25909 trim_cache size 0 max 16384 2024-04-02T14:45:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:45:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:45:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:45:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 -- 172.21.15.62:0/1004941453 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb76c0a3ef0 msgr2=0x7fb76c0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb76c0a3ef0 0x7fb76c0a63b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009590 tx=0x7fb77c010040 comp rx=0 tx=0).stop 2024-04-02T14:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 -- 172.21.15.62:0/1004941453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb784102ab0 msgr2=0x7fb784100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb784102ab0 0x7fb784100700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb778037040 tx=0x7fb77802c850 comp rx=0 tx=0).stop 2024-04-02T14:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 -- 172.21.15.62:0/1004941453 shutdown_connections 2024-04-02T14:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb76c0a3ef0 0x7fb76c0a63b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb78410bed0 0x7fb784101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7841033d0 0x7fb784100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 --2- 172.21.15.62:0/1004941453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb784102ab0 0x7fb784100700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 -- 172.21.15.62:0/1004941453 >> 172.21.15.62:0/1004941453 conn(0x7fb7840fc310 msgr2=0x7fb7840fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 -- 172.21.15.62:0/1004941453 shutdown_connections 2024-04-02T14:45:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:31.960+0000 7fb78ac00640 1 -- 172.21.15.62:0/1004941453 wait complete. 2024-04-02T14:45:33.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:33 smithi062 ceph-mon[32116]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-02T14:45:33.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:33 smithi062 ceph-mon[32116]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-02T14:45:33.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:33 smithi113 ceph-mon[36135]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-02T14:45:33.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:33 smithi113 ceph-mon[36135]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-02T14:45:34.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:33 smithi204 ceph-mon[35948]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-02T14:45:34.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:33 smithi204 ceph-mon[35948]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-02T14:45:35.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:35 smithi204 ceph-mon[35948]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-02T14:45:35.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:35 smithi062 ceph-mon[32116]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-02T14:45:35.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:35 smithi113 ceph-mon[36135]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-02T14:45:37.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:37 smithi062 ceph-mon[32116]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-02T14:45:37.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:37 smithi113 ceph-mon[36135]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-02T14:45:38.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:37 smithi204 ceph-mon[35948]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-02T14:45:38.354 INFO:tasks.workunit.client.0.smithi062.stderr:Updating files: 59% (7360/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 99% (12226/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-02T14:45:38.969 INFO:tasks.workunit.client.0.smithi062.stderr:Note: switching to '191d963ac9de7322da8aff20b567b3fa458e2ef6'. 2024-04-02T14:45:38.970 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-04-02T14:45:38.970 INFO:tasks.workunit.client.0.smithi062.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-02T14:45:38.970 INFO:tasks.workunit.client.0.smithi062.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-02T14:45:38.970 INFO:tasks.workunit.client.0.smithi062.stderr:state without impacting any branches by switching back to a branch. 2024-04-02T14:45:38.970 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-04-02T14:45:38.970 INFO:tasks.workunit.client.0.smithi062.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-02T14:45:38.970 INFO:tasks.workunit.client.0.smithi062.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-02T14:45:38.970 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-04-02T14:45:38.970 INFO:tasks.workunit.client.0.smithi062.stderr: git switch -c 2024-04-02T14:45:38.970 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-04-02T14:45:38.970 INFO:tasks.workunit.client.0.smithi062.stderr:Or undo this operation with: 2024-04-02T14:45:38.971 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-04-02T14:45:38.971 INFO:tasks.workunit.client.0.smithi062.stderr: git switch - 2024-04-02T14:45:38.971 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-04-02T14:45:38.971 INFO:tasks.workunit.client.0.smithi062.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-02T14:45:38.971 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-04-02T14:45:38.971 INFO:tasks.workunit.client.0.smithi062.stderr:HEAD is now at 191d963ac9d Revert "Merge pull request #55436 from tchaikov/mgr-python-3.12" 2024-04-02T14:45:38.982 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-02T14:45:39.089 INFO:tasks.workunit.client.0.smithi062.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-02T14:45:39.097 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T14:45:39.097 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-02T14:45:39.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:39 smithi062 ceph-mon[32116]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-02T14:45:39.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:39 smithi113 ceph-mon[36135]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-02T14:45:40.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:39 smithi204 ceph-mon[35948]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-02T14:45:40.070 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-02T14:45:40.240 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-02T14:45:40.320 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T14:45:40.325 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T14:45:40.326 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-02T14:45:40.412 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T14:45:40.424 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:45:40.424 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-02T14:45:40.475 INFO:tasks.workunit:Running workunits matching fs/norstats on client.0... 2024-04-02T14:45:40.476 INFO:tasks.workunit:Running workunit fs/norstats/kernel_untar_tar.sh... 2024-04-02T14:45:40.477 DEBUG:teuthology.orchestra.run.smithi062:workunit test fs/norstats/kernel_untar_tar.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=191d963ac9de7322da8aff20b567b3fa458e2ef6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh 2024-04-02T14:45:40.557 INFO:tasks.workunit.client.0.smithi062.stderr:+ KERNEL=linux-4.0.5 2024-04-02T14:45:40.557 INFO:tasks.workunit.client.0.smithi062.stderr:+ wget -q http://download.ceph.com/qa/linux-4.0.5.tar.xz 2024-04-02T14:45:41.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:41 smithi062 ceph-mon[32116]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-02T14:45:41.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:41 smithi113 ceph-mon[36135]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-02T14:45:42.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:41 smithi204 ceph-mon[35948]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-02T14:45:43.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:43 smithi062 ceph-mon[32116]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-02T14:45:43.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:43 smithi113 ceph-mon[36135]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-02T14:45:44.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:43 smithi204 ceph-mon[35948]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-02T14:45:45.010 INFO:tasks.workunit.client.0.smithi062.stderr:+ mkdir untar_tar 2024-04-02T14:45:45.019 INFO:tasks.workunit.client.0.smithi062.stderr:+ cd untar_tar 2024-04-02T14:45:45.019 INFO:tasks.workunit.client.0.smithi062.stderr:+ tar Jxvf ../linux-4.0.5.tar.xz linux-4.0.5/Documentation/ 2024-04-02T14:45:45.105 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ 2024-04-02T14:45:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.110+0000 7f4242400640 1 -- 172.21.15.62:0/2034336863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f423c109b90 msgr2=0x7f423c109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.110+0000 7f4242400640 1 --2- 172.21.15.62:0/2034336863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f423c109b90 0x7f423c109f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4230009900 tx=0x7f423002bc30 comp rx=0 tx=0).stop 2024-04-02T14:45:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.111+0000 7f4242400640 1 -- 172.21.15.62:0/2034336863 shutdown_connections 2024-04-02T14:45:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.111+0000 7f4242400640 1 --2- 172.21.15.62:0/2034336863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c1044b0 0x7f423c1109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.111+0000 7f4242400640 1 --2- 172.21.15.62:0/2034336863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423c103af0 0x7f423c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.111+0000 7f4242400640 1 --2- 172.21.15.62:0/2034336863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f423c109b90 0x7f423c109f70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.111+0000 7f4242400640 1 -- 172.21.15.62:0/2034336863 >> 172.21.15.62:0/2034336863 conn(0x7f423c0ff5e0 msgr2=0x7f423c101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.111+0000 7f4242400640 1 -- 172.21.15.62:0/2034336863 shutdown_connections 2024-04-02T14:45:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.111+0000 7f4242400640 1 -- 172.21.15.62:0/2034336863 wait complete. 2024-04-02T14:45:45.107 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/00-INDEX 2024-04-02T14:45:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.112+0000 7f4242400640 1 Processor -- start 2024-04-02T14:45:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.112+0000 7f4242400640 1 -- start start 2024-04-02T14:45:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4242400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c103af0 0x7f423c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4242400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423c1044b0 0x7f423c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4242400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f423c109b90 0x7f423c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4242400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f423c114030 con 0x7f423c1044b0 2024-04-02T14:45:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4242400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f423c113eb0 con 0x7f423c109b90 2024-04-02T14:45:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4242400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1141b0 con 0x7f423c103af0 2024-04-02T14:45:45.109 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/ 2024-04-02T14:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4241000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c103af0 0x7f423c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f423be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423c1044b0 0x7f423c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4241000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c103af0 0x7f423c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:41968/0 (socket says 172.21.15.62:41968) 2024-04-02T14:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4241000640 1 -- 172.21.15.62:0/663867786 learned_addr learned my addr 172.21.15.62:0/663867786 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4241a00640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f423c109b90 0x7f423c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4241000640 1 -- 172.21.15.62:0/663867786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f423c109b90 msgr2=0x7f423c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4241000640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f423c109b90 0x7f423c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4241000640 1 -- 172.21.15.62:0/663867786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423c1044b0 msgr2=0x7f423c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4241000640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423c1044b0 0x7f423c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.113+0000 7f4241000640 1 -- 172.21.15.62:0/663867786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c19f4a0 con 0x7f423c103af0 2024-04-02T14:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.114+0000 7f4241000640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c103af0 0x7f423c19a4b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4230005ab0 tx=0x7f423002dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:45.111 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/README 2024-04-02T14:45:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.114+0000 7f4239600640 1 -- 172.21.15.62:0/663867786 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230043020 con 0x7f423c103af0 2024-04-02T14:45:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.114+0000 7f4239600640 1 -- 172.21.15.62:0/663867786 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4230004250 con 0x7f423c103af0 2024-04-02T14:45:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.114+0000 7f4239600640 1 -- 172.21.15.62:0/663867786 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230003ab0 con 0x7f423c103af0 2024-04-02T14:45:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.114+0000 7f4242400640 1 -- 172.21.15.62:0/663867786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c19f730 con 0x7f423c103af0 2024-04-02T14:45:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.114+0000 7f4242400640 1 -- 172.21.15.62:0/663867786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c1a7020 con 0x7f423c103af0 2024-04-02T14:45:45.112 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/obsolete/ 2024-04-02T14:45:45.112 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/obsolete/proc-sys-vm-nr_pdflush_threads 2024-04-02T14:45:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.118+0000 7f4239600640 1 -- 172.21.15.62:0/663867786 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4230046020 con 0x7f423c103af0 2024-04-02T14:45:45.113 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-bus-usb 2024-04-02T14:45:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.118+0000 7f4246784740 1 Processor -- start 2024-04-02T14:45:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f4246784740 1 -- start start 2024-04-02T14:45:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f4246784740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 0x564fddbea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f4246784740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbeaa00 0x564fddbf30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f4246784740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbf35e0 0x564fddbf59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f4246784740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564fddb4ed70 con 0x564fddbf35e0 2024-04-02T14:45:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f4246784740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564fddaf72a0 con 0x564fddbea0e0 2024-04-02T14:45:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f4246784740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564fddb4ad20 con 0x564fddbeaa00 2024-04-02T14:45:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f4241a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 0x564fddbea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f423be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbf35e0 0x564fddbf59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f4241a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 0x564fddbea4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48622/0 (socket says 172.21.15.62:48622) 2024-04-02T14:45:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f4241a00640 1 -- 172.21.15.62:0/1893677246 learned_addr learned my addr 172.21.15.62:0/1893677246 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.119+0000 7f4241000640 1 --2- 172.21.15.62:0/1893677246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbeaa00 0x564fddbf30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.116 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-class-rfkill 2024-04-02T14:45:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.120+0000 7f4241a00640 1 -- 172.21.15.62:0/1893677246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbeaa00 msgr2=0x564fddbf30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.120+0000 7f4241a00640 1 --2- 172.21.15.62:0/1893677246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbeaa00 0x564fddbf30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.120+0000 7f4241a00640 1 -- 172.21.15.62:0/1893677246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbf35e0 msgr2=0x564fddbf59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.120+0000 7f4241a00640 1 --2- 172.21.15.62:0/1893677246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbf35e0 0x564fddbf59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.120+0000 7f4241a00640 1 -- 172.21.15.62:0/1893677246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fddb0b840 con 0x564fddbea0e0 2024-04-02T14:45:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.120+0000 7f4241000640 1 --2- 172.21.15.62:0/1893677246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbeaa00 0x564fddbf30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.120+0000 7f423be00640 1 --2- 172.21.15.62:0/1893677246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbf35e0 0x564fddbf59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.120+0000 7f4241a00640 1 --2- 172.21.15.62:0/1893677246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 0x564fddbea4c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4234009f50 tx=0x7f42340118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f808a17c1d242d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.120+0000 7f4239600640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f42240a3d90 0x7f42240a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.120+0000 7f422e000640 1 -- 172.21.15.62:0/1893677246 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234004d20 con 0x564fddbea0e0 2024-04-02T14:45:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.121+0000 7f4239600640 1 -- 172.21.15.62:0/663867786 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f42300f2690 con 0x7f423c103af0 2024-04-02T14:45:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.121+0000 7f422e000640 1 -- 172.21.15.62:0/1893677246 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4234004050 con 0x564fddbea0e0 2024-04-02T14:45:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.121+0000 7f4241000640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f42240a3d90 0x7f42240a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.121+0000 7f422e000640 1 -- 172.21.15.62:0/1893677246 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234005260 con 0x564fddbea0e0 2024-04-02T14:45:45.119 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-koneplus 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.121+0000 7f4246784740 1 -- 172.21.15.62:0/1893677246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 msgr2=0x564fddbea4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.121+0000 7f4246784740 1 --2- 172.21.15.62:0/1893677246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 0x564fddbea4c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4234009f50 tx=0x7f42340118d0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.122+0000 7f4246784740 1 -- 172.21.15.62:0/1893677246 shutdown_connections 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.122+0000 7f4246784740 1 --2- 172.21.15.62:0/1893677246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbf35e0 0x564fddbf59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.122+0000 7f4246784740 1 --2- 172.21.15.62:0/1893677246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbeaa00 0x564fddbf30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.122+0000 7f4246784740 1 --2- 172.21.15.62:0/1893677246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 0x564fddbea4c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.122+0000 7f4246784740 1 -- 172.21.15.62:0/1893677246 >> 172.21.15.62:0/1893677246 conn(0x564fddbe4810 msgr2=0x564fddbe4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.122+0000 7f4246784740 1 -- 172.21.15.62:0/1893677246 shutdown_connections 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.122+0000 7f4246784740 1 -- 172.21.15.62:0/1893677246 wait complete. 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.122+0000 7f4241000640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f42240a3d90 0x7f42240a6250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f42300efff0 tx=0x7f423002bc30 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.122+0000 7f4246784740 1 -- start start 2024-04-02T14:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.122+0000 7f4246784740 1 Processor -- start 2024-04-02T14:45:45.121 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-kovaplus 2024-04-02T14:45:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f4246784740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbea0e0 0x564fddbf0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f4246784740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f4246784740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbf35e0 0x564fddbf10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f4246784740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564fddbf63d0 con 0x564fddbeaa00 2024-04-02T14:45:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f4246784740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564fddb0a660 con 0x564fddbf35e0 2024-04-02T14:45:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f4246784740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564fddbf6550 con 0x564fddbea0e0 2024-04-02T14:45:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f4246784740 10 client.0 _ll_register_callbacks cb 0x564fddb0a050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f423be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f4241a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbea0e0 0x564fddbf0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f423be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35356/0 (socket says 172.21.15.62:35356) 2024-04-02T14:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f423be00640 1 -- 172.21.15.62:0/40283980 learned_addr learned my addr 172.21.15.62:0/40283980 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.123+0000 7f4241a00640 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbf35e0 0x564fddbf10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.124+0000 7f423be00640 1 -- 172.21.15.62:0/40283980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbea0e0 msgr2=0x564fddbf0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.124+0000 7f423be00640 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbea0e0 0x564fddbf0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.124+0000 7f423be00640 1 -- 172.21.15.62:0/40283980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbf35e0 msgr2=0x564fddbf10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.124+0000 7f423be00640 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbf35e0 0x564fddbf10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.124+0000 7f423be00640 1 -- 172.21.15.62:0/40283980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fddb19e80 con 0x564fddbeaa00 2024-04-02T14:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.124+0000 7f4241a00640 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbf35e0 0x564fddbf10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:45.124 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra 2024-04-02T14:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.124+0000 7f423be00640 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf0b60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f422800bf30 tx=0x7f42280118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea996722d0223a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.124+0000 7f4222a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.124+0000 7f4222a00640 1 -- 172.21.15.62:0/40283980 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228004020 con 0x564fddbeaa00 2024-04-02T14:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.124+0000 7f4222a00640 1 -- 172.21.15.62:0/40283980 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f42280041c0 con 0x564fddbeaa00 2024-04-02T14:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.124+0000 7f4246784740 1 -- 172.21.15.62:0/40283980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4234004050 con 0x564fddbeaa00 2024-04-02T14:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.125+0000 7f4222a00640 1 -- 172.21.15.62:0/40283980 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42280043c0 con 0x564fddbeaa00 2024-04-02T14:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.125+0000 7f4246784740 10 client.15822 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.125+0000 7f4246784740 1 -- 172.21.15.62:0/40283980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564fdd863210 con 0x564fddbeaa00 2024-04-02T14:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.125+0000 7f4222a00640 1 -- 172.21.15.62:0/40283980 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4234004050 con 0x564fddbeaa00 2024-04-02T14:45:45.125 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/removed/ 2024-04-02T14:45:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.125+0000 7f4222a00640 1 -- 172.21.15.62:0/40283980 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f42280107c0 con 0x564fddbeaa00 2024-04-02T14:45:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.125+0000 7f4246784740 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.125+0000 7f4246784740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.125+0000 7f4246784740 20 client.15822 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.125+0000 7f4246784740 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x564fddc03a40 0x564fddc05e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.125+0000 7f4246784740 4 client.15822 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.125+0000 7f4246784740 1 -- 172.21.15.62:0/40283980 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fddafecd0 con 0x564fddc03a40 2024-04-02T14:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.126+0000 7f4241a00640 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x564fddc03a40 0x564fddc05e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.126+0000 7f4241a00640 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x564fddc03a40 0x564fddc05e50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c39bd49848d89512 server_cookie=e7940ff034f82b0d in_seq=0 out_seq=0 2024-04-02T14:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.126+0000 7f4222a00640 10 client.15822 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:45.128 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/removed/devfs 2024-04-02T14:45:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.127+0000 7f4222a00640 1 -- 172.21.15.62:0/40283980 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fddafecd0 con 0x564fddc03a40 2024-04-02T14:45:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.127+0000 7f4222a00640 10 client.15822 handle_command_reply: tid=0 2024-04-02T14:45:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.127+0000 7f4246784740 1 client.15822 shutdown 2024-04-02T14:45:45.128 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/removed/dv1394 2024-04-02T14:45:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4246784740 1 -- 172.21.15.62:0/40283980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 msgr2=0x564fddbf0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4246784740 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf0b60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f422800bf30 tx=0x7f42280118d0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4246784740 1 -- 172.21.15.62:0/40283980 shutdown_connections 2024-04-02T14:45:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4246784740 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x564fddc03a40 0x564fddc05e50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4246784740 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbf35e0 0x564fddbf10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4246784740 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf0b60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4246784740 1 --2- 172.21.15.62:0/40283980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbea0e0 0x564fddbf0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4222a00640 0 client.15822 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4246784740 1 -- 172.21.15.62:0/40283980 >> 172.21.15.62:0/40283980 conn(0x564fddbe4810 msgr2=0x564fddbe6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4246784740 1 -- 172.21.15.62:0/40283980 shutdown_connections 2024-04-02T14:45:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4246784740 1 -- 172.21.15.62:0/40283980 wait complete. 2024-04-02T14:45:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.128+0000 7f4246784740 20 client.15822 trim_cache size 0 max 16384 2024-04-02T14:45:45.131 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/removed/ip_queue 2024-04-02T14:45:45.131 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/removed/net_dma 2024-04-02T14:45:45.131 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/removed/o2cb 2024-04-02T14:45:45.131 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/removed/raw1394 2024-04-02T14:45:45.131 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/removed/video1394 2024-04-02T14:45:45.131 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/ 2024-04-02T14:45:45.131 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/firewire-cdev 2024-04-02T14:45:45.132 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/o2cb 2024-04-02T14:45:45.133 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/syscalls 2024-04-02T14:45:45.134 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-acpi-pmprofile 2024-04-02T14:45:45.135 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-firewire 2024-04-02T14:45:45.137 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-usb 2024-04-02T14:45:45.138 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-xen-backend 2024-04-02T14:45:45.139 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-backlight 2024-04-02T14:45:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.145+0000 7f4246784740 1 Processor -- start 2024-04-02T14:45:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.145+0000 7f4246784740 1 -- start start 2024-04-02T14:45:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.145+0000 7f4246784740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 0x564fddbff290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.146+0000 7f4246784740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbeade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.146+0000 7f4246784740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbf35e0 0x564fddbf0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.146+0000 7f4246784740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1af810 con 0x564fddbeaa00 2024-04-02T14:45:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.146+0000 7f4246784740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1b1350 con 0x564fddbea0e0 2024-04-02T14:45:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.146+0000 7f4246784740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1afee0 con 0x564fddbf35e0 2024-04-02T14:45:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.146+0000 7f4241a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 0x564fddbff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.146+0000 7f423be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbeade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.146+0000 7f423be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbeade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35360/0 (socket says 172.21.15.62:35360) 2024-04-02T14:45:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.146+0000 7f423be00640 1 -- 172.21.15.62:0/1904281932 learned_addr learned my addr 172.21.15.62:0/1904281932 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:45.142 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-rfkill 2024-04-02T14:45:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.146+0000 7f4241a00640 1 --2- 172.21.15.62:0/1904281932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbf35e0 0x564fddbf0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.147+0000 7f423be00640 1 -- 172.21.15.62:0/1904281932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbf35e0 msgr2=0x564fddbf0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.147+0000 7f423be00640 1 --2- 172.21.15.62:0/1904281932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbf35e0 0x564fddbf0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.147+0000 7f423be00640 1 -- 172.21.15.62:0/1904281932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 msgr2=0x564fddbff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.147+0000 7f423be00640 1 --2- 172.21.15.62:0/1904281932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 0x564fddbff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.147+0000 7f423be00640 1 -- 172.21.15.62:0/1904281932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fddbef5d0 con 0x564fddbeaa00 2024-04-02T14:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.147+0000 7f4241a00640 1 --2- 172.21.15.62:0/1904281932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbf35e0 0x564fddbf0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.147+0000 7f423be00640 1 --2- 172.21.15.62:0/1904281932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbeade0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4228004610 tx=0x7f4228005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edf9bc11957ce3a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.147+0000 7f422d600640 1 -- 172.21.15.62:0/1904281932 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228027040 con 0x564fddbeaa00 2024-04-02T14:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.147+0000 7f422d600640 1 -- 172.21.15.62:0/1904281932 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f422800b480 con 0x564fddbeaa00 2024-04-02T14:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.147+0000 7f422d600640 1 -- 172.21.15.62:0/1904281932 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228004b90 con 0x564fddbeaa00 2024-04-02T14:45:45.144 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-tpm 2024-04-02T14:45:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.148+0000 7f4246784740 1 -- 172.21.15.62:0/1904281932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 msgr2=0x564fddbeade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.148+0000 7f4246784740 1 --2- 172.21.15.62:0/1904281932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbeade0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4228004610 tx=0x7f4228005900 comp rx=0 tx=0).stop 2024-04-02T14:45:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.148+0000 7f4246784740 1 -- 172.21.15.62:0/1904281932 shutdown_connections 2024-04-02T14:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.148+0000 7f4246784740 1 --2- 172.21.15.62:0/1904281932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbf35e0 0x564fddbf0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.148+0000 7f4246784740 1 --2- 172.21.15.62:0/1904281932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbeade0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.148+0000 7f4246784740 1 --2- 172.21.15.62:0/1904281932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbea0e0 0x564fddbff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.148+0000 7f4246784740 1 -- 172.21.15.62:0/1904281932 >> 172.21.15.62:0/1904281932 conn(0x564fddbe4810 msgr2=0x564fddbe63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.148+0000 7f4246784740 1 -- 172.21.15.62:0/1904281932 shutdown_connections 2024-04-02T14:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.148+0000 7f4246784740 1 -- 172.21.15.62:0/1904281932 wait complete. 2024-04-02T14:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.149+0000 7f4246784740 1 -- start start 2024-04-02T14:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.149+0000 7f4246784740 1 Processor -- start 2024-04-02T14:45:45.146 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-ubi 2024-04-02T14:45:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.149+0000 7f4246784740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbea0e0 0x564fddbf73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.149+0000 7f4246784740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.149+0000 7f4246784740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbf35e0 0x564fddbf7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.149+0000 7f4246784740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1b0c80 con 0x564fddbeaa00 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.149+0000 7f4246784740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1b05b0 con 0x564fddbf35e0 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.149+0000 7f4246784740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1ad7f0 con 0x564fddbea0e0 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.149+0000 7f4246784740 10 client.0 _ll_register_callbacks cb 0x7f422800b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.149+0000 7f423be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.150+0000 7f423be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35368/0 (socket says 172.21.15.62:35368) 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.150+0000 7f423be00640 1 -- 172.21.15.62:0/1099993507 learned_addr learned my addr 172.21.15.62:0/1099993507 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.150+0000 7f4241a00640 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbf35e0 0x564fddbf7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.150+0000 7f4241a00640 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbea0e0 0x564fddbf73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.150+0000 7f423be00640 1 -- 172.21.15.62:0/1099993507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbea0e0 msgr2=0x564fddbf73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.150+0000 7f423be00640 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbea0e0 0x564fddbf73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.150+0000 7f423be00640 1 -- 172.21.15.62:0/1099993507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbf35e0 msgr2=0x564fddbf7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.150+0000 7f423be00640 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbf35e0 0x564fddbf7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.150+0000 7f423be00640 1 -- 172.21.15.62:0/1099993507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fddbf5d40 con 0x564fddbeaa00 2024-04-02T14:45:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.150+0000 7f423be00640 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf78f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4228011910 tx=0x7f4228010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5019f1066e0d740 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.150+0000 7f4222a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:45:45.149 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-udc 2024-04-02T14:45:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.151+0000 7f4222a00640 1 -- 172.21.15.62:0/1099993507 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228027030 con 0x564fddbeaa00 2024-04-02T14:45:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.151+0000 7f4222a00640 1 -- 172.21.15.62:0/1099993507 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f4228031070 con 0x564fddbeaa00 2024-04-02T14:45:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.151+0000 7f4246784740 1 -- 172.21.15.62:0/1099993507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f422800b480 con 0x564fddbeaa00 2024-04-02T14:45:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.151+0000 7f4222a00640 1 -- 172.21.15.62:0/1099993507 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f422802d8a0 con 0x564fddbeaa00 2024-04-02T14:45:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.151+0000 7f4246784740 10 client.15828 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.151+0000 7f4246784740 1 -- 172.21.15.62:0/1099993507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564fdd92f620 con 0x564fddbeaa00 2024-04-02T14:45:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.151+0000 7f4222a00640 1 -- 172.21.15.62:0/1099993507 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f422800b480 con 0x564fddbeaa00 2024-04-02T14:45:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.151+0000 7f4222a00640 1 -- 172.21.15.62:0/1099993507 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f422802a070 con 0x564fddbeaa00 2024-04-02T14:45:45.152 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-node 2024-04-02T14:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.152+0000 7f4246784740 10 client.15828 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.152+0000 7f4246784740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.152+0000 7f4246784740 20 client.15828 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.152+0000 7f4246784740 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x564fddc6a690 0x564fddc6ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.152+0000 7f4246784740 4 client.15828 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.152+0000 7f4246784740 1 -- 172.21.15.62:0/1099993507 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fddb5c2b0 con 0x564fddc6a690 2024-04-02T14:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.152+0000 7f4241a00640 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x564fddc6a690 0x564fddc6ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.152+0000 7f4241a00640 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x564fddc6a690 0x564fddc6ca80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b8ccffd052a63e server_cookie=aeb45373c6d73bed in_seq=0 out_seq=0 2024-04-02T14:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.152+0000 7f4222a00640 10 client.15828 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:45.154 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-cpu 2024-04-02T14:45:45.154 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-xen_memory 2024-04-02T14:45:45.154 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-ib_srp 2024-04-02T14:45:45.154 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-qla2xxx 2024-04-02T14:45:45.154 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-usb-usbtmc 2024-04-02T14:45:45.155 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-w1_ds28e04 2024-04-02T14:45:45.156 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-efi-vars 2024-04-02T14:45:45.158 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-dump 2024-04-02T14:45:45.160 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-elog 2024-04-02T14:45:45.161 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-module 2024-04-02T14:45:45.164 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-transport-srp 2024-04-02T14:45:45.165 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/thermal-notification 2024-04-02T14:45:45.167 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/stable/vdso 2024-04-02T14:45:45.170 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/ 2024-04-02T14:45:45.171 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-spear-pcie-gadget 2024-04-02T14:45:45.173 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget 2024-04-02T14:45:45.175 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-acm 2024-04-02T14:45:45.177 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ecm 2024-04-02T14:45:45.179 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-eem 2024-04-02T14:45:45.181 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ffs 2024-04-02T14:45:45.182 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-hid 2024-04-02T14:45:45.184 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-loopback 2024-04-02T14:45:45.185 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-mass-storage 2024-04-02T14:45:45.187 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-midi 2024-04-02T14:45:45.188 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ncm 2024-04-02T14:45:45.190 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-obex 2024-04-02T14:45:45.191 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-phonet 2024-04-02T14:45:45.193 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-rndis 2024-04-02T14:45:45.195 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-serial 2024-04-02T14:45:45.196 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-sourcesink 2024-04-02T14:45:45.198 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-subset 2024-04-02T14:45:45.199 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac1 2024-04-02T14:45:45.201 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac2 2024-04-02T14:45:45.202 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uvc 2024-04-02T14:45:45.204 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-driver-genwqe 2024-04-02T14:45:45.206 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ec 2024-04-02T14:45:45.207 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ideapad 2024-04-02T14:45:45.209 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-olpc 2024-04-02T14:45:45.210 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pfo-nx-crypto 2024-04-02T14:45:45.212 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pktcdvd 2024-04-02T14:45:45.214 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/dev-kmsg 2024-04-02T14:45:45.215 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/evm 2024-04-02T14:45:45.217 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/ima_policy 2024-04-02T14:45:45.219 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/procfs-diskstats 2024-04-02T14:45:45.220 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/pstore 2024-04-02T14:45:45.222 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ata 2024-04-02T14:45:45.223 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block 2024-04-02T14:45:45.225 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-bcache 2024-04-02T14:45:45.227 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-dm 2024-04-02T14:45:45.228 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-rssd 2024-04-02T14:45:45.230 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-zram 2024-04-02T14:45:45.232 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-acpi 2024-04-02T14:45:45.233 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-amba 2024-04-02T14:45:45.235 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-bcma 2024-04-02T14:45:45.237 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10 2024-04-02T14:45:45.238 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm3x 2024-04-02T14:45:45.240 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-funnel 2024-04-02T14:45:45.241 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc 2024-04-02T14:45:45.243 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-css 2024-04-02T14:45:45.245 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-events 2024-04-02T14:45:45.247 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-format 2024-04-02T14:45:45.248 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 2024-04-02T14:45:45.250 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_gpci 2024-04-02T14:45:45.251 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-fcoe 2024-04-02T14:45:45.253 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-hsi 2024-04-02T14:45:45.255 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-fsa9480 2024-04-02T14:45:45.256 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 2024-04-02T14:45:45.258 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-lm3533 2024-04-02T14:45:45.259 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio 2024-04-02T14:45:45.261 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-accel-bmc150 2024-04-02T14:45:45.263 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-ad9523 2024-04-02T14:45:45.265 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4350 2024-04-02T14:45:45.266 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-gyro-bmg160 2024-04-02T14:45:45.268 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-light-lm3533-als 2024-04-02T14:45:45.270 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-mpu6050 2024-04-02T14:45:45.271 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 2024-04-02T14:45:45.273 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-trigger-sysfs 2024-04-02T14:45:45.275 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mdio 2024-04-02T14:45:45.276 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-media 2024-04-02T14:45:45.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mei 2024-04-02T14:45:45.279 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci 2024-04-02T14:45:45.281 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss 2024-04-02T14:45:45.283 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-drivers-ehci_hcd 2024-04-02T14:45:45.284 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-platform 2024-04-02T14:45:45.286 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rbd 2024-04-02T14:45:45.288 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rpmsg 2024-04-02T14:45:45.289 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-umc 2024-04-02T14:45:45.291 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb 2024-04-02T14:45:45.293 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg 2024-04-02T14:45:45.295 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-lvstest 2024-04-02T14:45:45.296 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-c2port 2024-04-02T14:45:45.298 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-cfq-target-latency 2024-04-02T14:45:45.300 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class 2024-04-02T14:45:45.301 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-adp8870 2024-04-02T14:45:45.303 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 2024-04-02T14:45:45.304 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-bdi 2024-04-02T14:45:45.306 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-cxl 2024-04-02T14:45:45.308 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-devfreq 2024-04-02T14:45:45.309 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-extcon 2024-04-02T14:45:45.311 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu 2024-04-02T14:45:45.312 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-amd-iommu 2024-04-02T14:45:45.314 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-intel-iommu 2024-04-02T14:45:45.316 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-lcd 2024-04-02T14:45:45.317 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led 2024-04-02T14:45:45.319 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 2024-04-02T14:45:45.321 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-leds-gt683r 2024-04-02T14:45:45.322 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mei 2024-04-02T14:45:45.324 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mic.txt 2024-04-02T14:45:45.326 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mtd 2024-04-02T14:45:45.327 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net 2024-04-02T14:45:45.329 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-batman-adv 2024-04-02T14:45:45.330 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-cdc_ncm 2024-04-02T14:45:45.332 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-grcan 2024-04-02T14:45:45.334 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-mesh 2024-04-02T14:45:45.335 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-queues 2024-04-02T14:45:45.337 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-statistics 2024-04-02T14:45:45.339 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pktcdvd 2024-04-02T14:45:45.340 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-power 2024-04-02T14:45:45.342 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-powercap 2024-04-02T14:45:45.344 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pwm 2024-04-02T14:45:45.345 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rc 2024-04-02T14:45:45.347 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-regulator 2024-04-02T14:45:45.349 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rtc-rtc0-device-rtc_calibration 2024-04-02T14:45:45.351 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-scsi_host 2024-04-02T14:45:45.352 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc 2024-04-02T14:45:45.354 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc-wusbhc 2024-04-02T14:45:45.355 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-dev 2024-04-02T14:45:45.357 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices 2024-04-02T14:45:45.359 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-edac 2024-04-02T14:45:45.361 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-firmware_node 2024-04-02T14:45:45.363 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-lpss_ltr 2024-04-02T14:45:45.364 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-memory 2024-04-02T14:45:45.366 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-mmc 2024-04-02T14:45:45.368 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-online 2024-04-02T14:45:45.370 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-_UDC_-gadget 2024-04-02T14:45:45.372 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-docg3 2024-04-02T14:45:45.377 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-sh_mobile_lcdc_fb 2024-04-02T14:45:45.377 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power 2024-04-02T14:45:45.378 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D0 2024-04-02T14:45:45.380 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D1 2024-04-02T14:45:45.382 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D2 2024-04-02T14:45:45.384 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D3hot 2024-04-02T14:45:45.386 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_wakeup 2024-04-02T14:45:45.389 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_state 2024-04-02T14:45:45.391 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-real_power_state 2024-04-02T14:45:45.394 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-resource_in_use 2024-04-02T14:45:45.395 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-soc 2024-04-02T14:45:45.398 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-sun 2024-04-02T14:45:45.400 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-cpu 2024-04-02T14:45:45.402 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-ibm-rtl 2024-04-02T14:45:45.404 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-xen_cpu 2024-04-02T14:45:45.406 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-genwqe 2024-04-02T14:45:45.408 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid 2024-04-02T14:45:45.410 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-lenovo 2024-04-02T14:45:45.412 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-logitech-lg4ff 2024-04-02T14:45:45.414 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-multitouch 2024-04-02T14:45:45.416 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-picolcd 2024-04-02T14:45:45.419 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-prodikeys 2024-04-02T14:45:45.421 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-arvo 2024-04-02T14:45:45.423 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-isku 2024-04-02T14:45:45.425 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone 2024-04-02T14:45:45.427 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-koneplus 2024-04-02T14:45:45.429 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-konepure 2024-04-02T14:45:45.430 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kovaplus 2024-04-02T14:45:45.433 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-lua 2024-04-02T14:45:45.434 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-pyra 2024-04-02T14:45:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.495+0000 7f4222a00640 1 -- 172.21.15.62:0/1099993507 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fddb5c2b0 con 0x564fddc6a690 2024-04-02T14:45:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.495+0000 7f4222a00640 10 client.15828 handle_command_reply: tid=0 2024-04-02T14:45:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.495+0000 7f4246784740 1 client.15828 shutdown 2024-04-02T14:45:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.495+0000 7f4246784740 1 -- 172.21.15.62:0/1099993507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 msgr2=0x564fddbf78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.495+0000 7f4246784740 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf78f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4228011910 tx=0x7f4228010f80 comp rx=0 tx=0).stop 2024-04-02T14:45:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.496+0000 7f4246784740 1 -- 172.21.15.62:0/1099993507 shutdown_connections 2024-04-02T14:45:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.496+0000 7f4246784740 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x564fddc6a690 0x564fddc6ca80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.496+0000 7f4246784740 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564fddbf35e0 0x564fddbf7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.496+0000 7f4246784740 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fddbeaa00 0x564fddbf78f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.496+0000 7f4246784740 1 --2- 172.21.15.62:0/1099993507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564fddbea0e0 0x564fddbf73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.496+0000 7f4246784740 1 -- 172.21.15.62:0/1099993507 >> 172.21.15.62:0/1099993507 conn(0x564fddbe4810 msgr2=0x564fddbe5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.496+0000 7f4222a00640 0 client.15828 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.496+0000 7f4246784740 1 -- 172.21.15.62:0/1099993507 shutdown_connections 2024-04-02T14:45:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.497+0000 7f4246784740 1 -- 172.21.15.62:0/1099993507 wait complete. 2024-04-02T14:45:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.497+0000 7f4246784740 20 client.15828 trim_cache size 0 max 16384 2024-04-02T14:45:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:45:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:45:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:45:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:45:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 -- 172.21.15.62:0/663867786 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f42240a3d90 msgr2=0x7f42240a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f42240a3d90 0x7f42240a6250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f42300efff0 tx=0x7f423002bc30 comp rx=0 tx=0).stop 2024-04-02T14:45:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 -- 172.21.15.62:0/663867786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c103af0 msgr2=0x7f423c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c103af0 0x7f423c19a4b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4230005ab0 tx=0x7f423002dcf0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 -- 172.21.15.62:0/663867786 shutdown_connections 2024-04-02T14:45:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f42240a3d90 0x7f42240a6250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f423c109b90 0x7f423c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423c1044b0 0x7f423c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 --2- 172.21.15.62:0/663867786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c103af0 0x7f423c19a4b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 -- 172.21.15.62:0/663867786 >> 172.21.15.62:0/663867786 conn(0x7f423c0ff5e0 msgr2=0x7f423c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 -- 172.21.15.62:0/663867786 shutdown_connections 2024-04-02T14:45:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:45.499+0000 7f4242400640 1 -- 172.21.15.62:0/663867786 wait complete. 2024-04-02T14:45:45.495 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-ryos 2024-04-02T14:45:45.496 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-savu 2024-04-02T14:45:45.497 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-srws1 2024-04-02T14:45:45.499 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-wiimote 2024-04-02T14:45:45.500 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-input-axp-pek 2024-04-02T14:45:45.501 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-intel-rapid-start 2024-04-02T14:45:45.502 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-pciback 2024-04-02T14:45:45.503 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-ppi 2024-04-02T14:45:45.504 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-samsung-laptop 2024-04-02T14:45:45.506 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-sunxi-sid 2024-04-02T14:45:45.507 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-tegra-fuse 2024-04-02T14:45:45.508 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-toshiba_acpi 2024-04-02T14:45:45.510 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-wacom 2024-04-02T14:45:45.511 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkback 2024-04-02T14:45:45.513 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkfront 2024-04-02T14:45:45.514 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-acpi 2024-04-02T14:45:45.516 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-dmi 2024-04-02T14:45:45.518 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi 2024-04-02T14:45:45.520 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi-runtime-map 2024-04-02T14:45:45.521 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-gsmi 2024-04-02T14:45:45.523 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-log 2024-04-02T14:45:45.524 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-memmap 2024-04-02T14:45:45.526 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-ofw 2024-04-02T14:45:45.527 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sfi 2024-04-02T14:45:45.529 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sgi_uv 2024-04-02T14:45:45.530 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-ext4 2024-04-02T14:45:45.532 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-f2fs 2024-04-02T14:45:45.534 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-nilfs2 2024-04-02T14:45:45.536 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-xfs 2024-04-02T14:45:45.537 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-gpio 2024-04-02T14:45:45.538 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-i2c-bmp085 2024-04-02T14:45:45.540 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ibft 2024-04-02T14:45:45.542 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-boot_params 2024-04-02T14:45:45.543 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-fscaps 2024-04-02T14:45:45.545 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-iommu_groups 2024-04-02T14:45:45.546 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-livepatch 2024-04-02T14:45:45.548 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm 2024-04-02T14:45:45.550 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-hugepages 2024-04-02T14:45:45.552 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-ksm 2024-04-02T14:45:45.554 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-slab 2024-04-02T14:45:45.556 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-uids 2024-04-02T14:45:45.559 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo 2024-04-02T14:45:45.561 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-memory-page-offline 2024-04-02T14:45:45.563 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-module 2024-04-02T14:45:45.565 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ocfs2 2024-04-02T14:45:45.567 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-laptop 2024-04-02T14:45:45.569 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-wmi 2024-04-02T14:45:45.571 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-at91 2024-04-02T14:45:45.574 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-brcmstb-gisb-arb 2024-04-02T14:45:45.576 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-chipidea-usb-otg 2024-04-02T14:45:45.578 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-eeepc-laptop 2024-04-02T14:45:45.580 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ideapad-laptop 2024-04-02T14:45:45.582 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-kim 2024-04-02T14:45:45.584 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-msi-laptop 2024-04-02T14:45:45.586 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-tahvo-usb 2024-04-02T14:45:45.589 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ts5500 2024-04-02T14:45:45.590 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-power 2024-04-02T14:45:45.592 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-pps 2024-04-02T14:45:45.594 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-profiling 2024-04-02T14:45:45.596 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ptp 2024-04-02T14:45:45.598 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-tty 2024-04-02T14:45:45.600 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-wusb_cbaf 2024-04-02T14:45:45.604 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/BUG-HUNTING 2024-04-02T14:45:45.604 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/Changes 2024-04-02T14:45:45.606 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/CodeOfConflict 2024-04-02T14:45:45.608 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/CodingStyle 2024-04-02T14:45:45.609 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DMA-API-HOWTO.txt 2024-04-02T14:45:45.611 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DMA-API.txt 2024-04-02T14:45:45.613 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DMA-ISA-LPC.txt 2024-04-02T14:45:45.615 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DMA-attributes.txt 2024-04-02T14:45:45.617 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/ 2024-04-02T14:45:45.618 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/.gitignore 2024-04-02T14:45:45.620 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/80211.tmpl 2024-04-02T14:45:45.621 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/Makefile 2024-04-02T14:45:45.623 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/alsa-driver-api.tmpl 2024-04-02T14:45:45.625 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/crypto-API.tmpl 2024-04-02T14:45:45.626 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/debugobjects.tmpl 2024-04-02T14:45:45.628 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/device-drivers.tmpl 2024-04-02T14:45:45.630 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/deviceiobook.tmpl 2024-04-02T14:45:45.631 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/drm.tmpl 2024-04-02T14:45:45.637 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/filesystems.tmpl 2024-04-02T14:45:45.639 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/gadget.tmpl 2024-04-02T14:45:45.640 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/genericirq.tmpl 2024-04-02T14:45:45.642 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/kernel-api.tmpl 2024-04-02T14:45:45.644 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/kernel-hacking.tmpl 2024-04-02T14:45:45.645 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/kernel-locking.tmpl 2024-04-02T14:45:45.648 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/kgdb.tmpl 2024-04-02T14:45:45.650 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/libata.tmpl 2024-04-02T14:45:45.653 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/librs.tmpl 2024-04-02T14:45:45.655 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/lsm.tmpl 2024-04-02T14:45:45.656 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/ 2024-04-02T14:45:45.657 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/Makefile 2024-04-02T14:45:45.659 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/bayer.png.b64 2024-04-02T14:45:45.661 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/constraints.png.b64 2024-04-02T14:45:45.662 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/crop.gif.b64 2024-04-02T14:45:45.664 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ 2024-04-02T14:45:45.665 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/.gitignore 2024-04-02T14:45:45.667 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/audio.xml 2024-04-02T14:45:45.669 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ca.xml 2024-04-02T14:45:45.671 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/demux.xml 2024-04-02T14:45:45.672 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbapi.xml 2024-04-02T14:45:45.674 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbproperty.xml 2024-04-02T14:45:45.676 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbstb.pdf 2024-04-02T14:45:45.678 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/examples.xml 2024-04-02T14:45:45.679 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/frontend.xml 2024-04-02T14:45:45.681 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/intro.xml 2024-04-02T14:45:45.683 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/kdapi.xml 2024-04-02T14:45:45.685 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/net.xml 2024-04-02T14:45:45.686 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/video.xml 2024-04-02T14:45:45.688 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/dvbstb.png.b64 2024-04-02T14:45:45.691 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_bt.gif.b64 2024-04-02T14:45:45.697 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_tb.gif.b64 2024-04-02T14:45:45.702 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt.gif.b64 2024-04-02T14:45:45.703 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt_example.gif.b64 2024-04-02T14:45:45.705 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/pipeline.png.b64 2024-04-02T14:45:45.708 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/selection.png.b64 2024-04-02T14:45:45.710 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/ 2024-04-02T14:45:45.711 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/.gitignore 2024-04-02T14:45:45.713 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/biblio.xml 2024-04-02T14:45:45.714 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/capture.c.xml 2024-04-02T14:45:45.716 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/common.xml 2024-04-02T14:45:45.718 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/compat.xml 2024-04-02T14:45:45.722 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/controls.xml 2024-04-02T14:45:45.729 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/crop.pdf 2024-04-02T14:45:45.731 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-capture.xml 2024-04-02T14:45:45.732 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-codec.xml 2024-04-02T14:45:45.734 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-effect.xml 2024-04-02T14:45:45.735 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-event.xml 2024-04-02T14:45:45.737 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-osd.xml 2024-04-02T14:45:45.739 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-output.xml 2024-04-02T14:45:45.740 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-overlay.xml 2024-04-02T14:45:45.742 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-radio.xml 2024-04-02T14:45:45.743 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-raw-vbi.xml 2024-04-02T14:45:45.745 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-rds.xml 2024-04-02T14:45:45.747 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sdr.xml 2024-04-02T14:45:45.748 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sliced-vbi.xml 2024-04-02T14:45:45.750 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-subdev.xml 2024-04-02T14:45:45.751 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-teletext.xml 2024-04-02T14:45:45.753 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/driver.xml 2024-04-02T14:45:45.755 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fdl-appendix.xml 2024-04-02T14:45:45.756 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_bt.pdf 2024-04-02T14:45:45.758 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_tb.pdf 2024-04-02T14:45:45.759 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-close.xml 2024-04-02T14:45:45.761 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-ioctl.xml 2024-04-02T14:45:45.762 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-mmap.xml 2024-04-02T14:45:45.764 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-munmap.xml 2024-04-02T14:45:45.765 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-open.xml 2024-04-02T14:45:45.767 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-poll.xml 2024-04-02T14:45:45.768 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-read.xml 2024-04-02T14:45:45.770 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-select.xml 2024-04-02T14:45:45.772 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-write.xml 2024-04-02T14:45:45.773 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/gen-errors.xml 2024-04-02T14:45:45.775 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/io.xml 2024-04-02T14:45:45.777 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/keytable.c.xml 2024-04-02T14:45:45.779 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/libv4l.xml 2024-04-02T14:45:45.780 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/lirc_device_interface.xml 2024-04-02T14:45:45.782 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-controller.xml 2024-04-02T14:45:45.783 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-close.xml 2024-04-02T14:45:45.785 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-ioctl.xml 2024-04-02T14:45:45.786 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-open.xml 2024-04-02T14:45:45.788 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-device-info.xml 2024-04-02T14:45:45.789 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-entities.xml 2024-04-02T14:45:45.791 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-links.xml 2024-04-02T14:45:45.793 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-setup-link.xml 2024-04-02T14:45:45.794 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pipeline.pdf 2024-04-02T14:45:45.796 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-grey.xml 2024-04-02T14:45:45.798 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-m420.xml 2024-04-02T14:45:45.799 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12.xml 2024-04-02T14:45:45.801 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12m.xml 2024-04-02T14:45:45.803 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12mt.xml 2024-04-02T14:45:45.804 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16.xml 2024-04-02T14:45:45.806 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml 2024-04-02T14:45:45.808 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv24.xml 2024-04-02T14:45:45.809 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-rgb.xml 2024-04-02T14:45:45.811 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-yuv.xml 2024-04-02T14:45:45.813 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr16.xml 2024-04-02T14:45:45.814 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr8.xml 2024-04-02T14:45:45.815 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs08.xml 2024-04-02T14:45:45.817 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs14le.xml 2024-04-02T14:45:45.818 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu08.xml 2024-04-02T14:45:45.820 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu16le.xml 2024-04-02T14:45:45.821 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-ru12le.xml 2024-04-02T14:45:45.823 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgbrg8.xml 2024-04-02T14:45:45.825 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgrbg8.xml 2024-04-02T14:45:45.826 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10.xml 2024-04-02T14:45:45.828 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10alaw8.xml 2024-04-02T14:45:45.829 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10dpcm8.xml 2024-04-02T14:45:45.831 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10p.xml 2024-04-02T14:45:45.833 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb12.xml 2024-04-02T14:45:45.834 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb8.xml 2024-04-02T14:45:45.836 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uv8.xml 2024-04-02T14:45:45.837 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uyvy.xml 2024-04-02T14:45:45.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:45 smithi062 ceph-mon[32116]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T14:45:45.839 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-vyuy.xml 2024-04-02T14:45:45.841 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10.xml 2024-04-02T14:45:45.842 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10b.xml 2024-04-02T14:45:45.844 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y12.xml 2024-04-02T14:45:45.846 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y16.xml 2024-04-02T14:45:45.847 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y41p.xml 2024-04-02T14:45:45.849 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv410.xml 2024-04-02T14:45:45.851 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv411p.xml 2024-04-02T14:45:45.852 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420.xml 2024-04-02T14:45:45.854 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420m.xml 2024-04-02T14:45:45.856 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv422p.xml 2024-04-02T14:45:45.857 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuyv.xml 2024-04-02T14:45:45.859 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvu420m.xml 2024-04-02T14:45:45.861 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvyu.xml 2024-04-02T14:45:45.863 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt.xml 2024-04-02T14:45:45.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:45 smithi113 ceph-mon[36135]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T14:45:45.866 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/planar-apis.xml 2024-04-02T14:45:45.867 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/remote_controllers.xml 2024-04-02T14:45:45.869 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selection-api.xml 2024-04-02T14:45:45.870 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selections-common.xml 2024-04-02T14:45:45.872 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-formats.xml 2024-04-02T14:45:45.874 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.dia 2024-04-02T14:45:45.876 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.svg 2024-04-02T14:45:45.878 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.dia 2024-04-02T14:45:45.880 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.svg 2024-04-02T14:45:45.881 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.dia 2024-04-02T14:45:45.883 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.svg 2024-04-02T14:45:45.885 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2.xml 2024-04-02T14:45:45.887 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2grab.c.xml 2024-04-02T14:45:45.888 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_525.pdf 2024-04-02T14:45:45.890 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_625.pdf 2024-04-02T14:45:45.892 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_hsync.pdf 2024-04-02T14:45:45.894 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml 2024-04-02T14:45:45.895 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-cropcap.xml 2024-04-02T14:45:45.897 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-chip-info.xml 2024-04-02T14:45:45.899 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-register.xml 2024-04-02T14:45:45.900 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-decoder-cmd.xml 2024-04-02T14:45:45.902 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dqevent.xml 2024-04-02T14:45:45.904 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml 2024-04-02T14:45:45.906 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-encoder-cmd.xml 2024-04-02T14:45:45.907 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml 2024-04-02T14:45:45.909 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-fmt.xml 2024-04-02T14:45:45.911 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-frameintervals.xml 2024-04-02T14:45:45.912 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-framesizes.xml 2024-04-02T14:45:45.914 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml 2024-04-02T14:45:45.916 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudio.xml 2024-04-02T14:45:45.918 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudioout.xml 2024-04-02T14:45:45.920 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enuminput.xml 2024-04-02T14:45:45.921 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumoutput.xml 2024-04-02T14:45:45.923 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumstd.xml 2024-04-02T14:45:45.925 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-expbuf.xml 2024-04-02T14:45:45.927 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audio.xml 2024-04-02T14:45:45.930 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audioout.xml 2024-04-02T14:45:45.931 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-crop.xml 2024-04-02T14:45:45.933 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ctrl.xml 2024-04-02T14:45:45.935 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-dv-timings.xml 2024-04-02T14:45:45.937 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-edid.xml 2024-04-02T14:45:45.939 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-enc-index.xml 2024-04-02T14:45:45.941 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ext-ctrls.xml 2024-04-02T14:45:45.943 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fbuf.xml 2024-04-02T14:45:45.945 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fmt.xml 2024-04-02T14:45:45.947 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-frequency.xml 2024-04-02T14:45:45.948 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-input.xml 2024-04-02T14:45:45.950 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-jpegcomp.xml 2024-04-02T14:45:45.951 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-modulator.xml 2024-04-02T14:45:45.953 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-output.xml 2024-04-02T14:45:45.957 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-parm.xml 2024-04-02T14:45:45.958 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-priority.xml 2024-04-02T14:45:45.960 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-selection.xml 2024-04-02T14:45:45.962 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-sliced-vbi-cap.xml 2024-04-02T14:45:45.964 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-std.xml 2024-04-02T14:45:45.966 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-tuner.xml 2024-04-02T14:45:45.969 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-log-status.xml 2024-04-02T14:45:45.973 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-overlay.xml 2024-04-02T14:45:45.975 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml 2024-04-02T14:45:45.977 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-qbuf.xml 2024-04-02T14:45:45.978 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml 2024-04-02T14:45:45.981 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querybuf.xml 2024-04-02T14:45:46.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:45 smithi204 ceph-mon[35948]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T14:45:47.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:47 smithi062 ceph-mon[32116]: pgmap v56: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 27 op/s 2024-04-02T14:45:47.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:47 smithi113 ceph-mon[36135]: pgmap v56: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 27 op/s 2024-04-02T14:45:48.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:47 smithi204 ceph-mon[35948]: pgmap v56: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 27 op/s 2024-04-02T14:45:48.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:45:48.347 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:45:48.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.593+0000 7f9d22e00640 1 -- 172.21.15.62:0/1832629388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d1c108fe0 msgr2=0x7f9d1c110bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.593+0000 7f9d22e00640 1 --2- 172.21.15.62:0/1832629388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d1c108fe0 0x7f9d1c110bb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009ca0 tx=0x7f9d1402c100 comp rx=0 tx=0).stop 2024-04-02T14:45:48.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.594+0000 7f9d22e00640 1 -- 172.21.15.62:0/1832629388 shutdown_connections 2024-04-02T14:45:48.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.594+0000 7f9d22e00640 1 --2- 172.21.15.62:0/1832629388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d1c108fe0 0x7f9d1c110bb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.594+0000 7f9d22e00640 1 --2- 172.21.15.62:0/1832629388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d1c1048c0 0x7f9d1c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.594+0000 7f9d22e00640 1 --2- 172.21.15.62:0/1832629388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c103f10 0x7f9d1c1042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.594+0000 7f9d22e00640 1 -- 172.21.15.62:0/1832629388 >> 172.21.15.62:0/1832629388 conn(0x7f9d1c0ff750 msgr2=0x7f9d1c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:48.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.594+0000 7f9d22e00640 1 -- 172.21.15.62:0/1832629388 shutdown_connections 2024-04-02T14:45:48.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.594+0000 7f9d22e00640 1 -- 172.21.15.62:0/1832629388 wait complete. 2024-04-02T14:45:48.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.595+0000 7f9d22e00640 1 Processor -- start 2024-04-02T14:45:48.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.595+0000 7f9d22e00640 1 -- start start 2024-04-02T14:45:48.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.595+0000 7f9d22e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d1c103f10 0x7f9d1c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.595+0000 7f9d22e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c108fe0 0x7f9d1c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.595+0000 7f9d22e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d1c19eef0 0x7f9d1c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.595+0000 7f9d22e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1141d0 con 0x7f9d1c108fe0 2024-04-02T14:45:48.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.595+0000 7f9d22e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c114050 con 0x7f9d1c103f10 2024-04-02T14:45:48.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.595+0000 7f9d22e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c114350 con 0x7f9d1c19eef0 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.596+0000 7f9d13e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c108fe0 0x7f9d1c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.596+0000 7f9d20e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d1c19eef0 0x7f9d1c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.596+0000 7f9d20e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d1c19eef0 0x7f9d1c19f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:42012/0 (socket says 172.21.15.62:42012) 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.596+0000 7f9d1be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d1c103f10 0x7f9d1c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.596+0000 7f9d20e00640 1 -- 172.21.15.62:0/4068599553 learned_addr learned my addr 172.21.15.62:0/4068599553 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.596+0000 7f9d20e00640 1 -- 172.21.15.62:0/4068599553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d1c103f10 msgr2=0x7f9d1c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.596+0000 7f9d20e00640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d1c103f10 0x7f9d1c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.596+0000 7f9d20e00640 1 -- 172.21.15.62:0/4068599553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c108fe0 msgr2=0x7f9d1c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.596+0000 7f9d20e00640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c108fe0 0x7f9d1c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.596+0000 7f9d20e00640 1 -- 172.21.15.62:0/4068599553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c19f9e0 con 0x7f9d1c19eef0 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.596+0000 7f9d20e00640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d1c19eef0 0x7f9d1c19f3a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9d14037040 tx=0x7f9d14032f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.597+0000 7f9d19600640 1 -- 172.21.15.62:0/4068599553 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14039040 con 0x7f9d1c19eef0 2024-04-02T14:45:48.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.597+0000 7f9d22e00640 1 -- 172.21.15.62:0/4068599553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c19fc70 con 0x7f9d1c19eef0 2024-04-02T14:45:48.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.597+0000 7f9d19600640 1 -- 172.21.15.62:0/4068599553 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9d14044070 con 0x7f9d1c19eef0 2024-04-02T14:45:48.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.597+0000 7f9d19600640 1 -- 172.21.15.62:0/4068599553 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14004280 con 0x7f9d1c19eef0 2024-04-02T14:45:48.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.597+0000 7f9d22e00640 1 -- 172.21.15.62:0/4068599553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c1a7430 con 0x7f9d1c19eef0 2024-04-02T14:45:48.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.600+0000 7f9d19600640 1 -- 172.21.15.62:0/4068599553 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9d14003970 con 0x7f9d1c19eef0 2024-04-02T14:45:48.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.600+0000 7f9d25ba5740 1 Processor -- start 2024-04-02T14:45:48.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.600+0000 7f9d25ba5740 1 -- start start 2024-04-02T14:45:48.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.601+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 0x557d81994b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.601+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 0x557d8199d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.601+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d8199dcb0 0x557d819a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.601+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557d818f8770 con 0x557d819950d0 2024-04-02T14:45:48.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.601+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557d818a0700 con 0x557d8199dcb0 2024-04-02T14:45:48.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.601+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557d818f4720 con 0x557d819947b0 2024-04-02T14:45:48.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.601+0000 7f9d1be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 0x557d8199d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.601+0000 7f9d13e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d8199dcb0 0x557d819a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.601+0000 7f9d13e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d8199dcb0 0x557d819a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48668/0 (socket says 172.21.15.62:48668) 2024-04-02T14:45:48.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.601+0000 7f9d13e00640 1 -- 172.21.15.62:0/4119638488 learned_addr learned my addr 172.21.15.62:0/4119638488 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:48.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.601+0000 7f9d13e00640 1 --2- 172.21.15.62:0/4119638488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 0x557d81994b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d13e00640 1 -- 172.21.15.62:0/4119638488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 msgr2=0x557d81994b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:48.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d13e00640 1 --2- 172.21.15.62:0/4119638488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 0x557d81994b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d13e00640 1 -- 172.21.15.62:0/4119638488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 msgr2=0x557d8199d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:48.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d13e00640 1 --2- 172.21.15.62:0/4119638488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 0x557d8199d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d13e00640 1 -- 172.21.15.62:0/4119638488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d816d67d0 con 0x557d8199dcb0 2024-04-02T14:45:48.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d19600640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9d000a3d20 0x7f9d000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d19600640 1 -- 172.21.15.62:0/4068599553 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f9d140f2b40 con 0x7f9d1c19eef0 2024-04-02T14:45:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d13e00640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9d000a3d20 0x7f9d000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d13e00640 1 --2- 172.21.15.62:0/4119638488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d8199dcb0 0x557d819a00a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9d04011740 tx=0x7f9d040046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50eb8108e71c4134 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d11600640 1 -- 172.21.15.62:0/4119638488 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04004370 con 0x557d8199dcb0 2024-04-02T14:45:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d11600640 1 -- 172.21.15.62:0/4119638488 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9d0400fcc0 con 0x557d8199dcb0 2024-04-02T14:45:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.602+0000 7f9d11600640 1 -- 172.21.15.62:0/4119638488 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04020670 con 0x557d8199dcb0 2024-04-02T14:45:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.603+0000 7f9d25ba5740 1 -- 172.21.15.62:0/4119638488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d8199dcb0 msgr2=0x557d819a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.603+0000 7f9d13e00640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9d000a3d20 0x7f9d000a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9d04020000 tx=0x7f9d04020cc0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.603+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/4119638488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d8199dcb0 0x557d819a00a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9d04011740 tx=0x7f9d040046c0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.603+0000 7f9d25ba5740 1 -- 172.21.15.62:0/4119638488 shutdown_connections 2024-04-02T14:45:48.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.603+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/4119638488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d8199dcb0 0x557d819a00a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.603+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/4119638488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 0x557d8199d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.603+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/4119638488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 0x557d81994b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.603+0000 7f9d25ba5740 1 -- 172.21.15.62:0/4119638488 >> 172.21.15.62:0/4119638488 conn(0x557d8198eee0 msgr2=0x557d8198f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:48.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.603+0000 7f9d25ba5740 1 -- 172.21.15.62:0/4119638488 shutdown_connections 2024-04-02T14:45:48.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.603+0000 7f9d25ba5740 1 -- 172.21.15.62:0/4119638488 wait complete. 2024-04-02T14:45:48.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.604+0000 7f9d25ba5740 1 -- start start 2024-04-02T14:45:48.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.604+0000 7f9d25ba5740 1 Processor -- start 2024-04-02T14:45:48.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.604+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819947b0 0x557d8199ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.604+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 0x557d8199b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d8199dcb0 0x557d8199b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557d819a0aa0 con 0x557d819950d0 2024-04-02T14:45:48.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557d818b3ac0 con 0x557d819947b0 2024-04-02T14:45:48.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557d819a0c20 con 0x557d8199dcb0 2024-04-02T14:45:48.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d25ba5740 10 client.0 _ll_register_callbacks cb 0x557d818b6180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:48.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d13e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 0x557d8199b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d13e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 0x557d8199b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35404/0 (socket says 172.21.15.62:35404) 2024-04-02T14:45:48.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d13e00640 1 -- 172.21.15.62:0/1017531484 learned_addr learned my addr 172.21.15.62:0/1017531484 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:48.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d1be00640 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819947b0 0x557d8199ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d20e00640 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d8199dcb0 0x557d8199b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d13e00640 1 -- 172.21.15.62:0/1017531484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d8199dcb0 msgr2=0x557d8199b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:48.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d13e00640 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d8199dcb0 0x557d8199b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d13e00640 1 -- 172.21.15.62:0/1017531484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819947b0 msgr2=0x557d8199ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d13e00640 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819947b0 0x557d8199ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d13e00640 1 -- 172.21.15.62:0/1017531484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d8160c3f0 con 0x557d819950d0 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d20e00640 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d8199dcb0 0x557d8199b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.605+0000 7f9d1be00640 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819947b0 0x557d8199ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.606+0000 7f9d13e00640 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 0x557d8199b1c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9d040207b0 tx=0x7f9d04008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58889eba80b8b2d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.606+0000 7f9d0a000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.606+0000 7f9d0a000640 1 -- 172.21.15.62:0/1017531484 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04039060 con 0x557d819950d0 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.606+0000 7f9d0a000640 1 -- 172.21.15.62:0/1017531484 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f9d040035f0 con 0x557d819950d0 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.606+0000 7f9d25ba5740 1 -- 172.21.15.62:0/1017531484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d0400fcc0 con 0x557d819950d0 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.606+0000 7f9d0a000640 1 -- 172.21.15.62:0/1017531484 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d0400f790 con 0x557d819950d0 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.606+0000 7f9d25ba5740 10 client.15840 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:48.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.606+0000 7f9d25ba5740 1 -- 172.21.15.62:0/1017531484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d815cc0f0 con 0x557d819950d0 2024-04-02T14:45:48.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.607+0000 7f9d0a000640 1 -- 172.21.15.62:0/1017531484 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9d0400fcc0 con 0x557d819950d0 2024-04-02T14:45:48.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.607+0000 7f9d0a000640 1 -- 172.21.15.62:0/1017531484 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9d0400f930 con 0x557d819950d0 2024-04-02T14:45:48.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.607+0000 7f9d25ba5740 10 client.15840 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:48.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.607+0000 7f9d25ba5740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:48.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.607+0000 7f9d25ba5740 20 client.15840 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:48.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.607+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x557d819adc90 0x557d819b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.607+0000 7f9d25ba5740 4 client.15840 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:48.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.607+0000 7f9d25ba5740 1 -- 172.21.15.62:0/1017531484 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d81899630 con 0x557d819adc90 2024-04-02T14:45:48.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.607+0000 7f9d1be00640 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x557d819adc90 0x557d819b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.608+0000 7f9d1be00640 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x557d819adc90 0x557d819b0080 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52bbf5e19928d325 server_cookie=edb951058c0b0212 in_seq=0 out_seq=0 2024-04-02T14:45:48.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.608+0000 7f9d0a000640 10 client.15840 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:48.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d0a000640 1 -- 172.21.15.62:0/1017531484 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d81899630 con 0x557d819adc90 2024-04-02T14:45:48.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d0a000640 10 client.15840 handle_command_reply: tid=0 2024-04-02T14:45:48.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d25ba5740 1 client.15840 shutdown 2024-04-02T14:45:48.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d25ba5740 1 -- 172.21.15.62:0/1017531484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 msgr2=0x557d8199b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 0x557d8199b1c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9d040207b0 tx=0x7f9d04008000 comp rx=0 tx=0).stop 2024-04-02T14:45:48.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d25ba5740 1 -- 172.21.15.62:0/1017531484 shutdown_connections 2024-04-02T14:45:48.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x557d819adc90 0x557d819b0080 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d8199dcb0 0x557d8199b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819950d0 0x557d8199b1c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/1017531484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819947b0 0x557d8199ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d25ba5740 1 -- 172.21.15.62:0/1017531484 >> 172.21.15.62:0/1017531484 conn(0x557d8198eee0 msgr2=0x557d81990c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:48.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.609+0000 7f9d0a000640 0 client.15840 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:48.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.610+0000 7f9d25ba5740 1 -- 172.21.15.62:0/1017531484 shutdown_connections 2024-04-02T14:45:48.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.610+0000 7f9d25ba5740 1 -- 172.21.15.62:0/1017531484 wait complete. 2024-04-02T14:45:48.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.610+0000 7f9d25ba5740 20 client.15840 trim_cache size 0 max 16384 2024-04-02T14:45:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.629+0000 7f9d25ba5740 1 Processor -- start 2024-04-02T14:45:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.629+0000 7f9d25ba5740 1 -- start start 2024-04-02T14:45:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819947b0 0x557d8199d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 0x557d819a9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d8199dcb0 0x557d8198f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1af940 con 0x557d819947b0 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1b1480 con 0x557d819950d0 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1b0010 con 0x557d8199dcb0 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d13e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 0x557d819a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d13e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 0x557d819a9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48676/0 (socket says 172.21.15.62:48676) 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d13e00640 1 -- 172.21.15.62:0/2688313548 learned_addr learned my addr 172.21.15.62:0/2688313548 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d20e00640 1 --2- 172.21.15.62:0/2688313548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d8199dcb0 0x557d8198f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d1be00640 1 --2- 172.21.15.62:0/2688313548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819947b0 0x557d8199d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d13e00640 1 -- 172.21.15.62:0/2688313548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d8199dcb0 msgr2=0x557d8198f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d13e00640 1 --2- 172.21.15.62:0/2688313548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d8199dcb0 0x557d8198f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d13e00640 1 -- 172.21.15.62:0/2688313548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819947b0 msgr2=0x557d8199d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d13e00640 1 --2- 172.21.15.62:0/2688313548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819947b0 0x557d8199d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.630+0000 7f9d13e00640 1 -- 172.21.15.62:0/2688313548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d819b0640 con 0x557d819950d0 2024-04-02T14:45:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.631+0000 7f9d13e00640 1 --2- 172.21.15.62:0/2688313548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 0x557d819a9960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9d04003fd0 tx=0x7f9d04009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bb628b6597dd864 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.631+0000 7f9d10c00640 1 -- 172.21.15.62:0/2688313548 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04039040 con 0x557d819950d0 2024-04-02T14:45:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.631+0000 7f9d10c00640 1 -- 172.21.15.62:0/2688313548 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9d04009dc0 con 0x557d819950d0 2024-04-02T14:45:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.631+0000 7f9d10c00640 1 -- 172.21.15.62:0/2688313548 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040035f0 con 0x557d819950d0 2024-04-02T14:45:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.632+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2688313548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 msgr2=0x557d819a9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.632+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/2688313548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 0x557d819a9960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9d04003fd0 tx=0x7f9d04009bb0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.632+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2688313548 shutdown_connections 2024-04-02T14:45:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.632+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/2688313548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d8199dcb0 0x557d8198f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.632+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/2688313548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 0x557d819a9960 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.632+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/2688313548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d819947b0 0x557d8199d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.632+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2688313548 >> 172.21.15.62:0/2688313548 conn(0x557d8198eee0 msgr2=0x557d81990330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.632+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2688313548 shutdown_connections 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.632+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2688313548 wait complete. 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.632+0000 7f9d25ba5740 1 -- start start 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.632+0000 7f9d25ba5740 1 Processor -- start 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 0x557d819a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 0x557d819a1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d25ba5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d8199dcb0 0x557d81a0c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1b0db0 con 0x557d8199dcb0 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1b06e0 con 0x557d819950d0 2024-04-02T14:45:48.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d25ba5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1ad940 con 0x557d819947b0 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d25ba5740 10 client.0 _ll_register_callbacks cb 0x7f9d0c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d1be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 0x557d819a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d20e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d8199dcb0 0x557d81a0c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d1be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 0x557d819a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:42048/0 (socket says 172.21.15.62:42048) 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d13e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 0x557d819a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.633+0000 7f9d1be00640 1 -- 172.21.15.62:0/2761995502 learned_addr learned my addr 172.21.15.62:0/2761995502 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.634+0000 7f9d1be00640 1 -- 172.21.15.62:0/2761995502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 msgr2=0x557d819a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.634+0000 7f9d1be00640 1 --2- 172.21.15.62:0/2761995502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 0x557d819a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.634+0000 7f9d1be00640 1 -- 172.21.15.62:0/2761995502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d8199dcb0 msgr2=0x557d81a0c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.634+0000 7f9d1be00640 1 --2- 172.21.15.62:0/2761995502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d8199dcb0 0x557d81a0c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.634+0000 7f9d1be00640 1 -- 172.21.15.62:0/2761995502 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d819a02d0 con 0x557d819947b0 2024-04-02T14:45:48.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.634+0000 7f9d1be00640 1 --2- 172.21.15.62:0/2761995502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 0x557d819a1120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c012600 tx=0x7f9d0c012ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29ae60bdc7e99d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.634+0000 7f9d0a000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.634+0000 7f9d0a000640 1 -- 172.21.15.62:0/2761995502 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c004030 con 0x557d819947b0 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.634+0000 7f9d0a000640 1 -- 172.21.15.62:0/2761995502 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f9d0c0041d0 con 0x557d819947b0 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.634+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2761995502 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d04009dc0 con 0x557d819947b0 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.635+0000 7f9d0a000640 1 -- 172.21.15.62:0/2761995502 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d0c02f050 con 0x557d819947b0 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.635+0000 7f9d25ba5740 10 client.25942 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.635+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2761995502 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d81a13970 con 0x557d819947b0 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.635+0000 7f9d0a000640 1 -- 172.21.15.62:0/2761995502 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9d04009dc0 con 0x557d819947b0 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.635+0000 7f9d0a000640 1 -- 172.21.15.62:0/2761995502 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9d0c003ae0 con 0x557d819947b0 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.635+0000 7f9d25ba5740 10 client.25942 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.635+0000 7f9d25ba5740 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.635+0000 7f9d25ba5740 20 client.25942 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.635+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/2761995502 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x557d81a140f0 0x557d81a16510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.635+0000 7f9d25ba5740 4 client.25942 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.635+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2761995502 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d81905cb0 con 0x557d81a140f0 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.636+0000 7f9d13e00640 1 --2- 172.21.15.62:0/2761995502 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x557d81a140f0 0x557d81a16510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.636+0000 7f9d13e00640 1 --2- 172.21.15.62:0/2761995502 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x557d81a140f0 0x557d81a16510 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14db4283ba6b1d34 server_cookie=4706331033ed4213 in_seq=0 out_seq=0 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.636+0000 7f9d0a000640 10 client.25942 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.636+0000 7f9d0a000640 1 -- 172.21.15.62:0/2761995502 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557d81905cb0 con 0x557d81a140f0 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.636+0000 7f9d0a000640 10 client.25942 handle_command_reply: tid=0 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 1 client.25942 shutdown 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2761995502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 msgr2=0x557d819a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/2761995502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 0x557d819a1120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c012600 tx=0x7f9d0c012ae0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2761995502 shutdown_connections 2024-04-02T14:45:48.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/2761995502 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x557d81a140f0 0x557d81a16510 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/2761995502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d8199dcb0 0x557d81a0c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/2761995502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557d819950d0 0x557d819a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 1 --2- 172.21.15.62:0/2761995502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d819947b0 0x557d819a1120 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d0a000640 0 client.25942 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:48.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2761995502 >> 172.21.15.62:0/2761995502 conn(0x557d8198eee0 msgr2=0x557d81990bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:48.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2761995502 shutdown_connections 2024-04-02T14:45:48.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 1 -- 172.21.15.62:0/2761995502 wait complete. 2024-04-02T14:45:48.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.637+0000 7f9d25ba5740 20 client.25942 trim_cache size 0 max 16384 2024-04-02T14:45:48.637 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:45:48.638 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-02T14:45:48.638 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-02T14:45:48.638 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:45:48.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 -- 172.21.15.62:0/4068599553 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9d000a3d20 msgr2=0x7f9d000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9d000a3d20 0x7f9d000a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9d04020000 tx=0x7f9d04020cc0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 -- 172.21.15.62:0/4068599553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d1c19eef0 msgr2=0x7f9d1c19f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d1c19eef0 0x7f9d1c19f3a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9d14037040 tx=0x7f9d14032f70 comp rx=0 tx=0).stop 2024-04-02T14:45:48.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 -- 172.21.15.62:0/4068599553 shutdown_connections 2024-04-02T14:45:48.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9d000a3d20 0x7f9d000a61e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d1c19eef0 0x7f9d1c19f3a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c108fe0 0x7f9d1c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 --2- 172.21.15.62:0/4068599553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d1c103f10 0x7f9d1c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 -- 172.21.15.62:0/4068599553 >> 172.21.15.62:0/4068599553 conn(0x7f9d1c0ff750 msgr2=0x7f9d1c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:48.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 -- 172.21.15.62:0/4068599553 shutdown_connections 2024-04-02T14:45:48.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.638+0000 7f9d22e00640 1 -- 172.21.15.62:0/4068599553 wait complete. 2024-04-02T14:45:48.651 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:45:48.651 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:45:48.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.877+0000 7f869b200640 1 -- 172.21.15.62:0/685175324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8694108ae0 msgr2=0x7f869410cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.877+0000 7f869b200640 1 --2- 172.21.15.62:0/685175324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8694108ae0 0x7f869410cb20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8684009860 tx=0x7f868402bb70 comp rx=0 tx=0).stop 2024-04-02T14:45:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.877+0000 7f869b200640 1 -- 172.21.15.62:0/685175324 shutdown_connections 2024-04-02T14:45:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.877+0000 7f869b200640 1 --2- 172.21.15.62:0/685175324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f869410d200 0x7f8694114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.877+0000 7f869b200640 1 --2- 172.21.15.62:0/685175324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8694108ae0 0x7f869410cb20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.877+0000 7f869b200640 1 --2- 172.21.15.62:0/685175324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8694108130 0x7f8694108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.877+0000 7f869b200640 1 -- 172.21.15.62:0/685175324 >> 172.21.15.62:0/685175324 conn(0x7f8694077620 msgr2=0x7f8694079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.877+0000 7f869b200640 1 -- 172.21.15.62:0/685175324 shutdown_connections 2024-04-02T14:45:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.877+0000 7f869b200640 1 -- 172.21.15.62:0/685175324 wait complete. 2024-04-02T14:45:48.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.878+0000 7f869b200640 1 Processor -- start 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.878+0000 7f869b200640 1 -- start start 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f869b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8694108130 0x7f86941a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f869b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f869410d200 0x7f86941a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f869b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f869419e750 0x7f869419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f869b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f86941183d0 con 0x7f869410d200 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f869b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8694118250 con 0x7f869419e750 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f869b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8694118550 con 0x7f8694108130 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f8699400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f869410d200 0x7f86941a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f8699400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f869410d200 0x7f86941a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35430/0 (socket says 172.21.15.62:35430) 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f8699400640 1 -- 172.21.15.62:0/1958862119 learned_addr learned my addr 172.21.15.62:0/1958862119 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f8699e00640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8694108130 0x7f86941a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f869a800640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f869419e750 0x7f869419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f8699400640 1 -- 172.21.15.62:0/1958862119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8694108130 msgr2=0x7f86941a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f8699400640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8694108130 0x7f86941a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f8699400640 1 -- 172.21.15.62:0/1958862119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f869419e750 msgr2=0x7f869419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:48.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f8699400640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f869419e750 0x7f869419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:48.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f8699400640 1 -- 172.21.15.62:0/1958862119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869419f350 con 0x7f869410d200 2024-04-02T14:45:48.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.879+0000 7f8699e00640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8694108130 0x7f86941a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:48.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.880+0000 7f8699400640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f869410d200 0x7f86941a4b80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8684037040 tx=0x7f86840045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:48.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.880+0000 7f868ea00640 1 -- 172.21.15.62:0/1958862119 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684041020 con 0x7f869410d200 2024-04-02T14:45:48.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.880+0000 7f868ea00640 1 -- 172.21.15.62:0/1958862119 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f868402ca10 con 0x7f869410d200 2024-04-02T14:45:48.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.880+0000 7f869b200640 1 -- 172.21.15.62:0/1958862119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869419f5e0 con 0x7f869410d200 2024-04-02T14:45:48.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.880+0000 7f868ea00640 1 -- 172.21.15.62:0/1958862119 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684039040 con 0x7f869410d200 2024-04-02T14:45:48.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.880+0000 7f869b200640 1 -- 172.21.15.62:0/1958862119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86941ab6d0 con 0x7f869410d200 2024-04-02T14:45:48.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.882+0000 7f869b200640 1 -- 172.21.15.62:0/1958862119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8694109600 con 0x7f869410d200 2024-04-02T14:45:48.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.884+0000 7f868ea00640 1 -- 172.21.15.62:0/1958862119 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8684044020 con 0x7f869410d200 2024-04-02T14:45:48.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.885+0000 7f868ea00640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f867c0a3d20 0x7f867c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:48.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.885+0000 7f868ea00640 1 -- 172.21.15.62:0/1958862119 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f86840f22c0 con 0x7f869410d200 2024-04-02T14:45:48.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.885+0000 7f8699e00640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f867c0a3d20 0x7f867c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:48.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.886+0000 7f8699e00640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f867c0a3d20 0x7f867c0a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8688005fd0 tx=0x7f8688009260 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:48.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:48.888+0000 7f868ea00640 1 -- 172.21.15.62:0/1958862119 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f868404b020 con 0x7f869410d200 2024-04-02T14:45:49.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.139+0000 7f869b200640 1 -- 172.21.15.62:0/1958862119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8694110490 con 0x7f869410d200 2024-04-02T14:45:49.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.140+0000 7f868ea00640 1 -- 172.21.15.62:0/1958862119 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f86840bea60 con 0x7f869410d200 2024-04-02T14:45:49.136 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:45:49.136 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:44:00.377074+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584,24535,24533,24553,24565]},"id":1}]} 2024-04-02T14:45:49.136 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-04-02T14:45:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.143+0000 7f869b200640 1 -- 172.21.15.62:0/1958862119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f867c0a3d20 msgr2=0x7f867c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.143+0000 7f869b200640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f867c0a3d20 0x7f867c0a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8688005fd0 tx=0x7f8688009260 comp rx=0 tx=0).stop 2024-04-02T14:45:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.143+0000 7f869b200640 1 -- 172.21.15.62:0/1958862119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f869410d200 msgr2=0x7f86941a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.143+0000 7f869b200640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f869410d200 0x7f86941a4b80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8684037040 tx=0x7f86840045b0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.143+0000 7f869b200640 1 -- 172.21.15.62:0/1958862119 shutdown_connections 2024-04-02T14:45:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.143+0000 7f869b200640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f867c0a3d20 0x7f867c0a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.143+0000 7f869b200640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f869419e750 0x7f869419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.143+0000 7f869b200640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f869410d200 0x7f86941a4b80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.143+0000 7f869b200640 1 --2- 172.21.15.62:0/1958862119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8694108130 0x7f86941a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.143+0000 7f869b200640 1 -- 172.21.15.62:0/1958862119 >> 172.21.15.62:0/1958862119 conn(0x7f8694077620 msgr2=0x7f8694078200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.144+0000 7f869b200640 1 -- 172.21.15.62:0/1958862119 shutdown_connections 2024-04-02T14:45:49.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.144+0000 7f869b200640 1 -- 172.21.15.62:0/1958862119 wait complete. 2024-04-02T14:45:49.160 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:45:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.390+0000 7f5fd6a00640 1 -- 172.21.15.62:0/1354015338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd0103df0 msgr2=0x7f5fd01041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.390+0000 7f5fd6a00640 1 --2- 172.21.15.62:0/1354015338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd0103df0 0x7f5fd01041d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009900 tx=0x7f5fc402bc30 comp rx=0 tx=0).stop 2024-04-02T14:45:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.390+0000 7f5fd6a00640 1 -- 172.21.15.62:0/1354015338 shutdown_connections 2024-04-02T14:45:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.390+0000 7f5fd6a00640 1 --2- 172.21.15.62:0/1354015338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd0108ec0 0x7f5fd0110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.390+0000 7f5fd6a00640 1 --2- 172.21.15.62:0/1354015338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd01047a0 0x7f5fd01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.390+0000 7f5fd6a00640 1 --2- 172.21.15.62:0/1354015338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd0103df0 0x7f5fd01041d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.390+0000 7f5fd6a00640 1 -- 172.21.15.62:0/1354015338 >> 172.21.15.62:0/1354015338 conn(0x7f5fd00ff610 msgr2=0x7f5fd0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.390+0000 7f5fd6a00640 1 -- 172.21.15.62:0/1354015338 shutdown_connections 2024-04-02T14:45:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.390+0000 7f5fd6a00640 1 -- 172.21.15.62:0/1354015338 wait complete. 2024-04-02T14:45:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.391+0000 7f5fd6a00640 1 Processor -- start 2024-04-02T14:45:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.392+0000 7f5fd6a00640 1 -- start start 2024-04-02T14:45:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.392+0000 7f5fd6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd0103df0 0x7f5fd01a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.392+0000 7f5fd6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd0108ec0 0x7f5fd01a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.392+0000 7f5fd6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd019a6a0 0x7f5fd019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.392+0000 7f5fd6a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0114040 con 0x7f5fd0108ec0 2024-04-02T14:45:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.392+0000 7f5fd6a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0113ec0 con 0x7f5fd019a6a0 2024-04-02T14:45:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.392+0000 7f5fd6a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01141c0 con 0x7f5fd0103df0 2024-04-02T14:45:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.393+0000 7f5fd6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd019a6a0 0x7f5fd019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.393+0000 7f5fd5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd0103df0 0x7f5fd01a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.393+0000 7f5fd4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd0108ec0 0x7f5fd01a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.393+0000 7f5fd5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd0103df0 0x7f5fd01a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:42088/0 (socket says 172.21.15.62:42088) 2024-04-02T14:45:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.393+0000 7f5fd5600640 1 -- 172.21.15.62:0/2465380618 learned_addr learned my addr 172.21.15.62:0/2465380618 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.393+0000 7f5fd6000640 1 -- 172.21.15.62:0/2465380618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd0103df0 msgr2=0x7f5fd01a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.393+0000 7f5fd6000640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd0103df0 0x7f5fd01a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.393+0000 7f5fd6000640 1 -- 172.21.15.62:0/2465380618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd0108ec0 msgr2=0x7f5fd01a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.393+0000 7f5fd6000640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd0108ec0 0x7f5fd01a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.393+0000 7f5fd6000640 1 -- 172.21.15.62:0/2465380618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd019b2a0 con 0x7f5fd019a6a0 2024-04-02T14:45:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.394+0000 7f5fd6000640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd019a6a0 0x7f5fd019ab20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009fd0 tx=0x7f5fc800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.394+0000 7f5fce000640 1 -- 172.21.15.62:0/2465380618 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800ce20 con 0x7f5fd019a6a0 2024-04-02T14:45:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.394+0000 7f5fd6a00640 1 -- 172.21.15.62:0/2465380618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd01a7380 con 0x7f5fd019a6a0 2024-04-02T14:45:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.394+0000 7f5fce000640 1 -- 172.21.15.62:0/2465380618 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5fc8004b20 con 0x7f5fd019a6a0 2024-04-02T14:45:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.394+0000 7f5fce000640 1 -- 172.21.15.62:0/2465380618 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8010790 con 0x7f5fd019a6a0 2024-04-02T14:45:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.394+0000 7f5fd6a00640 1 -- 172.21.15.62:0/2465380618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd01a7780 con 0x7f5fd019a6a0 2024-04-02T14:45:49.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.397+0000 7f5fdae28740 1 Processor -- start 2024-04-02T14:45:49.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.397+0000 7f5fce000640 1 -- 172.21.15.62:0/2465380618 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5fc8005020 con 0x7f5fd019a6a0 2024-04-02T14:45:49.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.398+0000 7f5fdae28740 1 -- start start 2024-04-02T14:45:49.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.398+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61dfb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.398+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 0x55fb61dfc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.398+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61e05230 0x55fb61e07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.398+0000 7f5fdae28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fb61d3c400 con 0x55fb61dfcba0 2024-04-02T14:45:49.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.398+0000 7f5fdae28740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fb61d20f10 con 0x55fb61dfbec0 2024-04-02T14:45:49.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.398+0000 7f5fdae28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb61d606f0 con 0x55fb61e05230 2024-04-02T14:45:49.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fd4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61dfb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fd5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 0x55fb61dfc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fd4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61dfb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35462/0 (socket says 172.21.15.62:35462) 2024-04-02T14:45:49.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fd4c00640 1 -- 172.21.15.62:0/2480621456 learned_addr learned my addr 172.21.15.62:0/2480621456 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/2480621456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61e05230 0x55fb61e07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fd5600640 1 -- 172.21.15.62:0/2480621456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61e05230 msgr2=0x55fb61e07620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fd5600640 1 --2- 172.21.15.62:0/2480621456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61e05230 0x55fb61e07620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fd5600640 1 -- 172.21.15.62:0/2480621456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 msgr2=0x55fb61dfb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fd5600640 1 --2- 172.21.15.62:0/2480621456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61dfb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fd5600640 1 -- 172.21.15.62:0/2480621456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb61a743f0 con 0x55fb61dfbec0 2024-04-02T14:45:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/2480621456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61dfb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.399+0000 7f5fce000640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5fb80a3da0 0x7f5fb80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.400+0000 7f5fd5600640 1 --2- 172.21.15.62:0/2480621456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 0x55fb61dfc340 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4005ec0 tx=0x7f5fc402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f1a1a02f8dbc17b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.400+0000 7f5fce000640 1 -- 172.21.15.62:0/2465380618 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f5fc80d0e80 con 0x7f5fd019a6a0 2024-04-02T14:45:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.400+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5fb80a3da0 0x7f5fb80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.400+0000 7f5fc2600640 1 -- 172.21.15.62:0/2480621456 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc402c9d0 con 0x55fb61dfbec0 2024-04-02T14:45:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.400+0000 7f5fc2600640 1 -- 172.21.15.62:0/2480621456 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5fc402cb70 con 0x55fb61dfbec0 2024-04-02T14:45:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.400+0000 7f5fc2600640 1 -- 172.21.15.62:0/2480621456 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40046c0 con 0x55fb61dfbec0 2024-04-02T14:45:49.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.401+0000 7f5fdae28740 1 -- 172.21.15.62:0/2480621456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 msgr2=0x55fb61dfc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.401+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5fb80a3da0 0x7f5fb80a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0058a0 tx=0x7f5fbc00b820 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.401+0000 7f5fdae28740 1 --2- 172.21.15.62:0/2480621456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 0x55fb61dfc340 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4005ec0 tx=0x7f5fc402c5d0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.401+0000 7f5fdae28740 1 -- 172.21.15.62:0/2480621456 shutdown_connections 2024-04-02T14:45:49.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.401+0000 7f5fdae28740 1 --2- 172.21.15.62:0/2480621456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61e05230 0x55fb61e07620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.401+0000 7f5fdae28740 1 --2- 172.21.15.62:0/2480621456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 0x55fb61dfc340 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.401+0000 7f5fdae28740 1 --2- 172.21.15.62:0/2480621456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61dfb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.401+0000 7f5fdae28740 1 -- 172.21.15.62:0/2480621456 >> 172.21.15.62:0/2480621456 conn(0x55fb61dc2a50 msgr2=0x55fb61df6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.401+0000 7f5fdae28740 1 -- 172.21.15.62:0/2480621456 shutdown_connections 2024-04-02T14:45:49.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.401+0000 7f5fdae28740 1 -- 172.21.15.62:0/2480621456 wait complete. 2024-04-02T14:45:49.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.401+0000 7f5fdae28740 1 -- start start 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.402+0000 7f5fdae28740 1 Processor -- start 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.402+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfbec0 0x55fb61e02200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.402+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e02740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.402+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 0x55fb61e02c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.402+0000 7f5fdae28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fb61d5c6a0 con 0x55fb61dfcba0 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.402+0000 7f5fdae28740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fb61d08980 con 0x55fb61e05230 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.402+0000 7f5fdae28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb61d1bcb0 con 0x55fb61dfbec0 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.402+0000 7f5fdae28740 10 client.0 _ll_register_callbacks cb 0x55fb61d1b6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e02740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfbec0 0x55fb61e02200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e02740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35470/0 (socket says 172.21.15.62:35470) 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd4c00640 1 -- 172.21.15.62:0/964439692 learned_addr learned my addr 172.21.15.62:0/964439692 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd6000640 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 0x55fb61e02c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd4c00640 1 -- 172.21.15.62:0/964439692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfbec0 msgr2=0x55fb61e02200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfbec0 0x55fb61e02200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd4c00640 1 -- 172.21.15.62:0/964439692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 msgr2=0x55fb61e02c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 0x55fb61e02c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd4c00640 1 -- 172.21.15.62:0/964439692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb61b3e7d0 con 0x55fb61dfcba0 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd6000640 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 0x55fb61e02c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e02740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc011730 tx=0x7f5fbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122d04054a124e63 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.403+0000 7f5fb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.404+0000 7f5fb6a00640 1 -- 172.21.15.62:0/964439692 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0035f0 con 0x55fb61dfcba0 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.404+0000 7f5fb6a00640 1 -- 172.21.15.62:0/964439692 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f5fbc003790 con 0x55fb61dfcba0 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.404+0000 7f5fdae28740 1 -- 172.21.15.62:0/964439692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc402cb70 con 0x55fb61dfcba0 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.404+0000 7f5fb6a00640 1 -- 172.21.15.62:0/964439692 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fbc003990 con 0x55fb61dfcba0 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.404+0000 7f5fdae28740 10 client.15864 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.404+0000 7f5fdae28740 1 -- 172.21.15.62:0/964439692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb61e140b0 con 0x55fb61dfcba0 2024-04-02T14:45:49.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.404+0000 7f5fb6a00640 1 -- 172.21.15.62:0/964439692 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5fc402cb70 con 0x55fb61dfcba0 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.405+0000 7f5fb6a00640 1 -- 172.21.15.62:0/964439692 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5fbc027020 con 0x55fb61dfcba0 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.405+0000 7f5fdae28740 10 client.15864 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.405+0000 7f5fdae28740 10 client.15864 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.405+0000 7f5fdae28740 20 client.15864 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.405+0000 7f5fdae28740 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55fb61e15510 0x55fb61e17970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.405+0000 7f5fdae28740 4 client.15864 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.405+0000 7f5fdae28740 1 -- 172.21.15.62:0/964439692 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb61d20040 con 0x55fb61e15510 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.405+0000 7f5fd5600640 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55fb61e15510 0x55fb61e17970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.406+0000 7f5fd5600640 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55fb61e15510 0x55fb61e17970 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=645dc5cf7f560bad server_cookie=28ee8c9c66e96806 in_seq=0 out_seq=0 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.406+0000 7f5fb6a00640 10 client.15864 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fb6a00640 1 -- 172.21.15.62:0/964439692 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb61d20040 con 0x55fb61e15510 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fb6a00640 10 client.15864 handle_command_reply: tid=0 2024-04-02T14:45:49.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 1 client.15864 shutdown 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 1 -- 172.21.15.62:0/964439692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 msgr2=0x55fb61e02740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e02740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc011730 tx=0x7f5fbc004730 comp rx=0 tx=0).stop 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 1 -- 172.21.15.62:0/964439692 shutdown_connections 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55fb61e15510 0x55fb61e17970 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 0x55fb61e02c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e02740 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fb6a00640 0 client.15864 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 1 --2- 172.21.15.62:0/964439692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfbec0 0x55fb61e02200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 1 -- 172.21.15.62:0/964439692 >> 172.21.15.62:0/964439692 conn(0x55fb61dc2a50 msgr2=0x55fb61df8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 1 -- 172.21.15.62:0/964439692 shutdown_connections 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 1 -- 172.21.15.62:0/964439692 wait complete. 2024-04-02T14:45:49.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.407+0000 7f5fdae28740 20 client.15864 trim_cache size 0 max 16384 2024-04-02T14:45:49.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.422+0000 7f5fdae28740 1 Processor -- start 2024-04-02T14:45:49.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.422+0000 7f5fdae28740 1 -- start start 2024-04-02T14:45:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfbec0 0x55fb61e04930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e10c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 0x55fb61e1d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fdae28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01af6f0 con 0x55fb61dfcba0 2024-04-02T14:45:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fdae28740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01b1230 con 0x55fb61e05230 2024-04-02T14:45:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fdae28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01afdc0 con 0x55fb61dfbec0 2024-04-02T14:45:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fd4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e10c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fd4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e10c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35476/0 (socket says 172.21.15.62:35476) 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fd4c00640 1 -- 172.21.15.62:0/1359065312 learned_addr learned my addr 172.21.15.62:0/1359065312 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fd5600640 1 --2- 172.21.15.62:0/1359065312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfbec0 0x55fb61e04930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fd6000640 1 --2- 172.21.15.62:0/1359065312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 0x55fb61e1d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fd4c00640 1 -- 172.21.15.62:0/1359065312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfbec0 msgr2=0x55fb61e04930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/1359065312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfbec0 0x55fb61e04930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fd4c00640 1 -- 172.21.15.62:0/1359065312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 msgr2=0x55fb61e1d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/1359065312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 0x55fb61e1d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fd4c00640 1 -- 172.21.15.62:0/1359065312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb61dfaef0 con 0x55fb61dfcba0 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.423+0000 7f5fd6000640 1 --2- 172.21.15.62:0/1359065312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 0x55fb61e1d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.424+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/1359065312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e10c40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00f010 tx=0x7f5fbc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb712bbdb82f3a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.424+0000 7f5fc1c00640 1 -- 172.21.15.62:0/1359065312 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc004320 con 0x55fb61dfcba0 2024-04-02T14:45:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.424+0000 7f5fc1c00640 1 -- 172.21.15.62:0/1359065312 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5fbc0044c0 con 0x55fb61dfcba0 2024-04-02T14:45:49.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.424+0000 7f5fc1c00640 1 -- 172.21.15.62:0/1359065312 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0337d0 con 0x55fb61dfcba0 2024-04-02T14:45:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.425+0000 7f5fdae28740 1 -- 172.21.15.62:0/1359065312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 msgr2=0x55fb61e10c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.425+0000 7f5fdae28740 1 --2- 172.21.15.62:0/1359065312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e10c40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00f010 tx=0x7f5fbc0057b0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.425+0000 7f5fdae28740 1 -- 172.21.15.62:0/1359065312 shutdown_connections 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.425+0000 7f5fdae28740 1 --2- 172.21.15.62:0/1359065312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61e05230 0x55fb61e1d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.425+0000 7f5fdae28740 1 --2- 172.21.15.62:0/1359065312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61dfcba0 0x55fb61e10c40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.425+0000 7f5fdae28740 1 --2- 172.21.15.62:0/1359065312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfbec0 0x55fb61e04930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.425+0000 7f5fdae28740 1 -- 172.21.15.62:0/1359065312 >> 172.21.15.62:0/1359065312 conn(0x55fb61dc2a50 msgr2=0x55fb61e00ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.425+0000 7f5fdae28740 1 -- 172.21.15.62:0/1359065312 shutdown_connections 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.425+0000 7f5fdae28740 1 -- 172.21.15.62:0/1359065312 wait complete. 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.425+0000 7f5fdae28740 1 -- start start 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.425+0000 7f5fdae28740 1 Processor -- start 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 0x55fb61e0d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfcba0 0x55fb61dff480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fdae28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61e05230 0x55fb61dff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fdae28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01b0b60 con 0x55fb61e05230 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fdae28740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01b0490 con 0x55fb61dfbec0 2024-04-02T14:45:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fdae28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ad6d0 con 0x55fb61dfcba0 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fdae28740 10 client.0 _ll_register_callbacks cb 0x7f5fbc00bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fd5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 0x55fb61e0d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fd5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 0x55fb61e0d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48750/0 (socket says 172.21.15.62:48750) 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fd5600640 1 -- 172.21.15.62:0/4170504946 learned_addr learned my addr 172.21.15.62:0/4170504946 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fd6000640 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61e05230 0x55fb61dff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.426+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfcba0 0x55fb61dff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.427+0000 7f5fd5600640 1 -- 172.21.15.62:0/4170504946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfcba0 msgr2=0x55fb61dff480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.427+0000 7f5fd5600640 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfcba0 0x55fb61dff480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.427+0000 7f5fd5600640 1 -- 172.21.15.62:0/4170504946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61e05230 msgr2=0x55fb61dff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.427+0000 7f5fd5600640 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61e05230 0x55fb61dff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.427+0000 7f5fd5600640 1 -- 172.21.15.62:0/4170504946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb61e00000 con 0x55fb61dfbec0 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.427+0000 7f5fd5600640 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 0x55fb61e0d340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009430 tx=0x7f5fc402c9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=926003ef71c7136b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.427+0000 7f5fb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.427+0000 7f5fb6a00640 1 -- 172.21.15.62:0/4170504946 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4004240 con 0x55fb61dfbec0 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.427+0000 7f5fb6a00640 1 -- 172.21.15.62:0/4170504946 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f5fc40043e0 con 0x55fb61dfbec0 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.427+0000 7f5fdae28740 1 -- 172.21.15.62:0/4170504946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fbc0044c0 con 0x55fb61dfbec0 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.428+0000 7f5fb6a00640 1 -- 172.21.15.62:0/4170504946 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fc4005040 con 0x55fb61dfbec0 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.428+0000 7f5fdae28740 10 client.25994 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.428+0000 7f5fdae28740 1 -- 172.21.15.62:0/4170504946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb61e7bab0 con 0x55fb61dfbec0 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.428+0000 7f5fb6a00640 1 -- 172.21.15.62:0/4170504946 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5fbc0044c0 con 0x55fb61dfbec0 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.428+0000 7f5fb6a00640 1 -- 172.21.15.62:0/4170504946 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5fc4050070 con 0x55fb61dfbec0 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.428+0000 7f5fdae28740 10 client.25994 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.428+0000 7f5fdae28740 10 client.25994 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.428+0000 7f5fdae28740 20 client.25994 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.428+0000 7f5fdae28740 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55fb61e7bcd0 0x55fb61e7e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.428+0000 7f5fdae28740 4 client.25994 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:45:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.428+0000 7f5fdae28740 1 -- 172.21.15.62:0/4170504946 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb61d6dc30 con 0x55fb61e7bcd0 2024-04-02T14:45:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.429+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55fb61e7bcd0 0x55fb61e7e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.429+0000 7f5fd4c00640 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55fb61e7bcd0 0x55fb61e7e0f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e3c983d3f6bdaf2 server_cookie=659c1f2c43163730 in_seq=0 out_seq=0 2024-04-02T14:45:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.429+0000 7f5fb6a00640 10 client.25994 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:45:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.429+0000 7f5fb6a00640 1 -- 172.21.15.62:0/4170504946 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb61d6dc30 con 0x55fb61e7bcd0 2024-04-02T14:45:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.429+0000 7f5fb6a00640 10 client.25994 handle_command_reply: tid=0 2024-04-02T14:45:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 1 client.25994 shutdown 2024-04-02T14:45:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 1 -- 172.21.15.62:0/4170504946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 msgr2=0x55fb61e0d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 0x55fb61e0d340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009430 tx=0x7f5fc402c9d0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 1 -- 172.21.15.62:0/4170504946 shutdown_connections 2024-04-02T14:45:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x55fb61e7bcd0 0x55fb61e7e0f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb61e05230 0x55fb61dff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb61dfcba0 0x55fb61dff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 1 --2- 172.21.15.62:0/4170504946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fb61dfbec0 0x55fb61e0d340 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fb6a00640 0 client.25994 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:45:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 1 -- 172.21.15.62:0/4170504946 >> 172.21.15.62:0/4170504946 conn(0x55fb61dc2a50 msgr2=0x55fb61e00f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 1 -- 172.21.15.62:0/4170504946 shutdown_connections 2024-04-02T14:45:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 1 -- 172.21.15.62:0/4170504946 wait complete. 2024-04-02T14:45:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.430+0000 7f5fdae28740 20 client.25994 trim_cache size 0 max 16384 2024-04-02T14:45:49.429 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:45:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 -- 172.21.15.62:0/2465380618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5fb80a3da0 msgr2=0x7f5fb80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5fb80a3da0 0x7f5fb80a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0058a0 tx=0x7f5fbc00b820 comp rx=0 tx=0).stop 2024-04-02T14:45:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 -- 172.21.15.62:0/2465380618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd019a6a0 msgr2=0x7f5fd019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd019a6a0 0x7f5fd019ab20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009fd0 tx=0x7f5fc800ef10 comp rx=0 tx=0).stop 2024-04-02T14:45:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 -- 172.21.15.62:0/2465380618 shutdown_connections 2024-04-02T14:45:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f5fb80a3da0 0x7f5fb80a6260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd019a6a0 0x7f5fd019ab20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd0108ec0 0x7f5fd01a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 --2- 172.21.15.62:0/2465380618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd0103df0 0x7f5fd01a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 -- 172.21.15.62:0/2465380618 >> 172.21.15.62:0/2465380618 conn(0x7f5fd00ff610 msgr2=0x7f5fd01019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 -- 172.21.15.62:0/2465380618 shutdown_connections 2024-04-02T14:45:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.431+0000 7f5fd6a00640 1 -- 172.21.15.62:0/2465380618 wait complete. 2024-04-02T14:45:49.445 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:45:49.669 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querycap.xml 2024-04-02T14:45:49.670 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-queryctrl.xml 2024-04-02T14:45:49.671 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querystd.xml 2024-04-02T14:45:49.672 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-reqbufs.xml 2024-04-02T14:45:49.674 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-s-hw-freq-seek.xml 2024-04-02T14:45:49.675 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-streamon.xml 2024-04-02T14:45:49.676 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml 2024-04-02T14:45:49.678 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml 2024-04-02T14:45:49.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.683+0000 7f4e8c800640 1 -- 172.21.15.62:0/1109849804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e84073eb0 msgr2=0x7f4e84074290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.683+0000 7f4e8c800640 1 --2- 172.21.15.62:0/1109849804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e84073eb0 0x7f4e84074290 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009900 tx=0x7f4e7802bcc0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.683+0000 7f4e8c800640 1 -- 172.21.15.62:0/1109849804 shutdown_connections 2024-04-02T14:45:49.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.683+0000 7f4e8c800640 1 --2- 172.21.15.62:0/1109849804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e8410e350 0x7f4e84110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.683+0000 7f4e8c800640 1 --2- 172.21.15.62:0/1109849804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e840747d0 0x7f4e8410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.683+0000 7f4e8c800640 1 --2- 172.21.15.62:0/1109849804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e84073eb0 0x7f4e84074290 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.683+0000 7f4e8c800640 1 -- 172.21.15.62:0/1109849804 >> 172.21.15.62:0/1109849804 conn(0x7f4e84078f00 msgr2=0x7f4e84079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.684+0000 7f4e8c800640 1 -- 172.21.15.62:0/1109849804 shutdown_connections 2024-04-02T14:45:49.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.684+0000 7f4e8c800640 1 -- 172.21.15.62:0/1109849804 wait complete. 2024-04-02T14:45:49.680 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml 2024-04-02T14:45:49.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.685+0000 7f4e8c800640 1 Processor -- start 2024-04-02T14:45:49.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.685+0000 7f4e8c800640 1 -- start start 2024-04-02T14:45:49.681 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-crop.xml 2024-04-02T14:45:49.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.685+0000 7f4e8c800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073eb0 0x7f4e8419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.685+0000 7f4e8c800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840747d0 0x7f4e8419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.685+0000 7f4e8c800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8410e350 0x7f4e8419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.685+0000 7f4e8c800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84114190 con 0x7f4e840747d0 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.685+0000 7f4e8c800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84114010 con 0x7f4e8410e350 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.685+0000 7f4e8c800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84114310 con 0x7f4e84073eb0 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e89400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840747d0 0x7f4e8419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e8a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8410e350 0x7f4e8419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e89e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073eb0 0x7f4e8419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e89400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840747d0 0x7f4e8419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35502/0 (socket says 172.21.15.62:35502) 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e89e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073eb0 0x7f4e8419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:42128/0 (socket says 172.21.15.62:42128) 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e8a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8410e350 0x7f4e8419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48770/0 (socket says 172.21.15.62:48770) 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e89400640 1 -- 172.21.15.62:0/4164113776 learned_addr learned my addr 172.21.15.62:0/4164113776 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e89400640 1 -- 172.21.15.62:0/4164113776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073eb0 msgr2=0x7f4e8419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e89400640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073eb0 0x7f4e8419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e89400640 1 -- 172.21.15.62:0/4164113776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8410e350 msgr2=0x7f4e8419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e89400640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8410e350 0x7f4e8419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e89400640 1 -- 172.21.15.62:0/4164113776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8419f660 con 0x7f4e840747d0 2024-04-02T14:45:49.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e8a800640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8410e350 0x7f4e8419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:49.685 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml 2024-04-02T14:45:49.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.686+0000 7f4e89400640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840747d0 0x7f4e8419abb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400e9f0 tx=0x7f4e7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.687+0000 7f4e7ea00640 1 -- 172.21.15.62:0/4164113776 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7400cd80 con 0x7f4e840747d0 2024-04-02T14:45:49.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.687+0000 7f4e8c800640 1 -- 172.21.15.62:0/4164113776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8419f950 con 0x7f4e840747d0 2024-04-02T14:45:49.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.687+0000 7f4e7ea00640 1 -- 172.21.15.62:0/4164113776 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e74005520 con 0x7f4e840747d0 2024-04-02T14:45:49.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.687+0000 7f4e7ea00640 1 -- 172.21.15.62:0/4164113776 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74010690 con 0x7f4e840747d0 2024-04-02T14:45:49.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.687+0000 7f4e8c800640 1 -- 172.21.15.62:0/4164113776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e841a71e0 con 0x7f4e840747d0 2024-04-02T14:45:49.686 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml 2024-04-02T14:45:49.686 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml 2024-04-02T14:45:49.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.690+0000 7f4e8f473740 1 Processor -- start 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.690+0000 7f4e8f473740 1 -- start start 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e7ea00640 1 -- 172.21.15.62:0/4164113776 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4e740046c0 con 0x7f4e840747d0 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a899f0 0x560169a88740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a88d10 0x560169a89190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 0x560169a94470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e8f473740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5601699c9400 con 0x560169a899f0 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e8f473740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5601699adf10 con 0x560169a88d10 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e8f473740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5601699ed6f0 con 0x560169a92080 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e8a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a899f0 0x560169a88740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e89e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a88d10 0x560169a89190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e89400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 0x560169a94470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e89e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a88d10 0x560169a89190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48782/0 (socket says 172.21.15.62:48782) 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.691+0000 7f4e89e00640 1 -- 172.21.15.62:0/145764004 learned_addr learned my addr 172.21.15.62:0/145764004 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.692+0000 7f4e89e00640 1 -- 172.21.15.62:0/145764004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 msgr2=0x560169a94470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.692+0000 7f4e89e00640 1 --2- 172.21.15.62:0/145764004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 0x560169a94470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.692+0000 7f4e89e00640 1 -- 172.21.15.62:0/145764004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a899f0 msgr2=0x560169a88740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.692+0000 7f4e89e00640 1 --2- 172.21.15.62:0/145764004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a899f0 0x560169a88740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.689 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml 2024-04-02T14:45:49.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.692+0000 7f4e89e00640 1 -- 172.21.15.62:0/145764004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601697013f0 con 0x560169a88d10 2024-04-02T14:45:49.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.692+0000 7f4e89e00640 1 --2- 172.21.15.62:0/145764004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a88d10 0x560169a89190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009900 tx=0x7f4e780095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cfce9ae6cb18ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.692+0000 7f4e73000640 1 -- 172.21.15.62:0/145764004 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7802c7a0 con 0x560169a88d10 2024-04-02T14:45:49.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.692+0000 7f4e73000640 1 -- 172.21.15.62:0/145764004 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e78004040 con 0x560169a88d10 2024-04-02T14:45:49.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.692+0000 7f4e73000640 1 -- 172.21.15.62:0/145764004 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e780046c0 con 0x560169a88d10 2024-04-02T14:45:49.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.692+0000 7f4e7ea00640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f4e6c0a3d20 0x7f4e6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.693+0000 7f4e7ea00640 1 -- 172.21.15.62:0/4164113776 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f4e740d0c20 con 0x7f4e840747d0 2024-04-02T14:45:49.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.693+0000 7f4e8a800640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f4e6c0a3d20 0x7f4e6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.691 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_525.gif.b64 2024-04-02T14:45:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.693+0000 7f4e8f473740 1 -- 172.21.15.62:0/145764004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a88d10 msgr2=0x560169a89190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.693+0000 7f4e8f473740 1 --2- 172.21.15.62:0/145764004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a88d10 0x560169a89190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009900 tx=0x7f4e780095f0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.693+0000 7f4e8f473740 1 -- 172.21.15.62:0/145764004 shutdown_connections 2024-04-02T14:45:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.693+0000 7f4e8f473740 1 --2- 172.21.15.62:0/145764004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 0x560169a94470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.693+0000 7f4e8f473740 1 --2- 172.21.15.62:0/145764004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a88d10 0x560169a89190 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.693+0000 7f4e8f473740 1 --2- 172.21.15.62:0/145764004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a899f0 0x560169a88740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.693+0000 7f4e8f473740 1 -- 172.21.15.62:0/145764004 >> 172.21.15.62:0/145764004 conn(0x560169a4fa50 msgr2=0x560169a83760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.694+0000 7f4e8f473740 1 -- 172.21.15.62:0/145764004 shutdown_connections 2024-04-02T14:45:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.694+0000 7f4e8f473740 1 -- 172.21.15.62:0/145764004 wait complete. 2024-04-02T14:45:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.694+0000 7f4e8a800640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f4e6c0a3d20 0x7f4e6c0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009fd0 tx=0x7f4e80009500 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.694+0000 7f4e8f473740 1 -- start start 2024-04-02T14:45:49.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.694+0000 7f4e8f473740 1 Processor -- start 2024-04-02T14:45:49.693 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_625.gif.b64 2024-04-02T14:45:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a88d10 0x560169a8f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 0x560169a8f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a92080 0x560169a8fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e8f473740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5601699e96a0 con 0x560169a92080 2024-04-02T14:45:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e8f473740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560169995980 con 0x560169a899f0 2024-04-02T14:45:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e8f473740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5601699a8cb0 con 0x560169a88d10 2024-04-02T14:45:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e8f473740 10 client.0 _ll_register_callbacks cb 0x5601699a86a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e89400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 0x560169a8f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e8a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a92080 0x560169a8fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e89400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 0x560169a8f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48784/0 (socket says 172.21.15.62:48784) 2024-04-02T14:45:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e89400640 1 -- 172.21.15.62:0/2972093583 learned_addr learned my addr 172.21.15.62:0/2972093583 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e89e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a88d10 0x560169a8f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e89400640 1 -- 172.21.15.62:0/2972093583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a88d10 msgr2=0x560169a8f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e89400640 1 --2- 172.21.15.62:0/2972093583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a88d10 0x560169a8f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e89400640 1 -- 172.21.15.62:0/2972093583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a92080 msgr2=0x560169a8fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.695+0000 7f4e89400640 1 --2- 172.21.15.62:0/2972093583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a92080 0x560169a8fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.696+0000 7f4e89400640 1 -- 172.21.15.62:0/2972093583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601697cb7d0 con 0x560169a899f0 2024-04-02T14:45:49.695 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_hsync.gif.b64 2024-04-02T14:45:49.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.696+0000 7f4e89400640 1 --2- 172.21.15.62:0/2972093583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 0x560169a8f570 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400c6a0 tx=0x7f4e7400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9174d0edd2b08f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.696+0000 7f4e67400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.696+0000 7f4e67400640 1 -- 172.21.15.62:0/2972093583 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7409c510 con 0x560169a899f0 2024-04-02T14:45:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.696+0000 7f4e67400640 1 -- 172.21.15.62:0/2972093583 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f4e740a5390 con 0x560169a899f0 2024-04-02T14:45:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.696+0000 7f4e8f473740 1 -- 172.21.15.62:0/2972093583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e78004040 con 0x560169a899f0 2024-04-02T14:45:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.696+0000 7f4e67400640 1 -- 172.21.15.62:0/2972093583 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e740b5030 con 0x560169a899f0 2024-04-02T14:45:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.696+0000 7f4e8f473740 10 client.26006 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.696+0000 7f4e8f473740 1 -- 172.21.15.62:0/2972093583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560169aa0f00 con 0x560169a899f0 2024-04-02T14:45:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.697+0000 7f4e67400640 1 -- 172.21.15.62:0/2972093583 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e78004040 con 0x560169a899f0 2024-04-02T14:45:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.697+0000 7f4e67400640 1 -- 172.21.15.62:0/2972093583 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4e7409c000 con 0x560169a899f0 2024-04-02T14:45:49.697 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/media_api.tmpl 2024-04-02T14:45:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.697+0000 7f4e8f473740 10 client.26006 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.697+0000 7f4e8f473740 10 client.26006 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:45:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.697+0000 7f4e8f473740 20 client.26006 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.697+0000 7f4e8f473740 1 --2- 172.21.15.62:0/2972093583 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560169aa23b0 0x560169aa47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.697+0000 7f4e8f473740 4 client.26006 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.697+0000 7f4e8f473740 1 -- 172.21.15.62:0/2972093583 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601699ad040 con 0x560169aa23b0 2024-04-02T14:45:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.697+0000 7f4e89e00640 1 --2- 172.21.15.62:0/2972093583 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560169aa23b0 0x560169aa47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.699 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/mtdnand.tmpl 2024-04-02T14:45:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.698+0000 7f4e89e00640 1 --2- 172.21.15.62:0/2972093583 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560169aa23b0 0x560169aa47a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b29a7585aacc0d7b server_cookie=8e7e4f79d330c4f9 in_seq=0 out_seq=0 2024-04-02T14:45:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.698+0000 7f4e67400640 10 client.26006 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:45:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e67400640 1 -- 172.21.15.62:0/2972093583 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601699ad040 con 0x560169aa23b0 2024-04-02T14:45:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e67400640 10 client.26006 handle_command_reply: tid=0 2024-04-02T14:45:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 1 client.26006 shutdown 2024-04-02T14:45:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 1 -- 172.21.15.62:0/2972093583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 msgr2=0x560169a8f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 1 --2- 172.21.15.62:0/2972093583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 0x560169a8f570 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400c6a0 tx=0x7f4e7400cf90 comp rx=0 tx=0).stop 2024-04-02T14:45:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 1 -- 172.21.15.62:0/2972093583 shutdown_connections 2024-04-02T14:45:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 1 --2- 172.21.15.62:0/2972093583 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560169aa23b0 0x560169aa47a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 1 --2- 172.21.15.62:0/2972093583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a92080 0x560169a8fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.701 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/networking.tmpl 2024-04-02T14:45:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 1 --2- 172.21.15.62:0/2972093583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 0x560169a8f570 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e67400640 0 client.26006 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:45:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 1 --2- 172.21.15.62:0/2972093583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a88d10 0x560169a8f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 1 -- 172.21.15.62:0/2972093583 >> 172.21.15.62:0/2972093583 conn(0x560169a4fa50 msgr2=0x560169a854b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 1 -- 172.21.15.62:0/2972093583 shutdown_connections 2024-04-02T14:45:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 1 -- 172.21.15.62:0/2972093583 wait complete. 2024-04-02T14:45:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.699+0000 7f4e8f473740 20 client.26006 trim_cache size 0 max 16384 2024-04-02T14:45:49.703 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/rapidio.tmpl 2024-04-02T14:45:49.703 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/regulator.tmpl 2024-04-02T14:45:49.703 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/s390-drivers.tmpl 2024-04-02T14:45:49.703 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/scsi.tmpl 2024-04-02T14:45:49.703 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/sh.tmpl 2024-04-02T14:45:49.703 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/stylesheet.xsl 2024-04-02T14:45:49.703 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/tracepoint.tmpl 2024-04-02T14:45:49.704 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/uio-howto.tmpl 2024-04-02T14:45:49.705 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/usb.tmpl 2024-04-02T14:45:49.707 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/w1.tmpl 2024-04-02T14:45:49.708 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/writing-an-alsa-driver.tmpl 2024-04-02T14:45:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.717+0000 7f4e8f473740 1 Processor -- start 2024-04-02T14:45:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.717+0000 7f4e8f473740 1 -- start start 2024-04-02T14:45:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.718+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a88d10 0x560169a9dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.718+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 0x560169a8e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.718+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 0x560169aaa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.718+0000 7f4e8f473740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841af960 con 0x560169a88d10 2024-04-02T14:45:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.718+0000 7f4e8f473740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841b14a0 con 0x560169a899f0 2024-04-02T14:45:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.718+0000 7f4e8f473740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841b0030 con 0x560169a92080 2024-04-02T14:45:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.718+0000 7f4e8a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 0x560169aaa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.718+0000 7f4e8a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 0x560169aaa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:42138/0 (socket says 172.21.15.62:42138) 2024-04-02T14:45:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.718+0000 7f4e8a800640 1 -- 172.21.15.62:0/4005690705 learned_addr learned my addr 172.21.15.62:0/4005690705 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.718+0000 7f4e89400640 1 --2- 172.21.15.62:0/4005690705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 0x560169a8e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.718+0000 7f4e89e00640 1 --2- 172.21.15.62:0/4005690705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a88d10 0x560169a9dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.719+0000 7f4e8a800640 1 -- 172.21.15.62:0/4005690705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 msgr2=0x560169a8e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.719+0000 7f4e8a800640 1 --2- 172.21.15.62:0/4005690705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 0x560169a8e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.719+0000 7f4e8a800640 1 -- 172.21.15.62:0/4005690705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a88d10 msgr2=0x560169a9dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.719+0000 7f4e8a800640 1 --2- 172.21.15.62:0/4005690705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a88d10 0x560169a9dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.719+0000 7f4e8a800640 1 -- 172.21.15.62:0/4005690705 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560169a87d40 con 0x560169a92080 2024-04-02T14:45:49.715 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/writing_musb_glue_layer.tmpl 2024-04-02T14:45:49.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.719+0000 7f4e8a800640 1 --2- 172.21.15.62:0/4005690705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 0x560169aaa070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e80003dd0 tx=0x7f4e8001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a98bdac5938837c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.719+0000 7f4e72600640 1 -- 172.21.15.62:0/4005690705 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8000fda0 con 0x560169a92080 2024-04-02T14:45:49.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.719+0000 7f4e72600640 1 -- 172.21.15.62:0/4005690705 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e800045b0 con 0x560169a92080 2024-04-02T14:45:49.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.719+0000 7f4e72600640 1 -- 172.21.15.62:0/4005690705 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80005900 con 0x560169a92080 2024-04-02T14:45:49.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.720+0000 7f4e8f473740 1 -- 172.21.15.62:0/4005690705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 msgr2=0x560169aaa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.720+0000 7f4e8f473740 1 --2- 172.21.15.62:0/4005690705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 0x560169aaa070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e80003dd0 tx=0x7f4e8001b000 comp rx=0 tx=0).stop 2024-04-02T14:45:49.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.720+0000 7f4e8f473740 1 -- 172.21.15.62:0/4005690705 shutdown_connections 2024-04-02T14:45:49.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.720+0000 7f4e8f473740 1 --2- 172.21.15.62:0/4005690705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a92080 0x560169aaa070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.720+0000 7f4e8f473740 1 --2- 172.21.15.62:0/4005690705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a899f0 0x560169a8e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.720+0000 7f4e8f473740 1 --2- 172.21.15.62:0/4005690705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a88d10 0x560169a9dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.720+0000 7f4e8f473740 1 -- 172.21.15.62:0/4005690705 >> 172.21.15.62:0/4005690705 conn(0x560169a4fa50 msgr2=0x560169a85440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.720+0000 7f4e8f473740 1 -- 172.21.15.62:0/4005690705 shutdown_connections 2024-04-02T14:45:49.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.720+0000 7f4e8f473740 1 -- 172.21.15.62:0/4005690705 wait complete. 2024-04-02T14:45:49.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.721+0000 7f4e8f473740 1 -- start start 2024-04-02T14:45:49.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.721+0000 7f4e8f473740 1 Processor -- start 2024-04-02T14:45:49.717 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/writing_usb_driver.tmpl 2024-04-02T14:45:49.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.721+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a88d10 0x560169a9a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.721+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a899f0 0x560169a955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.721+0000 7f4e8f473740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a92080 0x560169a95af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.721+0000 7f4e8f473740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841b0dd0 con 0x560169a88d10 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.721+0000 7f4e8f473740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841b0700 con 0x560169a92080 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.721+0000 7f4e8f473740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841ad940 con 0x560169a899f0 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.721+0000 7f4e8f473740 10 client.0 _ll_register_callbacks cb 0x7f4e74009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e8a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a92080 0x560169a95af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e8a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a92080 0x560169a95af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48796/0 (socket says 172.21.15.62:48796) 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e8a800640 1 -- 172.21.15.62:0/798247979 learned_addr learned my addr 172.21.15.62:0/798247979 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e89400640 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a899f0 0x560169a955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e89e00640 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a88d10 0x560169a9a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e8a800640 1 -- 172.21.15.62:0/798247979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a899f0 msgr2=0x560169a955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e8a800640 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a899f0 0x560169a955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e8a800640 1 -- 172.21.15.62:0/798247979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a88d10 msgr2=0x560169a9a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e8a800640 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a88d10 0x560169a9a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e8a800640 1 -- 172.21.15.62:0/798247979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560169a96030 con 0x560169a92080 2024-04-02T14:45:49.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e89400640 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a899f0 0x560169a955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:49.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e8a800640 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a92080 0x560169a95af0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4e8002b040 tx=0x7f4e80004e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cd5c261d8440e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.722+0000 7f4e67400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:49.720 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/DocBook/z8530book.tmpl 2024-04-02T14:45:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.723+0000 7f4e67400640 1 -- 172.21.15.62:0/798247979 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80036030 con 0x560169a92080 2024-04-02T14:45:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.723+0000 7f4e67400640 1 -- 172.21.15.62:0/798247979 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f4e8002f040 con 0x560169a92080 2024-04-02T14:45:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.723+0000 7f4e8f473740 1 -- 172.21.15.62:0/798247979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e800045b0 con 0x560169a92080 2024-04-02T14:45:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.723+0000 7f4e67400640 1 -- 172.21.15.62:0/798247979 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e80040050 con 0x560169a92080 2024-04-02T14:45:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.723+0000 7f4e8f473740 10 client.26012 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.723+0000 7f4e8f473740 1 -- 172.21.15.62:0/798247979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560169b085d0 con 0x560169a92080 2024-04-02T14:45:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.723+0000 7f4e67400640 1 -- 172.21.15.62:0/798247979 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e800045b0 con 0x560169a92080 2024-04-02T14:45:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.723+0000 7f4e67400640 1 -- 172.21.15.62:0/798247979 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4e80004f90 con 0x560169a92080 2024-04-02T14:45:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.724+0000 7f4e8f473740 10 client.26012 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.724+0000 7f4e8f473740 10 client.26012 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:45:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.724+0000 7f4e8f473740 20 client.26012 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.724+0000 7f4e8f473740 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560169aa23b0 0x560169b0ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.724+0000 7f4e8f473740 4 client.26012 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:45:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.724+0000 7f4e8f473740 1 -- 172.21.15.62:0/798247979 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601699fac30 con 0x560169aa23b0 2024-04-02T14:45:49.722 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/EDID/ 2024-04-02T14:45:49.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.724+0000 7f4e89e00640 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560169aa23b0 0x560169b0ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e89e00640 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560169aa23b0 0x560169b0ad50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d5d89b9ca8417c94 server_cookie=b4db67fc0da4c98c in_seq=0 out_seq=0 2024-04-02T14:45:49.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e67400640 10 client.26012 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:45:49.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e67400640 1 -- 172.21.15.62:0/798247979 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601699fac30 con 0x560169aa23b0 2024-04-02T14:45:49.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e67400640 10 client.26012 handle_command_reply: tid=0 2024-04-02T14:45:49.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 1 client.26012 shutdown 2024-04-02T14:45:49.724 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/EDID/1024x768.S 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 1 -- 172.21.15.62:0/798247979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a92080 msgr2=0x560169a95af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a92080 0x560169a95af0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4e8002b040 tx=0x7f4e80004e20 comp rx=0 tx=0).stop 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 1 -- 172.21.15.62:0/798247979 shutdown_connections 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x560169aa23b0 0x560169b0ad50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560169a92080 0x560169a95af0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560169a899f0 0x560169a955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 1 --2- 172.21.15.62:0/798247979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560169a88d10 0x560169a9a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e67400640 0 client.26012 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 1 -- 172.21.15.62:0/798247979 >> 172.21.15.62:0/798247979 conn(0x560169a4fa50 msgr2=0x560169a82010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 1 -- 172.21.15.62:0/798247979 shutdown_connections 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 1 -- 172.21.15.62:0/798247979 wait complete. 2024-04-02T14:45:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.725+0000 7f4e8f473740 20 client.26012 trim_cache size 0 max 16384 2024-04-02T14:45:49.726 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:45:49.726 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/EDID/1280x1024.S 2024-04-02T14:45:49.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 -- 172.21.15.62:0/4164113776 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f4e6c0a3d20 msgr2=0x7f4e6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f4e6c0a3d20 0x7f4e6c0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009fd0 tx=0x7f4e80009500 comp rx=0 tx=0).stop 2024-04-02T14:45:49.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 -- 172.21.15.62:0/4164113776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840747d0 msgr2=0x7f4e8419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840747d0 0x7f4e8419abb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400e9f0 tx=0x7f4e7400edf0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 -- 172.21.15.62:0/4164113776 shutdown_connections 2024-04-02T14:45:49.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f4e6c0a3d20 0x7f4e6c0a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8410e350 0x7f4e8419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e840747d0 0x7f4e8419abb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 --2- 172.21.15.62:0/4164113776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84073eb0 0x7f4e8419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 -- 172.21.15.62:0/4164113776 >> 172.21.15.62:0/4164113776 conn(0x7f4e84078f00 msgr2=0x7f4e840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 -- 172.21.15.62:0/4164113776 shutdown_connections 2024-04-02T14:45:49.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.726+0000 7f4e8c800640 1 -- 172.21.15.62:0/4164113776 wait complete. 2024-04-02T14:45:49.728 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/EDID/1600x1200.S 2024-04-02T14:45:49.728 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/EDID/1680x1050.S 2024-04-02T14:45:49.729 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/EDID/1920x1080.S 2024-04-02T14:45:49.729 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/EDID/800x600.S 2024-04-02T14:45:49.729 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/EDID/HOWTO.txt 2024-04-02T14:45:49.729 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/EDID/Makefile 2024-04-02T14:45:49.729 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/EDID/edid.S 2024-04-02T14:45:49.730 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/EDID/hex 2024-04-02T14:45:49.731 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/HOWTO 2024-04-02T14:45:49.733 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/IPMI.txt 2024-04-02T14:45:49.734 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/IRQ-affinity.txt 2024-04-02T14:45:49.735 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/IRQ-domain.txt 2024-04-02T14:45:49.737 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/IRQ.txt 2024-04-02T14:45:49.738 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/Intel-IOMMU.txt 2024-04-02T14:45:49.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:49 smithi062 ceph-mon[32116]: pgmap v57: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-04-02T14:45:49.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:49 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1958862119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:45:49.739 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/Makefile 2024-04-02T14:45:49.741 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ManagementStyle 2024-04-02T14:45:49.743 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:45:49.743 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/PCI/ 2024-04-02T14:45:49.744 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/PCI/00-INDEX 2024-04-02T14:45:49.745 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/PCI/MSI-HOWTO.txt 2024-04-02T14:45:49.748 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/PCI/PCIEBUS-HOWTO.txt 2024-04-02T14:45:49.749 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/PCI/pci-error-recovery.txt 2024-04-02T14:45:49.749 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/PCI/pci-iov-howto.txt 2024-04-02T14:45:49.751 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/PCI/pci.txt 2024-04-02T14:45:49.753 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/PCI/pcieaer-howto.txt 2024-04-02T14:45:49.754 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/ 2024-04-02T14:45:49.756 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/00-INDEX 2024-04-02T14:45:49.757 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/NMI-RCU.txt 2024-04-02T14:45:49.758 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/RTFP.txt 2024-04-02T14:45:49.760 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/UP.txt 2024-04-02T14:45:49.761 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/arrayRCU.txt 2024-04-02T14:45:49.763 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/checklist.txt 2024-04-02T14:45:49.764 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/listRCU.txt 2024-04-02T14:45:49.766 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/lockdep-splat.txt 2024-04-02T14:45:49.767 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/lockdep.txt 2024-04-02T14:45:49.768 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/rcu.txt 2024-04-02T14:45:49.769 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/rcu_dereference.txt 2024-04-02T14:45:49.771 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/rcubarrier.txt 2024-04-02T14:45:49.772 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/rculist_nulls.txt 2024-04-02T14:45:49.774 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/rcuref.txt 2024-04-02T14:45:49.775 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/stallwarn.txt 2024-04-02T14:45:49.777 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/torture.txt 2024-04-02T14:45:49.779 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/trace.txt 2024-04-02T14:45:49.779 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/RCU/whatisRCU.txt 2024-04-02T14:45:49.781 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/SAK.txt 2024-04-02T14:45:49.782 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/SM501.txt 2024-04-02T14:45:49.783 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/SecurityBugs 2024-04-02T14:45:49.785 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/SubmitChecklist 2024-04-02T14:45:49.786 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/SubmittingDrivers 2024-04-02T14:45:49.787 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/SubmittingPatches 2024-04-02T14:45:49.789 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/VGA-softcursor.txt 2024-04-02T14:45:49.790 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/accounting/ 2024-04-02T14:45:49.791 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/accounting/.gitignore 2024-04-02T14:45:49.792 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/accounting/Makefile 2024-04-02T14:45:49.793 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/accounting/cgroupstats.txt 2024-04-02T14:45:49.794 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/accounting/delay-accounting.txt 2024-04-02T14:45:49.795 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/accounting/getdelays.c 2024-04-02T14:45:49.796 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/accounting/taskstats-struct.txt 2024-04-02T14:45:49.798 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/accounting/taskstats.txt 2024-04-02T14:45:49.799 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/ 2024-04-02T14:45:49.800 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/apei/ 2024-04-02T14:45:49.801 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/apei/einj.txt 2024-04-02T14:45:49.802 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/apei/output_format.txt 2024-04-02T14:45:49.803 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/debug.txt 2024-04-02T14:45:49.804 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/dsdt-override.txt 2024-04-02T14:45:49.805 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/enumeration.txt 2024-04-02T14:45:49.806 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/gpio-properties.txt 2024-04-02T14:45:49.807 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/initrd_table_override.txt 2024-04-02T14:45:49.809 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/method-customizing.txt 2024-04-02T14:45:49.810 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/method-tracing.txt 2024-04-02T14:45:49.812 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/namespace.txt 2024-04-02T14:45:49.814 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/scan_handlers.txt 2024-04-02T14:45:49.815 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/acpi/video_extension.txt 2024-04-02T14:45:49.817 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/aoe/ 2024-04-02T14:45:49.819 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/aoe/aoe.txt 2024-04-02T14:45:49.821 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/aoe/autoload.sh 2024-04-02T14:45:49.822 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/aoe/status.sh 2024-04-02T14:45:49.823 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/aoe/todo.txt 2024-04-02T14:45:49.825 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/aoe/udev-install.sh 2024-04-02T14:45:49.827 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/aoe/udev.txt 2024-04-02T14:45:49.828 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/applying-patches.txt 2024-04-02T14:45:49.830 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/ 2024-04-02T14:45:49.831 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/00-INDEX 2024-04-02T14:45:49.833 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Atmel/ 2024-04-02T14:45:49.834 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Atmel/README 2024-04-02T14:45:49.835 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Booting 2024-04-02T14:45:49.837 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/CCN.txt 2024-04-02T14:45:49.838 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/IXP4xx 2024-04-02T14:45:49.840 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Interrupts 2024-04-02T14:45:49.842 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Makefile 2024-04-02T14:45:49.843 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Marvell/ 2024-04-02T14:45:49.844 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Marvell/README 2024-04-02T14:45:49.846 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Netwinder 2024-04-02T14:45:49.848 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/OMAP/ 2024-04-02T14:45:49.849 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/OMAP/DSS 2024-04-02T14:45:49.851 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/OMAP/omap_pm 2024-04-02T14:45:49.853 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Porting 2024-04-02T14:45:49.854 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/README 2024-04-02T14:45:49.855 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/ 2024-04-02T14:45:49.857 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/ADSBitsy 2024-04-02T14:45:49.858 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/Assabet 2024-04-02T14:45:49.860 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/Brutus 2024-04-02T14:45:49.861 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/CERF 2024-04-02T14:45:49.862 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/FreeBird 2024-04-02T14:45:49.865 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsClient 2024-04-02T14:45:49.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:49 smithi113 ceph-mon[36135]: pgmap v57: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-04-02T14:45:49.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:49 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1958862119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:45:49.866 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsMaster 2024-04-02T14:45:49.869 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/HUW_WEBPANEL 2024-04-02T14:45:49.870 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/Itsy 2024-04-02T14:45:49.871 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/LART 2024-04-02T14:45:49.872 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/PLEB 2024-04-02T14:45:49.873 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/Pangolin 2024-04-02T14:45:49.874 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/Tifon 2024-04-02T14:45:49.876 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/Victor 2024-04-02T14:45:49.877 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/Yopy 2024-04-02T14:45:49.878 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/empeg 2024-04-02T14:45:49.879 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/nanoEngine 2024-04-02T14:45:49.881 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SA1100/serial_UART 2024-04-02T14:45:49.882 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/ 2024-04-02T14:45:49.883 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/.gitignore 2024-04-02T14:45:49.885 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/Makefile 2024-04-02T14:45:49.886 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/vrl4.c 2024-04-02T14:45:49.887 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-mmcif.txt 2024-04-02T14:45:49.888 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-sdhi.txt 2024-04-02T14:45:49.890 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SPEAr/ 2024-04-02T14:45:49.891 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/SPEAr/overview.txt 2024-04-02T14:45:49.892 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/ 2024-04-02T14:45:49.893 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/CPUfreq.txt 2024-04-02T14:45:49.895 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/EB2410ITX.txt 2024-04-02T14:45:49.896 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/GPIO.txt 2024-04-02T14:45:49.897 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/H1940.txt 2024-04-02T14:45:49.899 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/NAND.txt 2024-04-02T14:45:49.900 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Overview.txt 2024-04-02T14:45:49.901 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2412.txt 2024-04-02T14:45:49.902 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2413.txt 2024-04-02T14:45:49.904 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/SMDK2440.txt 2024-04-02T14:45:49.905 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Suspend.txt 2024-04-02T14:45:49.907 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/USB-Host.txt 2024-04-02T14:45:49.909 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung/ 2024-04-02T14:45:49.910 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung/GPIO.txt 2024-04-02T14:45:49.911 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung/Overview.txt 2024-04-02T14:45:49.912 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Samsung/clksrc-change-registers.awk 2024-04-02T14:45:49.914 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/Setup 2024-04-02T14:45:49.916 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/VFP/ 2024-04-02T14:45:49.917 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/VFP/release-notes.txt 2024-04-02T14:45:49.918 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/cluster-pm-race-avoidance.txt 2024-04-02T14:45:49.920 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/firmware.txt 2024-04-02T14:45:49.922 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/kernel_mode_neon.txt 2024-04-02T14:45:49.924 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/kernel_user_helpers.txt 2024-04-02T14:45:49.925 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/mem_alignment 2024-04-02T14:45:49.927 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/memory.txt 2024-04-02T14:45:49.928 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/msm/ 2024-04-02T14:45:49.930 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/msm/gpiomux.txt 2024-04-02T14:45:49.931 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/nwfpe/ 2024-04-02T14:45:49.933 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/nwfpe/NOTES 2024-04-02T14:45:49.934 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/nwfpe/README 2024-04-02T14:45:49.936 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/nwfpe/README.FPE 2024-04-02T14:45:49.938 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/nwfpe/TODO 2024-04-02T14:45:49.939 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/pxa/ 2024-04-02T14:45:49.940 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/pxa/mfp.txt 2024-04-02T14:45:49.942 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/sti/ 2024-04-02T14:45:49.943 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/sti/overview.txt 2024-04-02T14:45:49.944 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/sti/stih407-overview.txt 2024-04-02T14:45:49.945 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/sti/stih415-overview.txt 2024-04-02T14:45:49.947 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/sti/stih416-overview.txt 2024-04-02T14:45:49.948 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/sti/stih418-overview.txt 2024-04-02T14:45:49.949 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/sunxi/ 2024-04-02T14:45:49.951 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/sunxi/README 2024-04-02T14:45:49.952 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/sunxi/clocks.txt 2024-04-02T14:45:49.953 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/swp_emulation 2024-04-02T14:45:49.955 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/tcm.txt 2024-04-02T14:45:49.958 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/uefi.txt 2024-04-02T14:45:49.959 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm/vlocks.txt 2024-04-02T14:45:49.961 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm64/ 2024-04-02T14:45:49.962 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm64/booting.txt 2024-04-02T14:45:49.963 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm64/legacy_instructions.txt 2024-04-02T14:45:49.965 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm64/memory.txt 2024-04-02T14:45:49.966 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/arm64/tagged-pointers.txt 2024-04-02T14:45:49.968 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/assoc_array.txt 2024-04-02T14:45:49.970 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/atomic_ops.txt 2024-04-02T14:45:49.972 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/auxdisplay/ 2024-04-02T14:45:49.973 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/auxdisplay/.gitignore 2024-04-02T14:45:49.974 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/auxdisplay/Makefile 2024-04-02T14:45:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.980+0000 7f9e32400640 1 -- 172.21.15.62:0/1045904068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e2c102450 msgr2=0x7f9e2c10e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.980+0000 7f9e32400640 1 --2- 172.21.15.62:0/1045904068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e2c102450 0x7f9e2c10e960 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009ca0 tx=0x7f9e2402c0f0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.980+0000 7f9e32400640 1 -- 172.21.15.62:0/1045904068 shutdown_connections 2024-04-02T14:45:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.980+0000 7f9e32400640 1 --2- 172.21.15.62:0/1045904068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e2c102450 0x7f9e2c10e960 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.980+0000 7f9e32400640 1 --2- 172.21.15.62:0/1045904068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e2c101a90 0x7f9e2c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.980+0000 7f9e32400640 1 --2- 172.21.15.62:0/1045904068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c107aa0 0x7f9e2c107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.980+0000 7f9e32400640 1 -- 172.21.15.62:0/1045904068 >> 172.21.15.62:0/1045904068 conn(0x7f9e2c078f00 msgr2=0x7f9e2c079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.980+0000 7f9e32400640 1 -- 172.21.15.62:0/1045904068 shutdown_connections 2024-04-02T14:45:49.976 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b 2024-04-02T14:45:49.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.980+0000 7f9e32400640 1 -- 172.21.15.62:0/1045904068 wait complete. 2024-04-02T14:45:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.981+0000 7f9e32400640 1 Processor -- start 2024-04-02T14:45:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.981+0000 7f9e32400640 1 -- start start 2024-04-02T14:45:49.977 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b-example.c 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e32400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e2c101a90 0x7f9e2c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e32400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c102450 0x7f9e2c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e32400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e2c107aa0 0x7f9e2c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e32400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c111fd0 con 0x7f9e2c102450 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e32400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c111e50 con 0x7f9e2c101a90 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e32400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c112150 con 0x7f9e2c107aa0 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e2b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e2c101a90 0x7f9e2c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e2b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e2c101a90 0x7f9e2c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48816/0 (socket says 172.21.15.62:48816) 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e2b400640 1 -- 172.21.15.62:0/4192223327 learned_addr learned my addr 172.21.15.62:0/4192223327 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e2aa00640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c102450 0x7f9e2c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e2be00640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e2c107aa0 0x7f9e2c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e2b400640 1 -- 172.21.15.62:0/4192223327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e2c107aa0 msgr2=0x7f9e2c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e2b400640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e2c107aa0 0x7f9e2c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e2b400640 1 -- 172.21.15.62:0/4192223327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c102450 msgr2=0x7f9e2c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e2b400640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c102450 0x7f9e2c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.982+0000 7f9e2b400640 1 -- 172.21.15.62:0/4192223327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c19f640 con 0x7f9e2c101a90 2024-04-02T14:45:49.980 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/auxdisplay/ks0108 2024-04-02T14:45:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.983+0000 7f9e2b400640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e2c101a90 0x7f9e2c19a650 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00e9f0 tx=0x7f9e1c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.983+0000 7f9e23e00640 1 -- 172.21.15.62:0/4192223327 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00cd80 con 0x7f9e2c101a90 2024-04-02T14:45:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.983+0000 7f9e23e00640 1 -- 172.21.15.62:0/4192223327 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9e1c010930 con 0x7f9e2c101a90 2024-04-02T14:45:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.983+0000 7f9e32400640 1 -- 172.21.15.62:0/4192223327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c19f930 con 0x7f9e2c101a90 2024-04-02T14:45:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.983+0000 7f9e23e00640 1 -- 172.21.15.62:0/4192223327 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c018b00 con 0x7f9e2c101a90 2024-04-02T14:45:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.983+0000 7f9e32400640 1 -- 172.21.15.62:0/4192223327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c1a71c0 con 0x7f9e2c101a90 2024-04-02T14:45:49.981 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/backlight/ 2024-04-02T14:45:49.982 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/backlight/lp855x-driver.txt 2024-04-02T14:45:49.982 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/bad_memory.txt 2024-04-02T14:45:49.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.987+0000 7f9e23e00640 1 -- 172.21.15.62:0/4192223327 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9e1c018ca0 con 0x7f9e2c101a90 2024-04-02T14:45:49.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.987+0000 7f9e351d1740 1 Processor -- start 2024-04-02T14:45:49.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.987+0000 7f9e351d1740 1 -- start start 2024-04-02T14:45:49.983 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/basic_profiling.txt 2024-04-02T14:45:49.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.987+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd1219f0 0x5567dd120740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.987+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd120d10 0x5567dd121190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.987+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd12c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.987+0000 7f9e351d1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5567dd061400 con 0x5567dd1219f0 2024-04-02T14:45:49.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.987+0000 7f9e351d1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5567dd045f10 con 0x5567dd12a080 2024-04-02T14:45:49.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.987+0000 7f9e351d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5567dd0856f0 con 0x5567dd120d10 2024-04-02T14:45:49.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.987+0000 7f9e2aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd12c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e2aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd12c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48832/0 (socket says 172.21.15.62:48832) 2024-04-02T14:45:49.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e2aa00640 1 -- 172.21.15.62:0/889649109 learned_addr learned my addr 172.21.15.62:0/889649109 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e2be00640 1 --2- 172.21.15.62:0/889649109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd1219f0 0x5567dd120740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e2b400640 1 --2- 172.21.15.62:0/889649109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd120d10 0x5567dd121190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e2aa00640 1 -- 172.21.15.62:0/889649109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd120d10 msgr2=0x5567dd121190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e2aa00640 1 --2- 172.21.15.62:0/889649109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd120d10 0x5567dd121190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e2aa00640 1 -- 172.21.15.62:0/889649109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd1219f0 msgr2=0x5567dd120740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e2aa00640 1 --2- 172.21.15.62:0/889649109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd1219f0 0x5567dd120740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e2aa00640 1 -- 172.21.15.62:0/889649109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567dcd993f0 con 0x5567dd12a080 2024-04-02T14:45:49.986 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/bcache.txt 2024-04-02T14:45:49.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e2aa00640 1 --2- 172.21.15.62:0/889649109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd12c470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800b740 tx=0x7f9e1800bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff31bcd63840a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e23e00640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9e100a3da0 0x7f9e100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e20c00640 1 -- 172.21.15.62:0/889649109 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18003dd0 con 0x5567dd12a080 2024-04-02T14:45:49.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e20c00640 1 -- 172.21.15.62:0/889649109 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9e18015070 con 0x5567dd12a080 2024-04-02T14:45:49.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.988+0000 7f9e20c00640 1 -- 172.21.15.62:0/889649109 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18011440 con 0x5567dd12a080 2024-04-02T14:45:49.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.989+0000 7f9e23e00640 1 -- 172.21.15.62:0/4192223327 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f9e1c0d0aa0 con 0x7f9e2c101a90 2024-04-02T14:45:49.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.989+0000 7f9e2be00640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9e100a3da0 0x7f9e100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.989+0000 7f9e351d1740 1 -- 172.21.15.62:0/889649109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 msgr2=0x5567dd12c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.989+0000 7f9e351d1740 1 --2- 172.21.15.62:0/889649109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd12c470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800b740 tx=0x7f9e1800bc00 comp rx=0 tx=0).stop 2024-04-02T14:45:49.988 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/binfmt_misc.txt 2024-04-02T14:45:49.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.990+0000 7f9e351d1740 1 -- 172.21.15.62:0/889649109 shutdown_connections 2024-04-02T14:45:49.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.990+0000 7f9e351d1740 1 --2- 172.21.15.62:0/889649109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd12c470 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.990+0000 7f9e351d1740 1 --2- 172.21.15.62:0/889649109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd120d10 0x5567dd121190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.990+0000 7f9e351d1740 1 --2- 172.21.15.62:0/889649109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd1219f0 0x5567dd120740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.990+0000 7f9e351d1740 1 -- 172.21.15.62:0/889649109 >> 172.21.15.62:0/889649109 conn(0x5567dd0e7a50 msgr2=0x5567dd11b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.990+0000 7f9e2be00640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9e100a3da0 0x7f9e100a6260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e24005e30 tx=0x7f9e24005d80 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.990+0000 7f9e351d1740 1 -- 172.21.15.62:0/889649109 shutdown_connections 2024-04-02T14:45:49.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.990+0000 7f9e351d1740 1 -- 172.21.15.62:0/889649109 wait complete. 2024-04-02T14:45:49.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.990+0000 7f9e351d1740 1 -- start start 2024-04-02T14:45:49.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.990+0000 7f9e351d1740 1 Processor -- start 2024-04-02T14:45:49.990 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blackfin/ 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd120d10 0x5567dd126fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd127520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd12a080 0x5567dd127a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e351d1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5567dd0816a0 con 0x5567dd12a080 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e351d1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5567dd02d980 con 0x5567dd120d10 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e351d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5567dd040cb0 con 0x5567dd1219f0 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e351d1740 10 client.0 _ll_register_callbacks cb 0x5567dd0406a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e2b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd120d10 0x5567dd126fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e2aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd127520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e2aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd127520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:42180/0 (socket says 172.21.15.62:42180) 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e2aa00640 1 -- 172.21.15.62:0/5569160 learned_addr learned my addr 172.21.15.62:0/5569160 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:49.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.991+0000 7f9e2be00640 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd12a080 0x5567dd127a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e2aa00640 1 -- 172.21.15.62:0/5569160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd120d10 msgr2=0x5567dd126fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e2aa00640 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd120d10 0x5567dd126fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e2aa00640 1 -- 172.21.15.62:0/5569160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd12a080 msgr2=0x5567dd127a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:49.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e2aa00640 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd12a080 0x5567dd127a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e2aa00640 1 -- 172.21.15.62:0/5569160 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567dce637d0 con 0x5567dd1219f0 2024-04-02T14:45:49.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e2b400640 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd120d10 0x5567dd126fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:49.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e2be00640 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd12a080 0x5567dd127a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:49.993 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blackfin/00-INDEX 2024-04-02T14:45:49.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e2aa00640 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd127520 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e18004ba0 tx=0x7f9e18005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9c4dc674fed5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:49.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e15600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:45:49.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e15600640 1 -- 172.21.15.62:0/5569160 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18003ea0 con 0x5567dd1219f0 2024-04-02T14:45:49.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e351d1740 1 -- 172.21.15.62:0/5569160 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e18015070 con 0x5567dd1219f0 2024-04-02T14:45:49.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.992+0000 7f9e15600640 1 -- 172.21.15.62:0/5569160 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f9e18004d00 con 0x5567dd1219f0 2024-04-02T14:45:49.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.993+0000 7f9e15600640 1 -- 172.21.15.62:0/5569160 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e18004f00 con 0x5567dd1219f0 2024-04-02T14:45:49.995 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blackfin/Makefile 2024-04-02T14:45:49.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.993+0000 7f9e351d1740 10 client.25996 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:49.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.993+0000 7f9e351d1740 1 -- 172.21.15.62:0/5569160 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567dd138ee0 con 0x5567dd1219f0 2024-04-02T14:45:49.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.993+0000 7f9e15600640 1 -- 172.21.15.62:0/5569160 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9e18015070 con 0x5567dd1219f0 2024-04-02T14:45:49.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.993+0000 7f9e15600640 1 -- 172.21.15.62:0/5569160 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9e18011af0 con 0x5567dd1219f0 2024-04-02T14:45:49.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.993+0000 7f9e351d1740 10 client.25996 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:49.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.993+0000 7f9e351d1740 10 client.25996 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:45:49.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.993+0000 7f9e351d1740 20 client.25996 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:49.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.993+0000 7f9e351d1740 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5567dd13a290 0x5567dd13c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:49.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.993+0000 7f9e351d1740 4 client.25996 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:49.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.993+0000 7f9e351d1740 1 -- 172.21.15.62:0/5569160 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567dd045040 con 0x5567dd13a290 2024-04-02T14:45:49.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.994+0000 7f9e2b400640 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5567dd13a290 0x5567dd13c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:49.997 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blackfin/bfin-gpio-notes.txt 2024-04-02T14:45:49.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.994+0000 7f9e2b400640 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5567dd13a290 0x5567dd13c680 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=406902d6dacfde2b server_cookie=dec9302159cd483 in_seq=0 out_seq=0 2024-04-02T14:45:49.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.994+0000 7f9e15600640 10 client.25996 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:45:49.997 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blackfin/bfin-spi-notes.txt 2024-04-02T14:45:49.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e15600640 1 -- 172.21.15.62:0/5569160 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567dd045040 con 0x5567dd13a290 2024-04-02T14:45:49.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e15600640 10 client.25996 handle_command_reply: tid=0 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e351d1740 1 client.25996 shutdown 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e351d1740 1 -- 172.21.15.62:0/5569160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 msgr2=0x5567dd127520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e351d1740 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd127520 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e18004ba0 tx=0x7f9e18005c70 comp rx=0 tx=0).stop 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e351d1740 1 -- 172.21.15.62:0/5569160 shutdown_connections 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e351d1740 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5567dd13a290 0x5567dd13c680 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e351d1740 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd12a080 0x5567dd127a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e351d1740 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd127520 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e351d1740 1 --2- 172.21.15.62:0/5569160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd120d10 0x5567dd126fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e15600640 0 client.25996 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.995+0000 7f9e351d1740 1 -- 172.21.15.62:0/5569160 >> 172.21.15.62:0/5569160 conn(0x5567dd0e7a50 msgr2=0x5567dd11d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.996+0000 7f9e351d1740 1 -- 172.21.15.62:0/5569160 shutdown_connections 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.996+0000 7f9e351d1740 1 -- 172.21.15.62:0/5569160 wait complete. 2024-04-02T14:45:49.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:49.996+0000 7f9e351d1740 20 client.25996 trim_cache size 0 max 16384 2024-04-02T14:45:50.000 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blackfin/gptimers-example.c 2024-04-02T14:45:50.000 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/ 2024-04-02T14:45:50.000 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/00-INDEX 2024-04-02T14:45:50.001 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/biodoc.txt 2024-04-02T14:45:50.001 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/biovecs.txt 2024-04-02T14:45:50.001 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/capability.txt 2024-04-02T14:45:50.001 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/cfq-iosched.txt 2024-04-02T14:45:50.001 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/cmdline-partition.txt 2024-04-02T14:45:50.001 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/data-integrity.txt 2024-04-02T14:45:50.001 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/deadline-iosched.txt 2024-04-02T14:45:50.002 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/ioprio.txt 2024-04-02T14:45:50.003 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/null_blk.txt 2024-04-02T14:45:50.004 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/queue-sysfs.txt 2024-04-02T14:45:50.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:49 smithi204 ceph-mon[35948]: pgmap v57: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-04-02T14:45:50.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:49 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1958862119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:45:50.005 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/request.txt 2024-04-02T14:45:50.006 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/stat.txt 2024-04-02T14:45:50.007 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/switching-sched.txt 2024-04-02T14:45:50.008 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/block/writeback_cache_control.txt 2024-04-02T14:45:50.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.013+0000 7f9e351d1740 1 Processor -- start 2024-04-02T14:45:50.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e351d1740 1 -- start start 2024-04-02T14:45:50.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 0x5567dd129710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd135b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd1276b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e351d1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1afa30 con 0x5567dd120d10 2024-04-02T14:45:50.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e351d1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1b1570 con 0x5567dd12a080 2024-04-02T14:45:50.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e351d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1b0100 con 0x5567dd1219f0 2024-04-02T14:45:50.010 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/ 2024-04-02T14:45:50.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e2be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd1276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e2b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 0x5567dd129710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e2be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd1276b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48852/0 (socket says 172.21.15.62:48852) 2024-04-02T14:45:50.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e2be00640 1 -- 172.21.15.62:0/2438913921 learned_addr learned my addr 172.21.15.62:0/2438913921 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.014+0000 7f9e2aa00640 1 --2- 172.21.15.62:0/2438913921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd135b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.015+0000 7f9e2be00640 1 -- 172.21.15.62:0/2438913921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 msgr2=0x5567dd135b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.015+0000 7f9e2be00640 1 --2- 172.21.15.62:0/2438913921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd135b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.015+0000 7f9e2be00640 1 -- 172.21.15.62:0/2438913921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 msgr2=0x5567dd129710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.015+0000 7f9e2be00640 1 --2- 172.21.15.62:0/2438913921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 0x5567dd129710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.015+0000 7f9e2be00640 1 -- 172.21.15.62:0/2438913921 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567dd11fd40 con 0x5567dd12a080 2024-04-02T14:45:50.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.015+0000 7f9e2be00640 1 --2- 172.21.15.62:0/2438913921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd1276b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009ca0 tx=0x7f9e24040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85cc593abebd0f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.012 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/00-INDEX 2024-04-02T14:45:50.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.015+0000 7f9e17e00640 1 -- 172.21.15.62:0/2438913921 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e24052070 con 0x5567dd12a080 2024-04-02T14:45:50.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.016+0000 7f9e17e00640 1 -- 172.21.15.62:0/2438913921 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9e2403a040 con 0x5567dd12a080 2024-04-02T14:45:50.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.016+0000 7f9e17e00640 1 -- 172.21.15.62:0/2438913921 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e24039040 con 0x5567dd12a080 2024-04-02T14:45:50.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.016+0000 7f9e351d1740 1 -- 172.21.15.62:0/2438913921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 msgr2=0x5567dd1276b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.016+0000 7f9e351d1740 1 --2- 172.21.15.62:0/2438913921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd1276b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009ca0 tx=0x7f9e24040000 comp rx=0 tx=0).stop 2024-04-02T14:45:50.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.016+0000 7f9e351d1740 1 -- 172.21.15.62:0/2438913921 shutdown_connections 2024-04-02T14:45:50.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.016+0000 7f9e351d1740 1 --2- 172.21.15.62:0/2438913921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd1276b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.016+0000 7f9e351d1740 1 --2- 172.21.15.62:0/2438913921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd135b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.016+0000 7f9e351d1740 1 --2- 172.21.15.62:0/2438913921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 0x5567dd129710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.016+0000 7f9e351d1740 1 -- 172.21.15.62:0/2438913921 >> 172.21.15.62:0/2438913921 conn(0x5567dd0e7a50 msgr2=0x5567dd126fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.013 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/README.DAC960 2024-04-02T14:45:50.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.016+0000 7f9e351d1740 1 -- 172.21.15.62:0/2438913921 shutdown_connections 2024-04-02T14:45:50.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.016+0000 7f9e351d1740 1 -- 172.21.15.62:0/2438913921 wait complete. 2024-04-02T14:45:50.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.017+0000 7f9e351d1740 1 -- start start 2024-04-02T14:45:50.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.017+0000 7f9e351d1740 1 Processor -- start 2024-04-02T14:45:50.014 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/cciss.txt 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 0x5567dd132540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd1242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e351d1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd124900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e351d1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1b0ea0 con 0x5567dd120d10 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e351d1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1b07d0 con 0x5567dd12a080 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e351d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1ada10 con 0x5567dd1219f0 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e351d1740 10 client.0 _ll_register_callbacks cb 0x7f9e18004310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e2b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 0x5567dd132540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e2b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 0x5567dd132540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35608/0 (socket says 172.21.15.62:35608) 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e2b400640 1 -- 172.21.15.62:0/446501840 learned_addr learned my addr 172.21.15.62:0/446501840 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e2be00640 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd124900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e2aa00640 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd1242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e2be00640 1 -- 172.21.15.62:0/446501840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 msgr2=0x5567dd1242d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e2be00640 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd1242d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e2be00640 1 -- 172.21.15.62:0/446501840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 msgr2=0x5567dd132540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e2be00640 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 0x5567dd132540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e2be00640 1 -- 172.21.15.62:0/446501840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567dd12ab20 con 0x5567dd12a080 2024-04-02T14:45:50.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.018+0000 7f9e2b400640 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 0x5567dd132540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:50.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.019+0000 7f9e2be00640 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd124900 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9e24037040 tx=0x7f9e24003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88646236cb489eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.019+0000 7f9e15600640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:50.017 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/cpqarray.txt 2024-04-02T14:45:50.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.019+0000 7f9e15600640 1 -- 172.21.15.62:0/446501840 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2405b070 con 0x5567dd12a080 2024-04-02T14:45:50.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.019+0000 7f9e15600640 1 -- 172.21.15.62:0/446501840 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f9e24004450 con 0x5567dd12a080 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.019+0000 7f9e351d1740 1 -- 172.21.15.62:0/446501840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e2403a040 con 0x5567dd12a080 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.019+0000 7f9e15600640 1 -- 172.21.15.62:0/446501840 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e24052040 con 0x5567dd12a080 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.019+0000 7f9e351d1740 10 client.26030 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.019+0000 7f9e351d1740 1 -- 172.21.15.62:0/446501840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567dd1a08e0 con 0x5567dd12a080 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.019+0000 7f9e15600640 1 -- 172.21.15.62:0/446501840 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9e2403a040 con 0x5567dd12a080 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.020+0000 7f9e15600640 1 -- 172.21.15.62:0/446501840 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9e240604f0 con 0x5567dd12a080 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.020+0000 7f9e351d1740 10 client.26030 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.020+0000 7f9e351d1740 10 client.26030 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.020+0000 7f9e351d1740 20 client.26030 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.020+0000 7f9e351d1740 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5567dd1a0f00 0x5567dd1a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.020+0000 7f9e351d1740 4 client.26030 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:45:50.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.020+0000 7f9e351d1740 1 -- 172.21.15.62:0/446501840 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567dd092c30 con 0x5567dd1a0f00 2024-04-02T14:45:50.020 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/drbd/ 2024-04-02T14:45:50.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.020+0000 7f9e2b400640 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5567dd1a0f00 0x5567dd1a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e2b400640 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5567dd1a0f00 0x5567dd1a32f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a0d4eed173cd65a server_cookie=119d346789427023 in_seq=0 out_seq=0 2024-04-02T14:45:50.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e15600640 10 client.26030 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:45:50.020 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-8.3-data-packets.svg 2024-04-02T14:45:50.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e15600640 1 -- 172.21.15.62:0/446501840 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567dd092c30 con 0x5567dd1a0f00 2024-04-02T14:45:50.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e15600640 10 client.26030 handle_command_reply: tid=0 2024-04-02T14:45:50.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 1 client.26030 shutdown 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 1 -- 172.21.15.62:0/446501840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 msgr2=0x5567dd124900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd124900 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9e24037040 tx=0x7f9e24003c00 comp rx=0 tx=0).stop 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 1 -- 172.21.15.62:0/446501840 shutdown_connections 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x5567dd1a0f00 0x5567dd1a32f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5567dd12a080 0x5567dd124900 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567dd1219f0 0x5567dd1242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 1 --2- 172.21.15.62:0/446501840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567dd120d10 0x5567dd132540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e15600640 0 client.26030 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 1 -- 172.21.15.62:0/446501840 >> 172.21.15.62:0/446501840 conn(0x5567dd0e7a50 msgr2=0x5567dd11a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 1 -- 172.21.15.62:0/446501840 shutdown_connections 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 1 -- 172.21.15.62:0/446501840 wait complete. 2024-04-02T14:45:50.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.021+0000 7f9e351d1740 20 client.26030 trim_cache size 0 max 16384 2024-04-02T14:45:50.023 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:45:50.023 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-data-packets.svg 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 -- 172.21.15.62:0/4192223327 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9e100a3da0 msgr2=0x7f9e100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9e100a3da0 0x7f9e100a6260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e24005e30 tx=0x7f9e24005d80 comp rx=0 tx=0).stop 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 -- 172.21.15.62:0/4192223327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e2c101a90 msgr2=0x7f9e2c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e2c101a90 0x7f9e2c19a650 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00e9f0 tx=0x7f9e1c00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 -- 172.21.15.62:0/4192223327 shutdown_connections 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9e100a3da0 0x7f9e100a6260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e2c107aa0 0x7f9e2c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c102450 0x7f9e2c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 --2- 172.21.15.62:0/4192223327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e2c101a90 0x7f9e2c19a650 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 -- 172.21.15.62:0/4192223327 >> 172.21.15.62:0/4192223327 conn(0x7f9e2c078f00 msgr2=0x7f9e2c10d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 -- 172.21.15.62:0/4192223327 shutdown_connections 2024-04-02T14:45:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.022+0000 7f9e32400640 1 -- 172.21.15.62:0/4192223327 wait complete. 2024-04-02T14:45:50.025 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/drbd/README.txt 2024-04-02T14:45:50.025 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/drbd/conn-states-8.dot 2024-04-02T14:45:50.025 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/drbd/data-structure-v9.txt 2024-04-02T14:45:50.026 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/drbd/disk-states-8.dot 2024-04-02T14:45:50.026 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/drbd/drbd-connection-state-overview.dot 2024-04-02T14:45:50.026 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/drbd/node-states-8.dot 2024-04-02T14:45:50.026 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/floppy.txt 2024-04-02T14:45:50.026 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/mflash.txt 2024-04-02T14:45:50.026 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/nbd.txt 2024-04-02T14:45:50.027 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/paride.txt 2024-04-02T14:45:50.029 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/ramdisk.txt 2024-04-02T14:45:50.030 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/blockdev/zram.txt 2024-04-02T14:45:50.031 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/braille-console.txt 2024-04-02T14:45:50.032 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/bt8xxgpio.txt 2024-04-02T14:45:50.033 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/btmrvl.txt 2024-04-02T14:45:50.035 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:45:50.036 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/bus-devices/ 2024-04-02T14:45:50.036 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt 2024-04-02T14:45:50.041 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/bus-virt-phys-mapping.txt 2024-04-02T14:45:50.041 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cachetlb.txt 2024-04-02T14:45:50.041 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cdrom/ 2024-04-02T14:45:50.044 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cdrom/00-INDEX 2024-04-02T14:45:50.046 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cdrom/Makefile 2024-04-02T14:45:50.047 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cdrom/cdrom-standard.tex 2024-04-02T14:45:50.049 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cdrom/ide-cd 2024-04-02T14:45:50.050 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cdrom/packet-writing.txt 2024-04-02T14:45:50.051 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/ 2024-04-02T14:45:50.053 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/00-INDEX 2024-04-02T14:45:50.054 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/blkio-controller.txt 2024-04-02T14:45:50.055 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/cgroups.txt 2024-04-02T14:45:50.056 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/cpuacct.txt 2024-04-02T14:45:50.057 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/cpusets.txt 2024-04-02T14:45:50.058 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/devices.txt 2024-04-02T14:45:50.060 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/freezer-subsystem.txt 2024-04-02T14:45:50.061 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/hugetlb.txt 2024-04-02T14:45:50.062 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/memcg_test.txt 2024-04-02T14:45:50.063 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/memory.txt 2024-04-02T14:45:50.064 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/net_cls.txt 2024-04-02T14:45:50.066 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/net_prio.txt 2024-04-02T14:45:50.067 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cgroups/unified-hierarchy.txt 2024-04-02T14:45:50.068 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/circular-buffers.txt 2024-04-02T14:45:50.069 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/clk.txt 2024-04-02T14:45:50.071 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/coccinelle.txt 2024-04-02T14:45:50.074 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/connector/ 2024-04-02T14:45:50.074 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/connector/.gitignore 2024-04-02T14:45:50.074 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/connector/Makefile 2024-04-02T14:45:50.075 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/connector/cn_test.c 2024-04-02T14:45:50.076 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/connector/connector.txt 2024-04-02T14:45:50.077 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/connector/ucon.c 2024-04-02T14:45:50.079 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/console/ 2024-04-02T14:45:50.079 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/console/console.txt 2024-04-02T14:45:50.081 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/ 2024-04-02T14:45:50.082 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/amd-powernow.txt 2024-04-02T14:45:50.083 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/boost.txt 2024-04-02T14:45:50.084 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/core.txt 2024-04-02T14:45:50.085 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/cpu-drivers.txt 2024-04-02T14:45:50.087 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-nforce2.txt 2024-04-02T14:45:50.088 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-stats.txt 2024-04-02T14:45:50.089 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/governors.txt 2024-04-02T14:45:50.090 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/index.txt 2024-04-02T14:45:50.091 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/intel-pstate.txt 2024-04-02T14:45:50.095 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/pcc-cpufreq.txt 2024-04-02T14:45:50.096 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-freq/user-guide.txt 2024-04-02T14:45:50.098 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-hotplug.txt 2024-04-02T14:45:50.100 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpu-load.txt 2024-04-02T14:45:50.101 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpuidle/ 2024-04-02T14:45:50.102 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpuidle/core.txt 2024-04-02T14:45:50.103 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpuidle/driver.txt 2024-04-02T14:45:50.104 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpuidle/governor.txt 2024-04-02T14:45:50.105 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cpuidle/sysfs.txt 2024-04-02T14:45:50.106 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cputopology.txt 2024-04-02T14:45:50.108 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/crc32.txt 2024-04-02T14:45:50.110 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cris/ 2024-04-02T14:45:50.111 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/cris/README 2024-04-02T14:45:50.112 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/crypto/ 2024-04-02T14:45:50.114 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/crypto/api-intro.txt 2024-04-02T14:45:50.115 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/crypto/asymmetric-keys.txt 2024-04-02T14:45:50.116 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/crypto/async-tx-api.txt 2024-04-02T14:45:50.117 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/crypto/crypto-API-userspace.txt 2024-04-02T14:45:50.118 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/crypto/descore-readme.txt 2024-04-02T14:45:50.120 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dcdbas.txt 2024-04-02T14:45:50.121 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/debugging-modules.txt 2024-04-02T14:45:50.122 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/debugging-via-ohci1394.txt 2024-04-02T14:45:50.123 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dell_rbu.txt 2024-04-02T14:45:50.125 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/development-process/ 2024-04-02T14:45:50.126 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/development-process/1.Intro 2024-04-02T14:45:50.127 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/development-process/2.Process 2024-04-02T14:45:50.128 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/development-process/3.Early-stage 2024-04-02T14:45:50.129 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/development-process/4.Coding 2024-04-02T14:45:50.131 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/development-process/5.Posting 2024-04-02T14:45:50.132 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/development-process/6.Followthrough 2024-04-02T14:45:50.133 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/development-process/7.AdvancedTopics 2024-04-02T14:45:50.135 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/development-process/8.Conclusion 2024-04-02T14:45:50.136 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/ 2024-04-02T14:45:50.138 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/cache-policies.txt 2024-04-02T14:45:50.139 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/cache.txt 2024-04-02T14:45:50.140 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/delay.txt 2024-04-02T14:45:50.141 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/dm-crypt.txt 2024-04-02T14:45:50.142 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/dm-flakey.txt 2024-04-02T14:45:50.144 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/dm-io.txt 2024-04-02T14:45:50.145 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/dm-log.txt 2024-04-02T14:45:50.146 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/dm-queue-length.txt 2024-04-02T14:45:50.147 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/dm-raid.txt 2024-04-02T14:45:50.149 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/dm-service-time.txt 2024-04-02T14:45:50.150 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/dm-uevent.txt 2024-04-02T14:45:50.152 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/era.txt 2024-04-02T14:45:50.153 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/kcopyd.txt 2024-04-02T14:45:50.154 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/linear.txt 2024-04-02T14:45:50.156 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/persistent-data.txt 2024-04-02T14:45:50.157 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/snapshot.txt 2024-04-02T14:45:50.159 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/statistics.txt 2024-04-02T14:45:50.160 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/striped.txt 2024-04-02T14:45:50.162 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/switch.txt 2024-04-02T14:45:50.163 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/thin-provisioning.txt 2024-04-02T14:45:50.165 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/verity.txt 2024-04-02T14:45:50.166 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/device-mapper/zero.txt 2024-04-02T14:45:50.168 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devices.txt 2024-04-02T14:45:50.172 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/ 2024-04-02T14:45:50.174 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/00-INDEX 2024-04-02T14:45:50.175 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ 2024-04-02T14:45:50.176 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ABI.txt 2024-04-02T14:45:50.177 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/ 2024-04-02T14:45:50.179 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/interrupts.txt 2024-04-02T14:45:50.180 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/pmu.txt 2024-04-02T14:45:50.181 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ 2024-04-02T14:45:50.182 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/adapteva.txt 2024-04-02T14:45:50.184 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/ 2024-04-02T14:45:50.185 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.txt 2024-04-02T14:45:50.186 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-sdram-edac.txt 2024-04-02T14:45:50.187 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-system.txt 2024-04-02T14:45:50.189 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/amlogic.txt 2024-04-02T14:45:50.190 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arch_timer.txt 2024-04-02T14:45:50.192 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arm-boards 2024-04-02T14:45:50.193 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt 2024-04-02T14:45:50.194 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp.txt 2024-04-02T14:45:50.195 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-375.txt 2024-04-02T14:45:50.197 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-380-mpcore-soc-ctrl.txt 2024-04-02T14:45:50.198 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-38x.txt 2024-04-02T14:45:50.199 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-cpu-reset.txt 2024-04-02T14:45:50.201 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armadeus.txt 2024-04-02T14:45:50.202 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-at91.txt 2024-04-02T14:45:50.203 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-pmc.txt 2024-04-02T14:45:50.205 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/axxia.txt 2024-04-02T14:45:50.206 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/ 2024-04-02T14:45:50.207 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm11351.txt 2024-04-02T14:45:50.208 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm21664.txt 2024-04-02T14:45:50.210 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm63138.txt 2024-04-02T14:45:50.211 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/brcm,bcm11351-cpu-method 2024-04-02T14:45:50.212 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/cygnus.txt 2024-04-02T14:45:50.214 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-resetmgr.txt 2024-04-02T14:45:50.215 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-timer.txt 2024-04-02T14:45:50.217 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-wdt.txt 2024-04-02T14:45:50.218 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm2835.txt 2024-04-02T14:45:50.219 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm4708.txt 2024-04-02T14:45:50.220 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/brcm-brcmstb.txt 2024-04-02T14:45:50.222 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda.txt 2024-04-02T14:45:50.223 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/ 2024-04-02T14:45:50.224 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/combophy.txt 2024-04-02T14:45:50.226 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/l2ecc.txt 2024-04-02T14:45:50.227 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/mem-ctrlr.txt 2024-04-02T14:45:50.228 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cavium-thunder.txt 2024-04-02T14:45:50.230 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cci.txt 2024-04-02T14:45:50.231 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ccn.txt 2024-04-02T14:45:50.232 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coherency-fabric.txt 2024-04-02T14:45:50.234 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coresight.txt 2024-04-02T14:45:50.235 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/ 2024-04-02T14:45:50.236 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/marvell,berlin-smp 2024-04-02T14:45:50.237 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpus.txt 2024-04-02T14:45:50.238 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci.txt 2024-04-02T14:45:50.239 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/ 2024-04-02T14:45:50.240 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/cp-intc.txt 2024-04-02T14:45:50.242 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/digicolor.txt 2024-04-02T14:45:50.243 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/ 2024-04-02T14:45:50.244 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/power_domain.txt 2024-04-02T14:45:50.245 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/smp-sysram.txt 2024-04-02T14:45:50.248 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/ 2024-04-02T14:45:50.249 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/tlm,trusted-foundations.txt 2024-04-02T14:45:50.250 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fsl.txt 2024-04-02T14:45:50.251 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fw-cfg.txt 2024-04-02T14:45:50.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.257+0000 7fba97200640 1 -- 172.21.15.62:0/3974502179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba901048c0 msgr2=0x7fba90108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.257+0000 7fba97200640 1 --2- 172.21.15.62:0/3974502179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba901048c0 0x7fba90108900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009900 tx=0x7fba7c02bc40 comp rx=0 tx=0).stop 2024-04-02T14:45:50.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.257+0000 7fba97200640 1 -- 172.21.15.62:0/3974502179 shutdown_connections 2024-04-02T14:45:50.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.257+0000 7fba97200640 1 --2- 172.21.15.62:0/3974502179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba90108fe0 0x7fba90110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.257+0000 7fba97200640 1 --2- 172.21.15.62:0/3974502179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba901048c0 0x7fba90108900 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.257+0000 7fba97200640 1 --2- 172.21.15.62:0/3974502179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba90103f10 0x7fba901042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.257+0000 7fba97200640 1 -- 172.21.15.62:0/3974502179 >> 172.21.15.62:0/3974502179 conn(0x7fba900ff750 msgr2=0x7fba90101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.257+0000 7fba97200640 1 -- 172.21.15.62:0/3974502179 shutdown_connections 2024-04-02T14:45:50.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.257+0000 7fba97200640 1 -- 172.21.15.62:0/3974502179 wait complete. 2024-04-02T14:45:50.253 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/geniatech.txt 2024-04-02T14:45:50.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.258+0000 7fba97200640 1 Processor -- start 2024-04-02T14:45:50.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.258+0000 7fba97200640 1 -- start start 2024-04-02T14:45:50.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.258+0000 7fba97200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba90103f10 0x7fba9019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.258+0000 7fba97200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba90108fe0 0x7fba9019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba97200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba9019efd0 0x7fba9019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba97200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fba901141b0 con 0x7fba90103f10 2024-04-02T14:45:50.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba97200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fba90114030 con 0x7fba9019efd0 2024-04-02T14:45:50.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba97200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fba90114330 con 0x7fba90108fe0 2024-04-02T14:45:50.255 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic-v3.txt 2024-04-02T14:45:50.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba95200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba9019efd0 0x7fba9019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba90103f10 0x7fba9019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba95200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba9019efd0 0x7fba9019f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48874/0 (socket says 172.21.15.62:48874) 2024-04-02T14:45:50.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba95200640 1 -- 172.21.15.62:0/3089093961 learned_addr learned my addr 172.21.15.62:0/3089093961 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba8f400640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba90108fe0 0x7fba9019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba95200640 1 -- 172.21.15.62:0/3089093961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba90108fe0 msgr2=0x7fba9019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba95200640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba90108fe0 0x7fba9019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba95200640 1 -- 172.21.15.62:0/3089093961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba90103f10 msgr2=0x7fba9019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba95200640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba90103f10 0x7fba9019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba95200640 1 -- 172.21.15.62:0/3089093961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9019fac0 con 0x7fba9019efd0 2024-04-02T14:45:50.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.259+0000 7fba8f400640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba90108fe0 0x7fba9019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:50.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.260+0000 7fba95200640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba9019efd0 0x7fba9019f480 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fba88009fd0 tx=0x7fba8800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.260+0000 7fba8cc00640 1 -- 172.21.15.62:0/3089093961 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88017070 con 0x7fba9019efd0 2024-04-02T14:45:50.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.260+0000 7fba8cc00640 1 -- 172.21.15.62:0/3089093961 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fba88004020 con 0x7fba9019efd0 2024-04-02T14:45:50.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.260+0000 7fba97200640 1 -- 172.21.15.62:0/3089093961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9019fdb0 con 0x7fba9019efd0 2024-04-02T14:45:50.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.260+0000 7fba8cc00640 1 -- 172.21.15.62:0/3089093961 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880106c0 con 0x7fba9019efd0 2024-04-02T14:45:50.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.260+0000 7fba97200640 1 -- 172.21.15.62:0/3089093961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba901a7570 con 0x7fba9019efd0 2024-04-02T14:45:50.259 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic.txt 2024-04-02T14:45:50.259 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/global_timer.txt 2024-04-02T14:45:50.259 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/ 2024-04-02T14:45:50.259 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt 2024-04-02T14:45:50.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.263+0000 7fba8cc00640 1 -- 172.21.15.62:0/3089093961 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fba880041d0 con 0x7fba9019efd0 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.264+0000 7fba99f95740 1 Processor -- start 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.264+0000 7fba99f95740 1 -- start start 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.264+0000 7fba99f95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b182578740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.264+0000 7fba99f95740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182578d10 0x55b182579190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba99f95740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182582080 0x55b182584470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba99f95740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b1824b9400 con 0x55b182582080 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba99f95740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b18249df10 con 0x55b182578d10 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba99f95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b1824dd6f0 con 0x55b1825799f0 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba8f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b182578740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba8fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182578d10 0x55b182579190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba8f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182582080 0x55b182584470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba8fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182578d10 0x55b182579190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48888/0 (socket says 172.21.15.62:48888) 2024-04-02T14:45:50.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba8fe00640 1 -- 172.21.15.62:0/3714400524 learned_addr learned my addr 172.21.15.62:0/3714400524 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba8fe00640 1 -- 172.21.15.62:0/3714400524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 msgr2=0x55b182578740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba8fe00640 1 --2- 172.21.15.62:0/3714400524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b182578740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba8fe00640 1 -- 172.21.15.62:0/3714400524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182582080 msgr2=0x55b182584470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba8fe00640 1 --2- 172.21.15.62:0/3714400524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182582080 0x55b182584470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.265+0000 7fba8fe00640 1 -- 172.21.15.62:0/3714400524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1821f13f0 con 0x55b182578d10 2024-04-02T14:45:50.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.266+0000 7fba8fe00640 1 --2- 172.21.15.62:0/3714400524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182578d10 0x55b182579190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fba80009d20 tx=0x7fba80011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8398d74d405cb43 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.266+0000 7fba8cc00640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fba740a3da0 0x7fba740a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.266+0000 7fba85600640 1 -- 172.21.15.62:0/3714400524 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8001e070 con 0x55b182578d10 2024-04-02T14:45:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.266+0000 7fba8cc00640 1 -- 172.21.15.62:0/3089093961 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fba880d0460 con 0x7fba9019efd0 2024-04-02T14:45:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.266+0000 7fba85600640 1 -- 172.21.15.62:0/3714400524 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fba80019b30 con 0x55b182578d10 2024-04-02T14:45:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.266+0000 7fba8f400640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fba740a3da0 0x7fba740a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.266+0000 7fba85600640 1 -- 172.21.15.62:0/3714400524 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8000ec40 con 0x55b182578d10 2024-04-02T14:45:50.264 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/idle-states.txt 2024-04-02T14:45:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.267+0000 7fba99f95740 1 -- 172.21.15.62:0/3714400524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182578d10 msgr2=0x55b182579190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.267+0000 7fba99f95740 1 --2- 172.21.15.62:0/3714400524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182578d10 0x55b182579190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fba80009d20 tx=0x7fba80011810 comp rx=0 tx=0).stop 2024-04-02T14:45:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.267+0000 7fba8f400640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fba740a3da0 0x7fba740a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fba7c006010 tx=0x7fba7c000bc0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.267+0000 7fba99f95740 1 -- 172.21.15.62:0/3714400524 shutdown_connections 2024-04-02T14:45:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.267+0000 7fba99f95740 1 --2- 172.21.15.62:0/3714400524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182582080 0x55b182584470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.267+0000 7fba99f95740 1 --2- 172.21.15.62:0/3714400524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182578d10 0x55b182579190 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.267+0000 7fba99f95740 1 --2- 172.21.15.62:0/3714400524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b182578740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.267+0000 7fba99f95740 1 -- 172.21.15.62:0/3714400524 >> 172.21.15.62:0/3714400524 conn(0x55b18253fa50 msgr2=0x55b182573760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.267+0000 7fba99f95740 1 -- 172.21.15.62:0/3714400524 shutdown_connections 2024-04-02T14:45:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.267+0000 7fba99f95740 1 -- 172.21.15.62:0/3714400524 wait complete. 2024-04-02T14:45:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.267+0000 7fba99f95740 1 -- start start 2024-04-02T14:45:50.266 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/insignal-boards.txt 2024-04-02T14:45:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.268+0000 7fba99f95740 1 Processor -- start 2024-04-02T14:45:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.268+0000 7fba99f95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b182578d10 0x55b18257f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.268+0000 7fba99f95740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1825799f0 0x55b18257f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.268+0000 7fba99f95740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182582080 0x55b18257fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.268+0000 7fba99f95740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b1824d96a0 con 0x55b182582080 2024-04-02T14:45:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.268+0000 7fba99f95740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b182485980 con 0x55b1825799f0 2024-04-02T14:45:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.268+0000 7fba99f95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b182498cb0 con 0x55b182578d10 2024-04-02T14:45:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba99f95740 10 client.0 _ll_register_callbacks cb 0x55b1824986a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba8f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1825799f0 0x55b18257f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba95200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182582080 0x55b18257fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba8fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b182578d10 0x55b18257f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba8f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1825799f0 0x55b18257f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48898/0 (socket says 172.21.15.62:48898) 2024-04-02T14:45:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba8f400640 1 -- 172.21.15.62:0/878713798 learned_addr learned my addr 172.21.15.62:0/878713798 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.268 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/ 2024-04-02T14:45:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba8f400640 1 -- 172.21.15.62:0/878713798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b182578d10 msgr2=0x55b18257f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba8f400640 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b182578d10 0x55b18257f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba8f400640 1 -- 172.21.15.62:0/878713798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182582080 msgr2=0x55b18257fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba8f400640 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182582080 0x55b18257fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba8f400640 1 -- 172.21.15.62:0/878713798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1822bb7d0 con 0x55b1825799f0 2024-04-02T14:45:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.269+0000 7fba95200640 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182582080 0x55b18257fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.270+0000 7fba8f400640 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1825799f0 0x55b18257f5a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fba7c02c980 tx=0x7fba7c02bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af014f2741e496f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.270+0000 7fba7a000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.270+0000 7fba7a000640 1 -- 172.21.15.62:0/878713798 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c053060 con 0x55b1825799f0 2024-04-02T14:45:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.270+0000 7fba99f95740 1 -- 172.21.15.62:0/878713798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba80019b30 con 0x55b1825799f0 2024-04-02T14:45:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.270+0000 7fba7a000640 1 -- 172.21.15.62:0/878713798 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fba7c0053e0 con 0x55b1825799f0 2024-04-02T14:45:50.270 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/keystone.txt 2024-04-02T14:45:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.270+0000 7fba7a000640 1 -- 172.21.15.62:0/878713798 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba7c047070 con 0x55b1825799f0 2024-04-02T14:45:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.270+0000 7fba99f95740 10 client.26042 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.270+0000 7fba99f95740 1 -- 172.21.15.62:0/878713798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b182590f20 con 0x55b1825799f0 2024-04-02T14:45:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.271+0000 7fba7a000640 1 -- 172.21.15.62:0/878713798 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fba80019b30 con 0x55b1825799f0 2024-04-02T14:45:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.271+0000 7fba7a000640 1 -- 172.21.15.62:0/878713798 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fba7c040040 con 0x55b1825799f0 2024-04-02T14:45:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.271+0000 7fba99f95740 10 client.26042 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.271+0000 7fba99f95740 10 client.26042 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:45:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.271+0000 7fba99f95740 20 client.26042 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.271+0000 7fba99f95740 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55b182592380 0x55b1825947c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.271+0000 7fba99f95740 4 client.26042 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.271+0000 7fba99f95740 1 -- 172.21.15.62:0/878713798 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b18249d040 con 0x55b182592380 2024-04-02T14:45:50.272 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/kirkwood.txt 2024-04-02T14:45:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.271+0000 7fba8fe00640 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55b182592380 0x55b1825947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.272+0000 7fba8fe00640 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55b182592380 0x55b1825947c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c51d9b772143105 server_cookie=6304f6b8a9b168a5 in_seq=0 out_seq=0 2024-04-02T14:45:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.272+0000 7fba7a000640 10 client.26042 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:45:50.273 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/l2cc.txt 2024-04-02T14:45:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba7a000640 1 -- 172.21.15.62:0/878713798 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b18249d040 con 0x55b182592380 2024-04-02T14:45:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba7a000640 10 client.26042 handle_command_reply: tid=0 2024-04-02T14:45:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 1 client.26042 shutdown 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 1 -- 172.21.15.62:0/878713798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1825799f0 msgr2=0x55b18257f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1825799f0 0x55b18257f5a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fba7c02c980 tx=0x7fba7c02bc40 comp rx=0 tx=0).stop 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 1 -- 172.21.15.62:0/878713798 shutdown_connections 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55b182592380 0x55b1825947c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182582080 0x55b18257fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1825799f0 0x55b18257f5a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 1 --2- 172.21.15.62:0/878713798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b182578d10 0x55b18257f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 1 -- 172.21.15.62:0/878713798 >> 172.21.15.62:0/878713798 conn(0x55b18253fa50 msgr2=0x55b1825754b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba7a000640 0 client.26042 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 1 -- 172.21.15.62:0/878713798 shutdown_connections 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 1 -- 172.21.15.62:0/878713798 wait complete. 2024-04-02T14:45:50.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.273+0000 7fba99f95740 20 client.26042 trim_cache size 0 max 16384 2024-04-02T14:45:50.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx-mic.txt 2024-04-02T14:45:50.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx.txt 2024-04-02T14:45:50.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,berlin.txt 2024-04-02T14:45:50.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,dove.txt 2024-04-02T14:45:50.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,kirkwood.txt 2024-04-02T14:45:50.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek.txt 2024-04-02T14:45:50.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/ 2024-04-02T14:45:50.279 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/mediatek,sysirq.txt 2024-04-02T14:45:50.279 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/moxart.txt 2024-04-02T14:45:50.281 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/ 2024-04-02T14:45:50.282 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/feroceon.txt 2024-04-02T14:45:50.283 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/intc.txt 2024-04-02T14:45:50.284 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/mrvl.txt 2024-04-02T14:45:50.285 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/tauros2.txt 2024-04-02T14:45:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.291+0000 7fba99f95740 1 Processor -- start 2024-04-02T14:45:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.292+0000 7fba99f95740 1 -- start start 2024-04-02T14:45:50.287 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/timer.txt 2024-04-02T14:45:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.292+0000 7fba99f95740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 0x55b182581790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.292+0000 7fba99f95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b18258daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.292+0000 7fba99f95740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182582080 0x55b182572720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.292+0000 7fba99f95740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fba901afa90 con 0x55b182578d10 2024-04-02T14:45:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.292+0000 7fba99f95740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fba901b15d0 con 0x55b182582080 2024-04-02T14:45:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.292+0000 7fba99f95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fba901b0160 con 0x55b1825799f0 2024-04-02T14:45:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.292+0000 7fba8f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b18258daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.292+0000 7fba8f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b18258daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:42240/0 (socket says 172.21.15.62:42240) 2024-04-02T14:45:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.292+0000 7fba8f400640 1 -- 172.21.15.62:0/4077981521 learned_addr learned my addr 172.21.15.62:0/4077981521 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.293+0000 7fba95200640 1 --2- 172.21.15.62:0/4077981521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182582080 0x55b182572720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.293+0000 7fba8fe00640 1 --2- 172.21.15.62:0/4077981521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 0x55b182581790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.293+0000 7fba8f400640 1 -- 172.21.15.62:0/4077981521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182582080 msgr2=0x55b182572720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.293+0000 7fba8f400640 1 --2- 172.21.15.62:0/4077981521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182582080 0x55b182572720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.293+0000 7fba8f400640 1 -- 172.21.15.62:0/4077981521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 msgr2=0x55b182581790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.293+0000 7fba8f400640 1 --2- 172.21.15.62:0/4077981521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 0x55b182581790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.293+0000 7fba8f400640 1 -- 172.21.15.62:0/4077981521 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18257d5b0 con 0x55b1825799f0 2024-04-02T14:45:50.289 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ 2024-04-02T14:45:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.293+0000 7fba8f400640 1 --2- 172.21.15.62:0/4077981521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b18258daf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fba7c003b30 tx=0x7fba7c02c600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40c0f316f42c1105 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.293+0000 7fba84c00640 1 -- 172.21.15.62:0/4077981521 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c053070 con 0x55b1825799f0 2024-04-02T14:45:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.293+0000 7fba84c00640 1 -- 172.21.15.62:0/4077981521 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fba7c02c9e0 con 0x55b1825799f0 2024-04-02T14:45:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.293+0000 7fba84c00640 1 -- 172.21.15.62:0/4077981521 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c05d080 con 0x55b1825799f0 2024-04-02T14:45:50.291 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,kpss-acc.txt 2024-04-02T14:45:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.294+0000 7fba99f95740 1 -- 172.21.15.62:0/4077981521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 msgr2=0x55b18258daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.294+0000 7fba99f95740 1 --2- 172.21.15.62:0/4077981521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b18258daf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fba7c003b30 tx=0x7fba7c02c600 comp rx=0 tx=0).stop 2024-04-02T14:45:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.294+0000 7fba99f95740 1 -- 172.21.15.62:0/4077981521 shutdown_connections 2024-04-02T14:45:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.294+0000 7fba99f95740 1 --2- 172.21.15.62:0/4077981521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182582080 0x55b182572720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.294+0000 7fba99f95740 1 --2- 172.21.15.62:0/4077981521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b18258daf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.294+0000 7fba99f95740 1 --2- 172.21.15.62:0/4077981521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 0x55b182581790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.294+0000 7fba99f95740 1 -- 172.21.15.62:0/4077981521 >> 172.21.15.62:0/4077981521 conn(0x55b18253fa50 msgr2=0x55b182572010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.294+0000 7fba99f95740 1 -- 172.21.15.62:0/4077981521 shutdown_connections 2024-04-02T14:45:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.294+0000 7fba99f95740 1 -- 172.21.15.62:0/4077981521 wait complete. 2024-04-02T14:45:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.295+0000 7fba99f95740 1 -- start start 2024-04-02T14:45:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.295+0000 7fba99f95740 1 Processor -- start 2024-04-02T14:45:50.293 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,saw2.txt 2024-04-02T14:45:50.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.295+0000 7fba99f95740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 0x55b18258a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.295+0000 7fba99f95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b182582a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba99f95740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182582080 0x55b182582fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba99f95740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fba901b0f00 con 0x55b182578d10 2024-04-02T14:45:50.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba99f95740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fba901b0830 con 0x55b182582080 2024-04-02T14:45:50.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba99f95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fba901ada90 con 0x55b1825799f0 2024-04-02T14:45:50.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba99f95740 10 client.0 _ll_register_callbacks cb 0x7fba7c02cc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:50.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 0x55b18258a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 0x55b18258a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35662/0 (socket says 172.21.15.62:35662) 2024-04-02T14:45:50.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba8fe00640 1 -- 172.21.15.62:0/76781305 learned_addr learned my addr 172.21.15.62:0/76781305 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba95200640 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182582080 0x55b182582fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba8f400640 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b182582a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba8fe00640 1 -- 172.21.15.62:0/76781305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 msgr2=0x55b182582a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba8fe00640 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b182582a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba8fe00640 1 -- 172.21.15.62:0/76781305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182582080 msgr2=0x55b182582fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba8fe00640 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182582080 0x55b182582fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.296+0000 7fba8fe00640 1 -- 172.21.15.62:0/76781305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b182583780 con 0x55b182578d10 2024-04-02T14:45:50.296 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ssbi.txt 2024-04-02T14:45:50.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.297+0000 7fba8fe00640 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 0x55b18258a3c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fba80009250 tx=0x7fba80009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c076c35fe851ae03 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.297+0000 7fba7a000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:45:50.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.297+0000 7fba7a000640 1 -- 172.21.15.62:0/76781305 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8001e040 con 0x55b182578d10 2024-04-02T14:45:50.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.297+0000 7fba7a000640 1 -- 172.21.15.62:0/76781305 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fba8000e910 con 0x55b182578d10 2024-04-02T14:45:50.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.297+0000 7fba99f95740 1 -- 172.21.15.62:0/76781305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba7c02c9e0 con 0x55b182578d10 2024-04-02T14:45:50.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.297+0000 7fba7a000640 1 -- 172.21.15.62:0/76781305 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba8000eb10 con 0x55b182578d10 2024-04-02T14:45:50.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.297+0000 7fba99f95740 10 client.15924 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:50.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.297+0000 7fba99f95740 1 -- 172.21.15.62:0/76781305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1825f8900 con 0x55b182578d10 2024-04-02T14:45:50.297 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/timer.txt 2024-04-02T14:45:50.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.298+0000 7fba7a000640 1 -- 172.21.15.62:0/76781305 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fba7c02c9e0 con 0x55b182578d10 2024-04-02T14:45:50.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.298+0000 7fba7a000640 1 -- 172.21.15.62:0/76781305 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fba800035f0 con 0x55b182578d10 2024-04-02T14:45:50.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.298+0000 7fba99f95740 10 client.15924 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:50.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.298+0000 7fba99f95740 10 client.15924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:45:50.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.298+0000 7fba99f95740 20 client.15924 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:50.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.298+0000 7fba99f95740 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55b1825f8e10 0x55b1825fb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.298+0000 7fba99f95740 4 client.15924 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:45:50.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.298+0000 7fba99f95740 1 -- 172.21.15.62:0/76781305 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1824eac30 con 0x55b1825f8e10 2024-04-02T14:45:50.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.298+0000 7fba8f400640 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55b1825f8e10 0x55b1825fb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.299 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mvebu-system-controller.txt 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba8f400640 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55b1825f8e10 0x55b1825fb230 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=198ddbc37a76ece0 server_cookie=4f79985cb21eb85b in_seq=0 out_seq=0 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba7a000640 10 client.15924 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba7a000640 1 -- 172.21.15.62:0/76781305 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1824eac30 con 0x55b1825f8e10 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba7a000640 10 client.15924 handle_command_reply: tid=0 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba99f95740 1 client.15924 shutdown 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba99f95740 1 -- 172.21.15.62:0/76781305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 msgr2=0x55b18258a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba99f95740 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 0x55b18258a3c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fba80009250 tx=0x7fba80009c20 comp rx=0 tx=0).stop 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba99f95740 1 -- 172.21.15.62:0/76781305 shutdown_connections 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba99f95740 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x55b1825f8e10 0x55b1825fb230 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba99f95740 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b182582080 0x55b182582fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba99f95740 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1825799f0 0x55b182582a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba99f95740 1 --2- 172.21.15.62:0/76781305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b182578d10 0x55b18258a3c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba7a000640 0 client.15924 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:45:50.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.299+0000 7fba99f95740 1 -- 172.21.15.62:0/76781305 >> 172.21.15.62:0/76781305 conn(0x55b18253fa50 msgr2=0x55b18257f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.300+0000 7fba99f95740 1 -- 172.21.15.62:0/76781305 shutdown_connections 2024-04-02T14:45:50.301 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/nspire.txt 2024-04-02T14:45:50.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.300+0000 7fba99f95740 1 -- 172.21.15.62:0/76781305 wait complete. 2024-04-02T14:45:50.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.300+0000 7fba99f95740 20 client.15924 trim_cache size 0 max 16384 2024-04-02T14:45:50.302 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:45:50.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.300+0000 7fba97200640 1 -- 172.21.15.62:0/3089093961 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fba740a3da0 msgr2=0x7fba740a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.300+0000 7fba97200640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fba740a3da0 0x7fba740a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fba7c006010 tx=0x7fba7c000bc0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.300+0000 7fba97200640 1 -- 172.21.15.62:0/3089093961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba9019efd0 msgr2=0x7fba9019f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.300+0000 7fba97200640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba9019efd0 0x7fba9019f480 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fba88009fd0 tx=0x7fba8800b260 comp rx=0 tx=0).stop 2024-04-02T14:45:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.301+0000 7fba97200640 1 -- 172.21.15.62:0/3089093961 shutdown_connections 2024-04-02T14:45:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.301+0000 7fba97200640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fba740a3da0 0x7fba740a6260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.301+0000 7fba97200640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba9019efd0 0x7fba9019f480 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.301+0000 7fba97200640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba90108fe0 0x7fba9019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.301+0000 7fba97200640 1 --2- 172.21.15.62:0/3089093961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba90103f10 0x7fba9019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.301+0000 7fba97200640 1 -- 172.21.15.62:0/3089093961 >> 172.21.15.62:0/3089093961 conn(0x7fba900ff750 msgr2=0x7fba90101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.301+0000 7fba97200640 1 -- 172.21.15.62:0/3089093961 shutdown_connections 2024-04-02T14:45:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.301+0000 7fba97200640 1 -- 172.21.15.62:0/3089093961 wait complete. 2024-04-02T14:45:50.304 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/olimex.txt 2024-04-02T14:45:50.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/ 2024-04-02T14:45:50.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/counter.txt 2024-04-02T14:45:50.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/crossbar.txt 2024-04-02T14:45:50.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dmm.txt 2024-04-02T14:45:50.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dsp.txt 2024-04-02T14:45:50.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/intc.txt 2024-04-02T14:45:50.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/iva.txt 2024-04-02T14:45:50.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/l3-noc.txt 2024-04-02T14:45:50.306 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/mpu.txt 2024-04-02T14:45:50.307 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/omap.txt 2024-04-02T14:45:50.308 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/prcm.txt 2024-04-02T14:45:50.309 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/timer.txt 2024-04-02T14:45:50.311 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/picoxcell.txt 2024-04-02T14:45:50.311 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/pmu.txt 2024-04-02T14:45:50.313 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:45:50.313 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/primecell.txt 2024-04-02T14:45:50.314 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/psci.txt 2024-04-02T14:45:50.319 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip.txt 2024-04-02T14:45:50.319 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/ 2024-04-02T14:45:50.319 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu-sram.txt 2024-04-02T14:45:50.319 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu.txt 2024-04-02T14:45:50.320 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/smp-sram.txt 2024-04-02T14:45:50.321 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rtsm-dcscb.txt 2024-04-02T14:45:50.322 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung-boards.txt 2024-04-02T14:45:50.323 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/ 2024-04-02T14:45:50.324 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-adc.txt 2024-04-02T14:45:50.326 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.txt 2024-04-02T14:45:50.327 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/interrupt-combiner.txt 2024-04-02T14:45:50.328 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/pmu.txt 2024-04-02T14:45:50.329 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/sysreg.txt 2024-04-02T14:45:50.330 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/shmobile.txt 2024-04-02T14:45:50.331 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sirf.txt 2024-04-02T14:45:50.332 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-misc.txt 2024-04-02T14:45:50.334 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-timer.txt 2024-04-02T14:45:50.335 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear.txt 2024-04-02T14:45:50.336 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/ 2024-04-02T14:45:50.337 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/shirq.txt 2024-04-02T14:45:50.338 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sprd.txt 2024-04-02T14:45:50.339 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-nomadik.txt 2024-04-02T14:45:50.340 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-u300.txt 2024-04-02T14:45:50.342 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sti.txt 2024-04-02T14:45:50.343 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sunxi.txt 2024-04-02T14:45:50.344 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra.txt 2024-04-02T14:45:50.345 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/ 2024-04-02T14:45:50.346 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-ahb.txt 2024-04-02T14:45:50.347 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-emc.txt 2024-04-02T14:45:50.349 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-flowctrl.txt 2024-04-02T14:45:50.350 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-mc.txt 2024-04-02T14:45:50.351 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-pmc.txt 2024-04-02T14:45:50.352 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra30-mc.txt 2024-04-02T14:45:50.353 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/topology.txt 2024-04-02T14:45:50.354 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/twd.txt 2024-04-02T14:45:50.357 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/ 2024-04-02T14:45:50.359 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/power_domain.txt 2024-04-02T14:45:50.361 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-fpga-irq.txt 2024-04-02T14:45:50.363 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-sysreg.txt 2024-04-02T14:45:50.364 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-scc.txt 2024-04-02T14:45:50.367 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-sysreg.txt 2024-04-02T14:45:50.369 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress.txt 2024-04-02T14:45:50.371 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vic.txt 2024-04-02T14:45:50.373 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500.txt 2024-04-02T14:45:50.376 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/ 2024-04-02T14:45:50.378 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-intc.txt 2024-04-02T14:45:50.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.535+0000 7f23f2000640 1 -- 172.21.15.62:0/2629885520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23ec108cc0 msgr2=0x7f23ec10cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.535+0000 7f23f2000640 1 --2- 172.21.15.62:0/2629885520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23ec108cc0 0x7f23ec10cd00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009900 tx=0x7f23d802bc30 comp rx=0 tx=0).stop 2024-04-02T14:45:50.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.535+0000 7f23f2000640 1 -- 172.21.15.62:0/2629885520 shutdown_connections 2024-04-02T14:45:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.535+0000 7f23f2000640 1 --2- 172.21.15.62:0/2629885520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23ec10d3e0 0x7f23ec114fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.535+0000 7f23f2000640 1 --2- 172.21.15.62:0/2629885520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23ec108cc0 0x7f23ec10cd00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.535+0000 7f23f2000640 1 --2- 172.21.15.62:0/2629885520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ec108310 0x7f23ec1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.535+0000 7f23f2000640 1 -- 172.21.15.62:0/2629885520 >> 172.21.15.62:0/2629885520 conn(0x7f23ec076040 msgr2=0x7f23ec078460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.535+0000 7f23f2000640 1 -- 172.21.15.62:0/2629885520 shutdown_connections 2024-04-02T14:45:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.535+0000 7f23f2000640 1 -- 172.21.15.62:0/2629885520 wait complete. 2024-04-02T14:45:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.536+0000 7f23f2000640 1 Processor -- start 2024-04-02T14:45:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.536+0000 7f23f2000640 1 -- start start 2024-04-02T14:45:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.536+0000 7f23f2000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23ec108310 0x7f23ec1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.536+0000 7f23f2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ec10d3e0 0x7f23ec1a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.536+0000 7f23f2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23ec19e8e0 0x7f23ec19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.536+0000 7f23f2000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec118560 con 0x7f23ec10d3e0 2024-04-02T14:45:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.536+0000 7f23f2000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1183e0 con 0x7f23ec19e8e0 2024-04-02T14:45:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.536+0000 7f23f2000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1186e0 con 0x7f23ec108310 2024-04-02T14:45:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.537+0000 7f23ebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23ec19e8e0 0x7f23ec19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.537+0000 7f23eaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ec10d3e0 0x7f23ec1a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.537+0000 7f23ebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23ec19e8e0 0x7f23ec19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48920/0 (socket says 172.21.15.62:48920) 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.537+0000 7f23ebe00640 1 -- 172.21.15.62:0/2404321148 learned_addr learned my addr 172.21.15.62:0/2404321148 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.537+0000 7f23eb400640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23ec108310 0x7f23ec1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.537+0000 7f23ebe00640 1 -- 172.21.15.62:0/2404321148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23ec108310 msgr2=0x7f23ec1a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.537+0000 7f23ebe00640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23ec108310 0x7f23ec1a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.537+0000 7f23ebe00640 1 -- 172.21.15.62:0/2404321148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ec10d3e0 msgr2=0x7f23ec1a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.537+0000 7f23ebe00640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ec10d3e0 0x7f23ec1a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.537+0000 7f23ebe00640 1 -- 172.21.15.62:0/2404321148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec19f400 con 0x7f23ec19e8e0 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.537+0000 7f23ebe00640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23ec19e8e0 0x7f23ec19ed90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23e400eb10 tx=0x7f23e400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.538+0000 7f23e3e00640 1 -- 172.21.15.62:0/2404321148 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4004b20 con 0x7f23ec19e8e0 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.538+0000 7f23e3e00640 1 -- 172.21.15.62:0/2404321148 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23e4004cc0 con 0x7f23ec19e8e0 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.538+0000 7f23f2000640 1 -- 172.21.15.62:0/2404321148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ec19f6f0 con 0x7f23ec19e8e0 2024-04-02T14:45:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.538+0000 7f23e3e00640 1 -- 172.21.15.62:0/2404321148 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40107f0 con 0x7f23ec19e8e0 2024-04-02T14:45:50.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.538+0000 7f23f2000640 1 -- 172.21.15.62:0/2404321148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ec1ab990 con 0x7f23ec19e8e0 2024-04-02T14:45:50.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.541+0000 7f23e3e00640 1 -- 172.21.15.62:0/2404321148 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f23e4005020 con 0x7f23ec19e8e0 2024-04-02T14:45:50.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.542+0000 7f23f4c28740 1 Processor -- start 2024-04-02T14:45:50.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.542+0000 7f23f4c28740 1 -- start start 2024-04-02T14:45:50.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.542+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 0x55d687552740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.542+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d687553190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.542+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d68755e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.542+0000 7f23f4c28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d687493400 con 0x55d6875539f0 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.542+0000 7f23f4c28740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d687477f10 con 0x55d68755c080 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.542+0000 7f23f4c28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d6874b76f0 con 0x55d687552d10 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.542+0000 7f23e3e00640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23d00a3da0 0x7f23d00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23eaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 0x55d687552740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23eaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 0x55d687552740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35678/0 (socket says 172.21.15.62:35678) 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23eaa00640 1 -- 172.21.15.62:0/1340212547 learned_addr learned my addr 172.21.15.62:0/1340212547 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23e3e00640 1 -- 172.21.15.62:0/2404321148 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f23e40d0d00 con 0x7f23ec19e8e0 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23ebe00640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23d00a3da0 0x7f23d00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23eaa00640 1 --2- 172.21.15.62:0/1340212547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d68755e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23eb400640 1 --2- 172.21.15.62:0/1340212547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d687553190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23eaa00640 1 -- 172.21.15.62:0/1340212547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 msgr2=0x55d687553190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23eaa00640 1 --2- 172.21.15.62:0/1340212547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d687553190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23eaa00640 1 -- 172.21.15.62:0/1340212547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 msgr2=0x55d687552740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23eaa00640 1 --2- 172.21.15.62:0/1340212547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 0x55d687552740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.543+0000 7f23eaa00640 1 -- 172.21.15.62:0/1340212547 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6871cb3f0 con 0x55d68755c080 2024-04-02T14:45:50.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.544+0000 7f23eaa00640 1 --2- 172.21.15.62:0/1340212547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d68755e470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f23ec06b150 tx=0x7f23d8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=295593d73d646ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.544+0000 7f23ebe00640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23d00a3da0 0x7f23d00a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23e40a53d0 tx=0x7f23e400efd0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.544+0000 7f23d7e00640 1 -- 172.21.15.62:0/1340212547 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d802cc80 con 0x55d68755c080 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.544+0000 7f23d7e00640 1 -- 172.21.15.62:0/1340212547 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23d8048070 con 0x55d68755c080 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.544+0000 7f23d7e00640 1 -- 172.21.15.62:0/1340212547 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8003dd0 con 0x55d68755c080 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.545+0000 7f23f4c28740 1 -- 172.21.15.62:0/1340212547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 msgr2=0x55d68755e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.545+0000 7f23f4c28740 1 --2- 172.21.15.62:0/1340212547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d68755e470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f23ec06b150 tx=0x7f23d8005fd0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.545+0000 7f23f4c28740 1 -- 172.21.15.62:0/1340212547 shutdown_connections 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.545+0000 7f23f4c28740 1 --2- 172.21.15.62:0/1340212547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d68755e470 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.545+0000 7f23f4c28740 1 --2- 172.21.15.62:0/1340212547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d687553190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.545+0000 7f23f4c28740 1 --2- 172.21.15.62:0/1340212547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 0x55d687552740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.545+0000 7f23f4c28740 1 -- 172.21.15.62:0/1340212547 >> 172.21.15.62:0/1340212547 conn(0x55d687519a50 msgr2=0x55d68754d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.545+0000 7f23f4c28740 1 -- 172.21.15.62:0/1340212547 shutdown_connections 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.545+0000 7f23f4c28740 1 -- 172.21.15.62:0/1340212547 wait complete. 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.545+0000 7f23f4c28740 1 -- start start 2024-04-02T14:45:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23f4c28740 1 Processor -- start 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d6875590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 0x55d687559600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d687559b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23f4c28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d6874b36a0 con 0x55d6875539f0 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23f4c28740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d68745f980 con 0x55d68755c080 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23f4c28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d687472cb0 con 0x55d687552d10 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23f4c28740 10 client.0 _ll_register_callbacks cb 0x55d6874726a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23eaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d687559b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23eaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d687559b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48940/0 (socket says 172.21.15.62:48940) 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23eb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 0x55d687559600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23eaa00640 1 -- 172.21.15.62:0/207283405 learned_addr learned my addr 172.21.15.62:0/207283405 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.546+0000 7f23eaa00640 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d6875590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.547+0000 7f23eaa00640 1 -- 172.21.15.62:0/207283405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 msgr2=0x55d6875590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.547+0000 7f23eaa00640 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d6875590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.547+0000 7f23eaa00640 1 -- 172.21.15.62:0/207283405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 msgr2=0x55d687559600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.547+0000 7f23eaa00640 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 0x55d687559600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.547+0000 7f23eaa00640 1 -- 172.21.15.62:0/207283405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6872957d0 con 0x55d68755c080 2024-04-02T14:45:50.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.547+0000 7f23eb400640 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 0x55d687559600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:50.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.547+0000 7f23eaa00640 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d687559b40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005ec0 tx=0x7f23d80039b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9a660906d853a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.547+0000 7f23d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:50.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.547+0000 7f23d4c00640 1 -- 172.21.15.62:0/207283405 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8004450 con 0x55d68755c080 2024-04-02T14:45:50.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.547+0000 7f23d4c00640 1 -- 172.21.15.62:0/207283405 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f23d80344e0 con 0x55d68755c080 2024-04-02T14:45:50.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.547+0000 7f23f4c28740 1 -- 172.21.15.62:0/207283405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23d8048070 con 0x55d68755c080 2024-04-02T14:45:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.548+0000 7f23d4c00640 1 -- 172.21.15.62:0/207283405 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23d8003e50 con 0x55d68755c080 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.548+0000 7f23f4c28740 10 client.26060 fetch_fsmap learned FSMap version 28 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.548+0000 7f23f4c28740 1 -- 172.21.15.62:0/207283405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d68756af00 con 0x55d68755c080 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.548+0000 7f23d4c00640 1 -- 172.21.15.62:0/207283405 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23d8048070 con 0x55d68755c080 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.548+0000 7f23d4c00640 1 -- 172.21.15.62:0/207283405 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f23d8043910 con 0x55d68755c080 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.548+0000 7f23f4c28740 10 client.26060 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.548+0000 7f23f4c28740 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.548+0000 7f23f4c28740 20 client.26060 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.548+0000 7f23f4c28740 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55d68756c250 0x55d68756e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.548+0000 7f23f4c28740 4 client.26060 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.548+0000 7f23f4c28740 1 -- 172.21.15.62:0/207283405 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d687477040 con 0x55d68756c250 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.549+0000 7f23eb400640 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55d68756c250 0x55d68756e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.549+0000 7f23eb400640 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55d68756c250 0x55d68756e640 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a025a65b928559 server_cookie=9619f350ab052ded in_seq=0 out_seq=0 2024-04-02T14:45:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.549+0000 7f23d4c00640 10 client.26060 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.550+0000 7f23d4c00640 1 -- 172.21.15.62:0/207283405 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d687477040 con 0x55d68756c250 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.550+0000 7f23d4c00640 10 client.26060 handle_command_reply: tid=0 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.550+0000 7f23f4c28740 1 client.26060 shutdown 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.551+0000 7f23f4c28740 1 -- 172.21.15.62:0/207283405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 msgr2=0x55d687559b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.551+0000 7f23f4c28740 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d687559b40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005ec0 tx=0x7f23d80039b0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.551+0000 7f23f4c28740 1 -- 172.21.15.62:0/207283405 shutdown_connections 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.551+0000 7f23f4c28740 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55d68756c250 0x55d68756e640 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.551+0000 7f23f4c28740 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d687559b40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.551+0000 7f23f4c28740 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6875539f0 0x55d687559600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.551+0000 7f23f4c28740 1 --2- 172.21.15.62:0/207283405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d6875590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.551+0000 7f23f4c28740 1 -- 172.21.15.62:0/207283405 >> 172.21.15.62:0/207283405 conn(0x55d687519a50 msgr2=0x55d68754f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.551+0000 7f23d4c00640 0 client.26060 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.551+0000 7f23f4c28740 1 -- 172.21.15.62:0/207283405 shutdown_connections 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.552+0000 7f23f4c28740 1 -- 172.21.15.62:0/207283405 wait complete. 2024-04-02T14:45:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.552+0000 7f23f4c28740 20 client.26060 trim_cache size 0 max 16384 2024-04-02T14:45:50.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.567+0000 7f23f4c28740 1 Processor -- start 2024-04-02T14:45:50.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.567+0000 7f23f4c28740 1 -- start start 2024-04-02T14:45:50.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.567+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d687552d10 0x55d68755b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.567+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6875539f0 0x55d687558220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.567+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d687559960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.567+0000 7f23f4c28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1b3de0 con 0x55d687552d10 2024-04-02T14:45:50.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.567+0000 7f23f4c28740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1b5920 con 0x55d68755c080 2024-04-02T14:45:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.567+0000 7f23f4c28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1b44b0 con 0x55d6875539f0 2024-04-02T14:45:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.568+0000 7f23eaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d687552d10 0x55d68755b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.568+0000 7f23eb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6875539f0 0x55d687558220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.568+0000 7f23eb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6875539f0 0x55d687558220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:42310/0 (socket says 172.21.15.62:42310) 2024-04-02T14:45:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.568+0000 7f23eb400640 1 -- 172.21.15.62:0/2716788526 learned_addr learned my addr 172.21.15.62:0/2716788526 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.568+0000 7f23eaa00640 1 --2- 172.21.15.62:0/2716788526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d687559960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.568+0000 7f23eb400640 1 -- 172.21.15.62:0/2716788526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 msgr2=0x55d687559960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.568+0000 7f23eb400640 1 --2- 172.21.15.62:0/2716788526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d687559960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.568+0000 7f23eb400640 1 -- 172.21.15.62:0/2716788526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d687552d10 msgr2=0x55d68755b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.568+0000 7f23eb400640 1 --2- 172.21.15.62:0/2716788526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d687552d10 0x55d68755b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.568+0000 7f23eb400640 1 -- 172.21.15.62:0/2716788526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d687551d40 con 0x55d6875539f0 2024-04-02T14:45:50.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.568+0000 7f23eb400640 1 --2- 172.21.15.62:0/2716788526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6875539f0 0x55d687558220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f23ec19fa20 tx=0x7f23dc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b89c219f426ce02 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.569+0000 7f23d7400640 1 -- 172.21.15.62:0/2716788526 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc021460 con 0x55d6875539f0 2024-04-02T14:45:50.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.569+0000 7f23d7400640 1 -- 172.21.15.62:0/2716788526 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23dc025030 con 0x55d6875539f0 2024-04-02T14:45:50.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.569+0000 7f23d7400640 1 -- 172.21.15.62:0/2716788526 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc0116a0 con 0x55d6875539f0 2024-04-02T14:45:50.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.569+0000 7f23f4c28740 1 -- 172.21.15.62:0/2716788526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6875539f0 msgr2=0x55d687558220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.569+0000 7f23f4c28740 1 --2- 172.21.15.62:0/2716788526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6875539f0 0x55d687558220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f23ec19fa20 tx=0x7f23dc005f90 comp rx=0 tx=0).stop 2024-04-02T14:45:50.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.570+0000 7f23f4c28740 1 -- 172.21.15.62:0/2716788526 shutdown_connections 2024-04-02T14:45:50.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.570+0000 7f23f4c28740 1 --2- 172.21.15.62:0/2716788526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d68755c080 0x55d687559960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.570+0000 7f23f4c28740 1 --2- 172.21.15.62:0/2716788526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6875539f0 0x55d687558220 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.570+0000 7f23f4c28740 1 --2- 172.21.15.62:0/2716788526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d687552d10 0x55d68755b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.570+0000 7f23f4c28740 1 -- 172.21.15.62:0/2716788526 >> 172.21.15.62:0/2716788526 conn(0x55d687519a50 msgr2=0x55d687566fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.570+0000 7f23f4c28740 1 -- 172.21.15.62:0/2716788526 shutdown_connections 2024-04-02T14:45:50.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.570+0000 7f23f4c28740 1 -- 172.21.15.62:0/2716788526 wait complete. 2024-04-02T14:45:50.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.570+0000 7f23f4c28740 1 -- start start 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.570+0000 7f23f4c28740 1 Processor -- start 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d68756c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d6875539f0 0x55d68756c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23f4c28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d68755c080 0x55d6875d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23f4c28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1b5250 con 0x55d68755c080 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23f4c28740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1b4b80 con 0x55d6875539f0 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23f4c28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1b1de0 con 0x55d687552d10 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23f4c28740 10 client.0 _ll_register_callbacks cb 0x7f23d8003b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23eb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d6875539f0 0x55d68756c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23eb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d6875539f0 0x55d68756c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48966/0 (socket says 172.21.15.62:48966) 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23eb400640 1 -- 172.21.15.62:0/1028981455 learned_addr learned my addr 172.21.15.62:0/1028981455 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23eaa00640 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d68756c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23eaa00640 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d68755c080 0x55d6875d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23eb400640 1 -- 172.21.15.62:0/1028981455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 msgr2=0x55d68756c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23eb400640 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d68756c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23eb400640 1 -- 172.21.15.62:0/1028981455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d68755c080 msgr2=0x55d6875d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23eb400640 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d68755c080 0x55d6875d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.571+0000 7f23eb400640 1 -- 172.21.15.62:0/1028981455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6875d0950 con 0x55d6875539f0 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.572+0000 7f23eaa00640 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d68756c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.572+0000 7f23eb400640 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d6875539f0 0x55d68756c980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f23dc027000 tx=0x7f23dc009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b1edc87e1f1af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.572+0000 7f23d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.572+0000 7f23d4c00640 1 -- 172.21.15.62:0/1028981455 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc020f40 con 0x55d6875539f0 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.572+0000 7f23d4c00640 1 -- 172.21.15.62:0/1028981455 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f23dc005030 con 0x55d6875539f0 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.572+0000 7f23f4c28740 1 -- 172.21.15.62:0/1028981455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23dc025030 con 0x55d6875539f0 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.572+0000 7f23d4c00640 1 -- 172.21.15.62:0/1028981455 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f23dc005270 con 0x55d6875539f0 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.572+0000 7f23f4c28740 10 client.26066 fetch_fsmap learned FSMap version 29 2024-04-02T14:45:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.572+0000 7f23f4c28740 1 -- 172.21.15.62:0/1028981455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d6875d1bf0 con 0x55d6875539f0 2024-04-02T14:45:50.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.573+0000 7f23d4c00640 1 -- 172.21.15.62:0/1028981455 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23dc025030 con 0x55d6875539f0 2024-04-02T14:45:50.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.573+0000 7f23d4c00640 1 -- 172.21.15.62:0/1028981455 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f23dc032020 con 0x55d6875539f0 2024-04-02T14:45:50.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.573+0000 7f23f4c28740 10 client.26066 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T14:45:50.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.573+0000 7f23f4c28740 10 client.26066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.573+0000 7f23f4c28740 20 client.26066 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.573+0000 7f23f4c28740 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55d6875d2300 0x55d6875d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.573+0000 7f23f4c28740 4 client.26066 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.573+0000 7f23f4c28740 1 -- 172.21.15.62:0/1028981455 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6874c4c30 con 0x55d6875d2300 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.573+0000 7f23eaa00640 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55d6875d2300 0x55d6875d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.574+0000 7f23eaa00640 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55d6875d2300 0x55d6875d4720 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbd066b5eecfa073 server_cookie=1b438ece739b50cf in_seq=0 out_seq=0 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.574+0000 7f23d4c00640 10 client.26066 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23d4c00640 1 -- 172.21.15.62:0/1028981455 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6874c4c30 con 0x55d6875d2300 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23d4c00640 10 client.26066 handle_command_reply: tid=0 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 1 client.26066 shutdown 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 1 -- 172.21.15.62:0/1028981455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d6875539f0 msgr2=0x55d68756c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d6875539f0 0x55d68756c980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f23dc027000 tx=0x7f23dc009fb0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 1 -- 172.21.15.62:0/1028981455 shutdown_connections 2024-04-02T14:45:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55d6875d2300 0x55d6875d4720 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d68755c080 0x55d6875d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d6875539f0 0x55d68756c980 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 1 --2- 172.21.15.62:0/1028981455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d687552d10 0x55d68756c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 1 -- 172.21.15.62:0/1028981455 >> 172.21.15.62:0/1028981455 conn(0x55d687519a50 msgr2=0x55d6875599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23d4c00640 0 client.26066 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 1 -- 172.21.15.62:0/1028981455 shutdown_connections 2024-04-02T14:45:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 1 -- 172.21.15.62:0/1028981455 wait complete. 2024-04-02T14:45:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.577+0000 7f23f4c28740 20 client.26066 trim_cache size 0 max 16384 2024-04-02T14:45:50.578 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:45:50.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 -- 172.21.15.62:0/2404321148 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23d00a3da0 msgr2=0x7f23d00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23d00a3da0 0x7f23d00a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23e40a53d0 tx=0x7f23e400efd0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 -- 172.21.15.62:0/2404321148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23ec19e8e0 msgr2=0x7f23ec19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:50.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23ec19e8e0 0x7f23ec19ed90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23e400eb10 tx=0x7f23e400c2d0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 -- 172.21.15.62:0/2404321148 shutdown_connections 2024-04-02T14:45:50.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23d00a3da0 0x7f23d00a6260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23ec19e8e0 0x7f23ec19ed90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ec10d3e0 0x7f23ec1a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 --2- 172.21.15.62:0/2404321148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23ec108310 0x7f23ec1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:50.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 -- 172.21.15.62:0/2404321148 >> 172.21.15.62:0/2404321148 conn(0x7f23ec076040 msgr2=0x7f23ec078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:50.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 -- 172.21.15.62:0/2404321148 shutdown_connections 2024-04-02T14:45:50.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:50.578+0000 7f23f2000640 1 -- 172.21.15.62:0/2404321148 wait complete. 2024-04-02T14:45:50.592 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-pmc.txt 2024-04-02T14:45:50.595 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-timer.txt 2024-04-02T14:45:50.598 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xen.txt 2024-04-02T14:45:50.600 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xilinx.txt 2024-04-02T14:45:50.603 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ 2024-04-02T14:45:50.606 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-platform.txt 2024-04-02T14:45:50.610 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-st.txt 2024-04-02T14:45:50.619 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/apm-xgene.txt 2024-04-02T14:45:50.633 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/atmel-at91_cf.txt 2024-04-02T14:45:50.636 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/cavium-compact-flash.txt 2024-04-02T14:45:50.637 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/exynos-sata.txt 2024-04-02T14:45:50.639 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/fsl-sata.txt 2024-04-02T14:45:50.641 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-pata.txt 2024-04-02T14:45:50.643 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-sata.txt 2024-04-02T14:45:50.645 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/marvell.txt 2024-04-02T14:45:50.647 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/pata-arasan.txt 2024-04-02T14:45:50.648 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/qcom-sata.txt 2024-04-02T14:45:50.650 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_highbank.txt 2024-04-02T14:45:50.652 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_rcar.txt 2024-04-02T14:45:50.654 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/tegra-sata.txt 2024-04-02T14:45:50.656 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/btmrvl.txt 2024-04-02T14:45:50.657 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ 2024-04-02T14:45:50.659 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/bcma.txt 2024-04-02T14:45:50.661 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/brcm,gisb-arb.txt 2024-04-02T14:45:50.662 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/imx-weim.txt 2024-04-02T14:45:50.664 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/mvebu-mbus.txt 2024-04-02T14:45:50.666 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/omap-ocp2scp.txt 2024-04-02T14:45:50.667 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ti-gpmc.txt 2024-04-02T14:45:50.669 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/ 2024-04-02T14:45:50.671 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/clocks.txt 2024-04-02T14:45:50.672 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/dscr.txt 2024-04-02T14:45:50.674 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/emifa.txt 2024-04-02T14:45:50.676 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/interrupt.txt 2024-04-02T14:45:50.678 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/soc.txt 2024-04-02T14:45:50.680 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/timer64.txt 2024-04-02T14:45:50.682 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/chosen.txt 2024-04-02T14:45:50.683 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ 2024-04-02T14:45:50.685 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/alphascale,acc.txt 2024-04-02T14:45:50.686 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/altr_socfpga.txt 2024-04-02T14:45:50.688 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/arm-integrator.txt 2024-04-02T14:45:50.691 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/at91-clock.txt 2024-04-02T14:45:50.691 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/axi-clkgen.txt 2024-04-02T14:45:50.693 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-cygnus-clock.txt 2024-04-02T14:45:50.695 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt 2024-04-02T14:45:50.697 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/calxeda.txt 2024-04-02T14:45:50.698 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-exynos-audss.txt 2024-04-02T14:45:50.700 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-palmas-clk32kg-clocks.txt 2024-04-02T14:45:50.702 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-s5pv210-audss.txt 2024-04-02T14:45:50.704 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clock-bindings.txt 2024-04-02T14:45:50.706 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clps711x-clock.txt 2024-04-02T14:45:50.707 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/efm32-clock.txt 2024-04-02T14:45:50.709 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/emev2-clock.txt 2024-04-02T14:45:50.710 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos3250-clock.txt 2024-04-02T14:45:50.712 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4-clock.txt 2024-04-02T14:45:50.714 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4415-clock.txt 2024-04-02T14:45:50.716 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5250-clock.txt 2024-04-02T14:45:50.718 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5260-clock.txt 2024-04-02T14:45:50.720 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5410-clock.txt 2024-04-02T14:45:50.722 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5420-clock.txt 2024-04-02T14:45:50.724 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5440-clock.txt 2024-04-02T14:45:50.725 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos7-clock.txt 2024-04-02T14:45:50.727 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-clock.txt 2024-04-02T14:45:50.729 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-factor-clock.txt 2024-04-02T14:45:50.730 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/gpio-gate-clock.txt 2024-04-02T14:45:50.732 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hi3620-clock.txt 2024-04-02T14:45:50.734 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt 2024-04-02T14:45:50.736 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx1-clock.txt 2024-04-02T14:45:50.737 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx21-clock.txt 2024-04-02T14:45:50.739 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx23-clock.txt 2024-04-02T14:45:50.741 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx25-clock.txt 2024-04-02T14:45:50.743 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx27-clock.txt 2024-04-02T14:45:50.745 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx28-clock.txt 2024-04-02T14:45:50.746 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx31-clock.txt 2024-04-02T14:45:50.749 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx35-clock.txt 2024-04-02T14:45:50.750 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx5-clock.txt 2024-04-02T14:45:50.752 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6q-clock.txt 2024-04-02T14:45:50.754 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sl-clock.txt 2024-04-02T14:45:50.756 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sx-clock.txt 2024-04-02T14:45:50.757 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-gate.txt 2024-04-02T14:45:50.759 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-pll.txt 2024-04-02T14:45:50.760 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt 2024-04-02T14:45:50.762 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,mmp2.txt 2024-04-02T14:45:50.765 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa168.txt 2024-04-02T14:45:50.766 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa910.txt 2024-04-02T14:45:50.768 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77686.txt 2024-04-02T14:45:50.770 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77802.txt 2024-04-02T14:45:50.772 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/moxa,moxart-clock.txt 2024-04-02T14:45:50.774 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt 2024-04-02T14:45:50.775 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-corediv-clock.txt 2024-04-02T14:45:50.776 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt 2024-04-02T14:45:50.778 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-gated-clock.txt 2024-04-02T14:45:50.780 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nspire-clock.txt 2024-04-02T14:45:50.782 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra114-car.txt 2024-04-02T14:45:50.784 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra124-car.txt 2024-04-02T14:45:50.786 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.txt 2024-04-02T14:45:50.787 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra30-car.txt 2024-04-02T14:45:50.789 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/prima2-clock.txt 2024-04-02T14:45:50.791 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/pxa-clock.txt 2024-04-02T14:45:50.793 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,gcc.txt 2024-04-02T14:45:50.794 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,lcc.txt 2024-04-02T14:45:50.796 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,mmcc.txt 2024-04-02T14:45:50.798 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qoriq-clock.txt 2024-04-02T14:45:50.800 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-div6-clocks.txt 2024-04-02T14:45:50.802 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt 2024-04-02T14:45:50.803 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a73a4-cpg-clocks.txt 2024-04-02T14:45:50.805 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt 2024-04-02T14:45:50.807 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt 2024-04-02T14:45:50.809 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rcar-gen2-cpg-clocks.txt 2024-04-02T14:45:50.810 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rz-cpg-clocks.txt 2024-04-02T14:45:50.812 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,sh73a0-cpg-clocks.txt 2024-04-02T14:45:50.814 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt 2024-04-02T14:45:50.816 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt 2024-04-02T14:45:50.818 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip.txt 2024-04-02T14:45:50.820 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2410-clock.txt 2024-04-02T14:45:50.821 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2412-clock.txt 2024-04-02T14:45:50.823 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2443-clock.txt 2024-04-02T14:45:50.825 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c64xx-clock.txt 2024-04-02T14:45:50.826 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt 2024-04-02T14:45:50.828 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si5351.txt 2024-04-02T14:45:50.830 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si570.txt 2024-04-02T14:45:50.832 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st,nomadik.txt 2024-04-02T14:45:50.834 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/ 2024-04-02T14:45:50.835 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-divmux.txt 2024-04-02T14:45:50.837 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-mux.txt 2024-04-02T14:45:50.839 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-pll.txt 2024-04-02T14:45:50.841 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-prediv.txt 2024-04-02T14:45:50.843 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-vcc.txt 2024-04-02T14:45:50.845 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen.txt 2024-04-02T14:45:50.847 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,flexgen.txt 2024-04-02T14:45:50.849 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,quadfs.txt 2024-04-02T14:45:50.851 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ste-u300-syscon-clock.txt 2024-04-02T14:45:50.853 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/sunxi.txt 2024-04-02T14:45:50.855 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti,cdce706.txt 2024-04-02T14:45:50.857 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti-keystone-pllctrl.txt 2024-04-02T14:45:50.858 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/ 2024-04-02T14:45:50.860 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/apll.txt 2024-04-02T14:45:50.862 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/autoidle.txt 2024-04-02T14:45:50.863 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/clockdomain.txt 2024-04-02T14:45:50.865 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/composite.txt 2024-04-02T14:45:50.867 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/divider.txt 2024-04-02T14:45:50.869 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dpll.txt 2024-04-02T14:45:50.870 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dra7-atl.txt 2024-04-02T14:45:50.872 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fapll.txt 2024-04-02T14:45:50.874 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fixed-factor-clock.txt 2024-04-02T14:45:50.875 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/gate.txt 2024-04-02T14:45:50.877 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/interface.txt 2024-04-02T14:45:50.879 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/mux.txt 2024-04-02T14:45:50.881 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vf610-clock.txt 2024-04-02T14:45:50.883 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vt8500.txt 2024-04-02T14:45:50.884 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/xgene.txt 2024-04-02T14:45:50.886 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/zynq-7000.txt 2024-04-02T14:45:50.888 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/ 2024-04-02T14:45:50.889 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/arm_big_little_dt.txt 2024-04-02T14:45:50.891 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-dt.txt 2024-04-02T14:45:50.893 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-exynos5440.txt 2024-04-02T14:45:50.894 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-spear.txt 2024-04-02T14:45:50.896 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/ 2024-04-02T14:45:50.897 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/amd-ccp.txt 2024-04-02T14:45:50.899 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/atmel-crypto.txt 2024-04-02T14:45:50.901 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-dcp.txt 2024-04-02T14:45:50.902 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-imx-sahara.txt 2024-04-02T14:45:50.904 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec2.txt 2024-04-02T14:45:50.906 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec4.txt 2024-04-02T14:45:50.908 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec6.txt 2024-04-02T14:45:50.909 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/mv_cesa.txt 2024-04-02T14:45:50.911 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-aes.txt 2024-04-02T14:45:50.912 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-des.txt 2024-04-02T14:45:50.914 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-sham.txt 2024-04-02T14:45:50.916 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/picochip-spacc.txt 2024-04-02T14:45:50.917 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/qcom-qce.txt 2024-04-02T14:45:50.920 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/samsung-sss.txt 2024-04-02T14:45:50.922 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/ 2024-04-02T14:45:50.923 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/ 2024-04-02T14:45:50.924 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/exynos-ppmu.txt 2024-04-02T14:45:50.926 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ 2024-04-02T14:45:50.928 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/arm-pl330.txt 2024-04-02T14:45:50.929 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-dma.txt 2024-04-02T14:45:50.930 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-xdma.txt 2024-04-02T14:45:50.932 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/bcm2835-dma.txt 2024-04-02T14:45:50.934 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/dma.txt 2024-04-02T14:45:50.935 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-edma.txt 2024-04-02T14:45:50.937 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-dma.txt 2024-04-02T14:45:50.939 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-sdma.txt 2024-04-02T14:45:50.940 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-mxs-dma.txt 2024-04-02T14:45:50.942 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/img-mdc-dma.txt 2024-04-02T14:45:50.946 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/k3dma.txt 2024-04-02T14:45:50.946 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mmp-dma.txt 2024-04-02T14:45:50.947 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/moxa,moxart-dma.txt 2024-04-02T14:45:50.949 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mpc512x-dma.txt 2024-04-02T14:45:50.951 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mv-xor.txt 2024-04-02T14:45:50.953 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/nbpfaxi.txt 2024-04-02T14:45:50.955 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_adm.txt 2024-04-02T14:45:50.956 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt 2024-04-02T14:45:50.958 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/rcar-audmapp.txt 2024-04-02T14:45:50.960 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt 2024-04-02T14:45:50.962 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/shdma.txt 2024-04-02T14:45:50.964 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sirfsoc-dma.txt 2024-04-02T14:45:50.965 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/snps-dma.txt 2024-04-02T14:45:50.967 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-coh901318.txt 2024-04-02T14:45:50.969 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-dma40.txt 2024-04-02T14:45:50.971 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sun6i-dma.txt 2024-04-02T14:45:50.972 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/tegra20-apbdma.txt 2024-04-02T14:45:50.974 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ti-edma.txt 2024-04-02T14:45:50.976 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/ 2024-04-02T14:45:50.977 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt 2024-04-02T14:45:50.984 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt 2024-04-02T14:45:50.984 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/ 2024-04-02T14:45:50.984 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/ 2024-04-02T14:45:50.985 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/marvell,dove-lcd.txt 2024-04-02T14:45:50.986 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/ 2024-04-02T14:45:50.988 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/hlcdc-dc.txt 2024-04-02T14:45:50.989 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/ 2024-04-02T14:45:50.992 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/dw_hdmi.txt 2024-04-02T14:45:50.993 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/ 2024-04-02T14:45:50.994 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/tda998x.txt 2024-04-02T14:45:50.996 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ 2024-04-02T14:45:50.997 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/fsl-imx-drm.txt 2024-04-02T14:45:50.999 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/hdmi.txt 2024-04-02T14:45:51.001 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ldb.txt 2024-04-02T14:45:51.002 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/ 2024-04-02T14:45:51.004 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/gpu.txt 2024-04-02T14:45:51.006 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/hdmi.txt 2024-04-02T14:45:51.007 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/mdp.txt 2024-04-02T14:45:51.009 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/ 2024-04-02T14:45:51.010 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/panel.txt 2024-04-02T14:45:51.012 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/slave.txt 2024-04-02T14:45:51.014 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tfp410.txt 2024-04-02T14:45:51.016 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tilcdc.txt 2024-04-02T14:45:51.017 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/eeprom.txt 2024-04-02T14:45:51.019 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/ 2024-04-02T14:45:51.021 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-palmas.txt 2024-04-02T14:45:51.022 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-rt8973a.txt 2024-04-02T14:45:51.024 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-sm5502.txt 2024-04-02T14:45:51.025 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/ 2024-04-02T14:45:51.027 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/mxsfb.txt 2024-04-02T14:45:51.029 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/sm501fb.txt 2024-04-02T14:45:51.030 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/ 2024-04-02T14:45:51.032 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/altera-socfpga-fpga-mgr.txt 2024-04-02T14:45:51.033 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/ 2024-04-02T14:45:51.035 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt 2024-04-02T14:45:51.037 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/ 2024-04-02T14:45:51.038 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/8xxx_gpio.txt 2024-04-02T14:45:51.040 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/abilis,tb10x-gpio.txt 2024-04-02T14:45:51.042 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cavium-octeon-gpio.txt 2024-04-02T14:45:51.043 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cirrus,clps711x-mctrl-gpio.txt 2024-04-02T14:45:51.046 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fsl-imx-gpio.txt 2024-04-02T14:45:51.048 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fujitsu,mb86s70-gpio.txt 2024-04-02T14:45:51.049 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74x164.txt 2024-04-02T14:45:51.051 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74xx-mmio.txt 2024-04-02T14:45:51.053 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-adnp.txt 2024-04-02T14:45:51.054 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-bcm-kona.txt 2024-04-02T14:45:51.056 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-clps711x.txt 2024-04-02T14:45:51.058 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-davinci.txt 2024-04-02T14:45:51.060 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-dsp-keystone.txt 2024-04-02T14:45:51.062 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-fan.txt 2024-04-02T14:45:51.063 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-grgpio.txt 2024-04-02T14:45:51.065 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-lp3943.txt 2024-04-02T14:45:51.067 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-max732x.txt 2024-04-02T14:45:51.069 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mcp23s08.txt 2024-04-02T14:45:51.070 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mm-lantiq.txt 2024-04-02T14:45:51.072 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-msm.txt 2024-04-02T14:45:51.074 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mvebu.txt 2024-04-02T14:45:51.075 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mxs.txt 2024-04-02T14:45:51.077 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-nmk.txt 2024-04-02T14:45:51.079 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-omap.txt 2024-04-02T14:45:51.081 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-palmas.txt 2024-04-02T14:45:51.082 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pca953x.txt 2024-04-02T14:45:51.086 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pcf857x.txt 2024-04-02T14:45:51.086 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-poweroff.txt 2024-04-02T14:45:51.088 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-restart.txt 2024-04-02T14:45:51.090 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-samsung.txt 2024-04-02T14:45:51.097 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stericsson-coh901.txt 2024-04-02T14:45:51.098 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stmpe.txt 2024-04-02T14:45:51.098 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stp-xway.txt 2024-04-02T14:45:51.098 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-sx150x.txt 2024-04-02T14:45:51.099 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-twl4030.txt 2024-04-02T14:45:51.100 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090-pdc.txt 2024-04-02T14:45:51.102 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090.txt 2024-04-02T14:45:51.104 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-vf610.txt 2024-04-02T14:45:51.105 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene-sb.txt 2024-04-02T14:45:51.108 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene.txt 2024-04-02T14:45:51.109 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xilinx.txt 2024-04-02T14:45:51.110 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zevio.txt 2024-04-02T14:45:51.112 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zynq.txt 2024-04-02T14:45:51.114 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio.txt 2024-04-02T14:45:51.116 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_atmel.txt 2024-04-02T14:45:51.117 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_lpc32xx.txt 2024-04-02T14:45:51.119 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/moxa,moxart-gpio.txt 2024-04-02T14:45:51.121 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/mrvl-gpio.txt 2024-04-02T14:45:51.123 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/nvidia,tegra20-gpio.txt 2024-04-02T14:45:51.124 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/pl061-gpio.txt 2024-04-02T14:45:51.126 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt 2024-04-02T14:45:51.128 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt 2024-04-02T14:45:51.130 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/sodaville.txt 2024-04-02T14:45:51.131 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/spear_spics.txt 2024-04-02T14:45:51.133 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/ 2024-04-02T14:45:51.134 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,gk20a.txt 2024-04-02T14:45:51.136 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,tegra20-host1x.txt 2024-04-02T14:45:51.138 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-g2d.txt 2024-04-02T14:45:51.140 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-rotator.txt 2024-04-02T14:45:51.142 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/st,stih4xx.txt 2024-04-02T14:45:51.143 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/graph.txt 2024-04-02T14:45:51.146 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/ 2024-04-02T14:45:51.147 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/hid-over-i2c.txt 2024-04-02T14:45:51.150 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/ 2024-04-02T14:45:51.151 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/client-devices.txt 2024-04-02T14:45:51.152 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/nokia-modem.txt 2024-04-02T14:45:51.154 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/omap-ssi.txt 2024-04-02T14:45:51.156 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ 2024-04-02T14:45:51.163 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ads1015.txt 2024-04-02T14:45:51.163 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/g762.txt 2024-04-02T14:45:51.163 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ibmpowernv.txt 2024-04-02T14:45:51.163 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/lm90.txt 2024-04-02T14:45:51.164 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ltc2978.txt 2024-04-02T14:45:51.166 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ntc_thermistor.txt 2024-04-02T14:45:51.168 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/pwm-fan.txt 2024-04-02T14:45:51.170 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/vexpress.txt 2024-04-02T14:45:51.171 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/ 2024-04-02T14:45:51.173 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/atmel-trng.txt 2024-04-02T14:45:51.174 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/omap_rng.txt 2024-04-02T14:45:51.176 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/timeriomem_rng.txt 2024-04-02T14:45:51.178 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ 2024-04-02T14:45:51.179 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,bcm2835-i2c.txt 2024-04-02T14:45:51.181 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,iproc-i2c.txt 2024-04-02T14:45:51.183 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt 2024-04-02T14:45:51.184 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-at91.txt 2024-04-02T14:45:51.186 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-axxia.txt 2024-04-02T14:45:51.188 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-bcm-kona.txt 2024-04-02T14:45:51.189 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cadence.txt 2024-04-02T14:45:51.191 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cbus-gpio.txt 2024-04-02T14:45:51.193 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt 2024-04-02T14:45:51.194 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-davinci.txt 2024-04-02T14:45:51.196 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-designware.txt 2024-04-02T14:45:51.198 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-efm32.txt 2024-04-02T14:45:51.200 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt 2024-04-02T14:45:51.201 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-gpio.txt 2024-04-02T14:45:51.203 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-hix5hd2.txt 2024-04-02T14:45:51.204 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-img-scb.txt 2024-04-02T14:45:51.206 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-imx.txt 2024-04-02T14:45:51.208 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-meson.txt 2024-04-02T14:45:51.210 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mpc.txt 2024-04-02T14:45:51.211 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-gpio.txt 2024-04-02T14:45:51.213 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pca954x.txt 2024-04-02T14:45:51.214 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pinctrl.txt 2024-04-02T14:45:51.216 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux.txt 2024-04-02T14:45:51.218 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt 2024-04-02T14:45:51.220 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mxs.txt 2024-04-02T14:45:51.222 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-nomadik.txt 2024-04-02T14:45:51.223 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-ocores.txt 2024-04-02T14:45:51.225 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-octeon.txt 2024-04-02T14:45:51.226 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-omap.txt 2024-04-02T14:45:51.228 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-opal.txt 2024-04-02T14:45:51.230 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pnx.txt 2024-04-02T14:45:51.232 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa-pci-ce4100.txt 2024-04-02T14:45:51.233 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa.txt 2024-04-02T14:45:51.235 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rcar.txt 2024-04-02T14:45:51.237 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-riic.txt 2024-04-02T14:45:51.239 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rk3x.txt 2024-04-02T14:45:51.240 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-s3c2410.txt 2024-04-02T14:45:51.242 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt 2024-04-02T14:45:51.244 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sirf.txt 2024-04-02T14:45:51.246 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st-ddci2c.txt 2024-04-02T14:45:51.248 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st.txt 2024-04-02T14:45:51.250 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sunxi-p2wi.txt 2024-04-02T14:45:51.251 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-versatile.txt 2024-04-02T14:45:51.253 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-vt8500.txt 2024-04-02T14:45:51.255 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-xiic.txt 2024-04-02T14:45:51.256 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina209.txt 2024-04-02T14:45:51.258 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina2xx.txt 2024-04-02T14:45:51.259 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/max6697.txt 2024-04-02T14:45:51.261 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/nvidia,tegra20-i2c.txt 2024-04-02T14:45:51.263 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/qcom,i2c-qup.txt 2024-04-02T14:45:51.264 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ti,bq32k.txt 2024-04-02T14:45:51.267 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/trivial-devices.txt 2024-04-02T14:45:51.268 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/ 2024-04-02T14:45:51.269 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/ 2024-04-02T14:45:51.271 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/bma180.txt 2024-04-02T14:45:51.272 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/ 2024-04-02T14:45:51.274 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/at91_adc.txt 2024-04-02T14:45:51.276 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/cc10001_adc.txt 2024-04-02T14:45:51.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/max1027-adc.txt 2024-04-02T14:45:51.279 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/nuvoton-nau7802.txt 2024-04-02T14:45:51.281 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-iadc.txt 2024-04-02T14:45:51.283 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-vadc.txt 2024-04-02T14:45:51.285 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt 2024-04-02T14:45:51.287 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/twl4030-madc.txt 2024-04-02T14:45:51.288 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/vf610-adc.txt 2024-04-02T14:45:51.290 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/xilinx-xadc.txt 2024-04-02T14:45:51.293 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ 2024-04-02T14:45:51.294 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ad7303.txt 2024-04-02T14:45:51.297 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/max5821.txt 2024-04-02T14:45:51.298 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/ 2024-04-02T14:45:51.299 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/adf4350.txt 2024-04-02T14:45:51.300 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/ 2024-04-02T14:45:51.302 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/dht11.txt 2024-04-02T14:45:51.303 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/iio-bindings.txt 2024-04-02T14:45:51.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/ 2024-04-02T14:45:51.307 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/apds9300.txt 2024-04-02T14:45:51.308 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/cm36651.txt 2024-04-02T14:45:51.310 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/gp2ap020a00f.txt 2024-04-02T14:45:51.312 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/tsl2563.txt 2024-04-02T14:45:51.313 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ 2024-04-02T14:45:51.315 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt 2024-04-02T14:45:51.316 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/hmc5843.txt 2024-04-02T14:45:51.318 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/ 2024-04-02T14:45:51.320 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/as3935.txt 2024-04-02T14:45:51.321 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/sensorhub.txt 2024-04-02T14:45:51.323 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/st-sensors.txt 2024-04-02T14:45:51.325 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ 2024-04-02T14:45:51.327 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ads7846.txt 2024-04-02T14:45:51.328 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/atmel,maxtouch.txt 2024-04-02T14:45:51.330 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cap11xx.txt 2024-04-02T14:45:51.332 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/clps711x-keypad.txt 2024-04-02T14:45:51.333 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cros-ec-keyb.txt 2024-04-02T14:45:51.335 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/e3x0-button.txt 2024-04-02T14:45:51.337 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elan_i2c.txt 2024-04-02T14:45:51.338 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elants_i2c.txt 2024-04-02T14:45:51.340 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/fsl-mma8450.txt 2024-04-02T14:45:51.342 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-beeper.txt 2024-04-02T14:45:51.344 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys-polled.txt 2024-04-02T14:45:51.345 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys.txt 2024-04-02T14:45:51.347 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-matrix-keypad.txt 2024-04-02T14:45:51.349 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/imx-keypad.txt 2024-04-02T14:45:51.350 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/input-reset.txt 2024-04-02T14:45:51.352 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/lpc32xx-key.txt 2024-04-02T14:45:51.354 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/matrix-keymap.txt 2024-04-02T14:45:51.356 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/nvidia,tegra20-kbc.txt 2024-04-02T14:45:51.357 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/omap-keypad.txt 2024-04-02T14:45:51.359 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ps2keyb-mouse-apbps2.txt 2024-04-02T14:45:51.360 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pwm-beeper.txt 2024-04-02T14:45:51.362 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pxa27x-keypad.txt 2024-04-02T14:45:51.364 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-keypad.txt 2024-04-02T14:45:51.365 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-pwrkey.txt 2024-04-02T14:45:51.367 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-vib.txt 2024-04-02T14:45:51.369 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/regulator-haptic.txt 2024-04-02T14:45:51.371 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/rotary-encoder.txt 2024-04-02T14:45:51.372 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/samsung-keypad.txt 2024-04-02T14:45:51.374 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/spear-keyboard.txt 2024-04-02T14:45:51.376 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/st-keyscan.txt 2024-04-02T14:45:51.377 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/stmpe-keypad.txt 2024-04-02T14:45:51.379 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/sun4i-lradc-keys.txt 2024-04-02T14:45:51.381 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tca8418_keypad.txt 2024-04-02T14:45:51.382 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv260x.txt 2024-04-02T14:45:51.384 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv2667.txt 2024-04-02T14:45:51.386 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,nspire-keypad.txt 2024-04-02T14:45:51.387 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,palmas-pwrbutton.txt 2024-04-02T14:45:51.389 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ 2024-04-02T14:45:51.391 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/auo_pixcir_ts.txt 2024-04-02T14:45:51.392 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/bu21013.txt 2024-04-02T14:45:51.394 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/edt-ft5x06.txt 2024-04-02T14:45:51.395 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/egalax-ts.txt 2024-04-02T14:45:51.397 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/lpc32xx-tsc.txt 2024-04-02T14:45:51.399 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/mms114.txt 2024-04-02T14:45:51.400 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/pixcir_i2c_ts.txt 2024-04-02T14:45:51.402 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sitronix-st1232.txt 2024-04-02T14:45:51.404 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/stmpe.txt 2024-04-02T14:45:51.405 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt 2024-04-02T14:45:51.407 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ti-tsc-adc.txt 2024-04-02T14:45:51.409 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt 2024-04-02T14:45:51.410 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt 2024-04-02T14:45:51.412 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2007.txt 2024-04-02T14:45:51.414 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/zforce_ts.txt 2024-04-02T14:45:51.416 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tps65218-pwrbutton.txt 2024-04-02T14:45:51.417 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-keypad.txt 2024-04-02T14:45:51.419 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-pwrbutton.txt 2024-04-02T14:45:51.420 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ 2024-04-02T14:45:51.422 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/abilis,tb10x-ictl.txt 2024-04-02T14:45:51.424 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun4i-ic.txt 2024-04-02T14:45:51.426 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun67i-sc-nmi.txt 2024-04-02T14:45:51.428 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/atmel,aic.txt 2024-04-02T14:45:51.429 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt 2024-04-02T14:45:51.431 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm7120-l2-intc.txt 2024-04-02T14:45:51.433 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,l2-intc.txt 2024-04-02T14:45:51.435 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/cirrus,clps711x-intc.txt 2024-04-02T14:45:51.436 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/digicolor-ic.txt 2024-04-02T14:45:51.438 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/interrupts.txt 2024-04-02T14:45:51.440 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/lsi,zevio-intc.txt 2024-04-02T14:45:51.442 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt 2024-04-02T14:45:51.443 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,orion-intc.txt 2024-04-02T14:45:51.445 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/mips-gic.txt 2024-04-02T14:45:51.447 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/opencores,or1k-pic.txt 2024-04-02T14:45:51.448 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,intc-irqpin.txt 2024-04-02T14:45:51.450 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.txt 2024-04-02T14:45:51.452 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/samsung,s3c24xx-irq.txt 2024-04-02T14:45:51.454 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt 2024-04-02T14:45:51.456 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,keystone-irq.txt 2024-04-02T14:45:51.458 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,omap-intc-irq.txt 2024-04-02T14:45:51.460 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ 2024-04-02T14:45:51.462 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/arm,smmu.txt 2024-04-02T14:45:51.463 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/iommu.txt 2024-04-02T14:45:51.465 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra20-gart.txt 2024-04-02T14:45:51.467 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra30-smmu.txt 2024-04-02T14:45:51.469 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/renesas,ipmmu-vmsa.txt 2024-04-02T14:45:51.470 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/rockchip,iommu.txt 2024-04-02T14:45:51.472 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt 2024-04-02T14:45:51.474 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ti,omap-iommu.txt 2024-04-02T14:45:51.477 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/ 2024-04-02T14:45:51.478 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/common.txt 2024-04-02T14:45:51.479 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-gpio.txt 2024-04-02T14:45:51.481 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp55xx.txt 2024-04-02T14:45:51.483 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp8860.txt 2024-04-02T14:45:51.484 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-ns2.txt 2024-04-02T14:45:51.486 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-pwm.txt 2024-04-02T14:45:51.488 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/pca963x.txt 2024-04-02T14:45:51.489 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/register-bit-led.txt 2024-04-02T14:45:51.492 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/tca6507.txt 2024-04-02T14:45:51.493 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/ 2024-04-02T14:45:51.494 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2-timings.txt 2024-04-02T14:45:51.496 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2.txt 2024-04-02T14:45:51.498 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/ 2024-04-02T14:45:51.499 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/altera-mailbox.txt 2024-04-02T14:45:51.501 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/mailbox.txt 2024-04-02T14:45:51.503 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt 2024-04-02T14:45:51.505 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/marvell.txt 2024-04-02T14:45:51.507 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ 2024-04-02T14:45:51.508 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/atmel-isi.txt 2024-04-02T14:45:51.510 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/coda.txt 2024-04-02T14:45:51.512 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-fimc-lite.txt 2024-04-02T14:45:51.513 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-jpeg-codec.txt 2024-04-02T14:45:51.515 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos4-fimc-is.txt 2024-04-02T14:45:51.517 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos5-gsc.txt 2024-04-02T14:45:51.518 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/gpio-ir-receiver.txt 2024-04-02T14:45:51.520 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/hix5hd2-ir.txt 2024-04-02T14:45:51.522 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ 2024-04-02T14:45:51.523 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7343.txt 2024-04-02T14:45:51.525 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7604.txt 2024-04-02T14:45:51.526 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9m111.txt 2024-04-02T14:45:51.528 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9p031.txt 2024-04-02T14:45:51.530 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/nokia,smia.txt 2024-04-02T14:45:51.531 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ths8200.txt 2024-04-02T14:45:51.533 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp514x.txt 2024-04-02T14:45:51.535 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp7002.txt 2024-04-02T14:45:51.536 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/img-ir-rev1.txt 2024-04-02T14:45:51.538 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/meson-ir.txt 2024-04-02T14:45:51.540 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/pxa-camera.txt 2024-04-02T14:45:51.542 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/rcar_vin.txt 2024-04-02T14:45:51.543 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/renesas,vsp1.txt 2024-04-02T14:45:51.545 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/s5p-mfc.txt 2024-04-02T14:45:51.547 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-fimc.txt 2024-04-02T14:45:51.550 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-mipi-csis.txt 2024-04-02T14:45:51.551 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5c73m3.txt 2024-04-02T14:45:51.553 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k5baf.txt 2024-04-02T14:45:51.556 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k6a3.txt 2024-04-02T14:45:51.560 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sh_mobile_ceu.txt 2024-04-02T14:45:51.564 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/si4713.txt 2024-04-02T14:45:51.568 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/st-rc.txt 2024-04-02T14:45:51.571 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sunxi-ir.txt 2024-04-02T14:45:51.575 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ti-am437x-vpfe.txt 2024-04-02T14:45:51.578 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/video-interfaces.txt 2024-04-02T14:45:51.581 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ 2024-04-02T14:45:51.585 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ 2024-04-02T14:45:51.588 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt 2024-04-02T14:45:51.591 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-devbus.txt 2024-04-02T14:45:51.591 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:45:51.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:45:51.592 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:45:51.600 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-sdram-controller.txt 2024-04-02T14:45:51.609 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt 2024-04-02T14:45:51.642 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/renesas-memory-controllers.txt 2024-04-02T14:45:51.684 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/synopsys.txt 2024-04-02T14:45:51.706 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti-aemif.txt 2024-04-02T14:45:51.733 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/ 2024-04-02T14:45:51.750 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/emif.txt 2024-04-02T14:45:51.751 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/ 2024-04-02T14:45:51.752 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta-intc.txt 2024-04-02T14:45:51.753 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta.txt 2024-04-02T14:45:51.754 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/pdc-intc.txt 2024-04-02T14:45:51.755 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ 2024-04-02T14:45:51.757 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/88pm860x.txt 2024-04-02T14:45:51.758 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ab8500.txt 2024-04-02T14:45:51.759 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/arizona.txt 2024-04-02T14:45:51.760 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3711.txt 2024-04-02T14:45:51.761 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3722.txt 2024-04-02T14:45:51.762 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-gpbr.txt 2024-04-02T14:45:51.763 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-hlcdc.txt 2024-04-02T14:45:51.765 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-matrix.txt 2024-04-02T14:45:51.766 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-smc.txt 2024-04-02T14:45:51.767 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bcm590xx.txt 2024-04-02T14:45:51.768 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bfticu.txt 2024-04-02T14:45:51.769 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/cros-ec.txt 2024-04-02T14:45:51.770 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9052-i2c.txt 2024-04-02T14:45:51.771 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9055.txt 2024-04-02T14:45:51.772 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9063.txt 2024-04-02T14:45:51.773 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/hi6421.txt 2024-04-02T14:45:51.775 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/lp3943.txt 2024-04-02T14:45:51.776 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max14577.txt 2024-04-02T14:45:51.777 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77686.txt 2024-04-02T14:45:51.778 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77693.txt 2024-04-02T14:45:51.779 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8925.txt 2024-04-02T14:45:51.780 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8998.txt 2024-04-02T14:45:51.781 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/mc13xxx.txt 2024-04-02T14:45:51.783 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-host.txt 2024-04-02T14:45:51.784 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-tll.txt 2024-04-02T14:45:51.785 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/palmas.txt 2024-04-02T14:45:51.786 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom,spmi-pmic.txt 2024-04-02T14:45:51.788 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-pm8xxx.txt 2024-04-02T14:45:51.789 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-rpm.txt 2024-04-02T14:45:51.790 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qriox.txt 2024-04-02T14:45:51.791 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rk808.txt 2024-04-02T14:45:51.792 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rn5t618.txt 2024-04-02T14:45:51.794 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mpa01.txt 2024-04-02T14:45:51.795 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mps11.txt 2024-04-02T14:45:51.796 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/stmpe.txt 2024-04-02T14:45:51.797 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt 2024-04-02T14:45:51.798 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/syscon.txt 2024-04-02T14:45:51.799 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tc3589x.txt 2024-04-02T14:45:51.800 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt 2024-04-02T14:45:51.801 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps6507x.txt 2024-04-02T14:45:51.803 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps65910.txt 2024-04-02T14:45:51.804 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl-familly.txt 2024-04-02T14:45:51.805 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-audio.txt 2024-04-02T14:45:51.806 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-power.txt 2024-04-02T14:45:51.807 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl6040.txt 2024-04-02T14:45:51.808 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/ 2024-04-02T14:45:51.810 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/ 2024-04-02T14:45:51.811 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt 2024-04-02T14:45:51.812 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/nvidia,tegra114-mipi.txt 2024-04-02T14:45:51.813 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ 2024-04-02T14:45:51.814 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/ 2024-04-02T14:45:51.815 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bcm3384-intc.txt 2024-04-02T14:45:51.816 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bmips.txt 2024-04-02T14:45:51.817 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/cm-dsl.txt 2024-04-02T14:45:51.818 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/usb.txt 2024-04-02T14:45:51.820 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ 2024-04-02T14:45:51.821 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/bootbus.txt 2024-04-02T14:45:51.822 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/cib.txt 2024-04-02T14:45:51.824 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu.txt 2024-04-02T14:45:51.825 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu2.txt 2024-04-02T14:45:51.827 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/dma-engine.txt 2024-04-02T14:45:51.829 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/uctl.txt 2024-04-02T14:45:51.830 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cpu_irq.txt 2024-04-02T14:45:51.831 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ralink.txt 2024-04-02T14:45:51.833 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ 2024-04-02T14:45:51.834 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/allwinner,sunxi-sid.txt 2024-04-02T14:45:51.836 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/arm-charlcd.txt 2024-04-02T14:45:51.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:51 smithi062 ceph-mon[32116]: pgmap v58: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-04-02T14:45:51.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:51 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:45:51.838 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/at25.txt 2024-04-02T14:45:51.839 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/atmel-ssc.txt 2024-04-02T14:45:51.841 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/bmp085.txt 2024-04-02T14:45:51.843 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt 2024-04-02T14:45:51.844 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ifm-csi.txt 2024-04-02T14:45:51.846 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/lis302.txt 2024-04-02T14:45:51.848 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt 2024-04-02T14:45:51.849 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/smc.txt 2024-04-02T14:45:51.851 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/sram.txt 2024-04-02T14:45:51.852 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ti,dac7512.txt 2024-04-02T14:45:51.854 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ 2024-04-02T14:45:51.855 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/arasan,sdhci.txt 2024-04-02T14:45:51.857 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/atmel-hsmci.txt 2024-04-02T14:45:51.859 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/brcm,bcm2835-sdhci.txt 2024-04-02T14:45:51.863 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/davinci_mmc.txt 2024-04-02T14:45:51.864 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/exynos-dw-mshc.txt 2024-04-02T14:45:51.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:51 smithi113 ceph-mon[36135]: pgmap v58: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-04-02T14:45:51.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:51 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:45:51.866 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-esdhc.txt 2024-04-02T14:45:51.868 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt 2024-04-02T14:45:51.870 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-mmc.txt 2024-04-02T14:45:51.871 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/img-dw-mshc.txt 2024-04-02T14:45:51.872 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/k3-dw-mshc.txt 2024-04-02T14:45:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.878+0000 7f679a800640 1 -- 172.21.15.62:0/2362999303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6794107aa0 msgr2=0x7f6794107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.879+0000 7f679a800640 1 --2- 172.21.15.62:0/2362999303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6794107aa0 0x7f6794107e80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6784009900 tx=0x7f678402bc30 comp rx=0 tx=0).stop 2024-04-02T14:45:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.879+0000 7f679a800640 1 -- 172.21.15.62:0/2362999303 shutdown_connections 2024-04-02T14:45:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.879+0000 7f679a800640 1 --2- 172.21.15.62:0/2362999303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6794102450 0x7f679410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.879+0000 7f679a800640 1 --2- 172.21.15.62:0/2362999303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794101a90 0x7f6794101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.879+0000 7f679a800640 1 --2- 172.21.15.62:0/2362999303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6794107aa0 0x7f6794107e80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.879+0000 7f679a800640 1 -- 172.21.15.62:0/2362999303 >> 172.21.15.62:0/2362999303 conn(0x7f6794078f00 msgr2=0x7f6794079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.879+0000 7f679a800640 1 -- 172.21.15.62:0/2362999303 shutdown_connections 2024-04-02T14:45:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.879+0000 7f679a800640 1 -- 172.21.15.62:0/2362999303 wait complete. 2024-04-02T14:45:51.875 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/kona-sdhci.txt 2024-04-02T14:45:51.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.880+0000 7f679a800640 1 Processor -- start 2024-04-02T14:45:51.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.880+0000 7f679a800640 1 -- start start 2024-04-02T14:45:51.876 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt 2024-04-02T14:45:51.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.880+0000 7f679a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6794101a90 0x7f67941984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.880+0000 7f679a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794102450 0x7f6794198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.880+0000 7f679a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6794107aa0 0x7f679419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.880+0000 7f679a800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6794111fa0 con 0x7f6794102450 2024-04-02T14:45:51.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.880+0000 7f679a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6794111e20 con 0x7f6794107aa0 2024-04-02T14:45:51.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.880+0000 7f679a800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6794112120 con 0x7f6794101a90 2024-04-02T14:45:51.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.880+0000 7f6792a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794102450 0x7f6794198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6793e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6794107aa0 0x7f679419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6792a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794102450 0x7f6794198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35738/0 (socket says 172.21.15.62:35738) 2024-04-02T14:45:51.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6792a00640 1 -- 172.21.15.62:0/239400269 learned_addr learned my addr 172.21.15.62:0/239400269 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:51.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6793e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6794107aa0 0x7f679419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:48984/0 (socket says 172.21.15.62:48984) 2024-04-02T14:45:51.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6793400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6794101a90 0x7f67941984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6793e00640 1 -- 172.21.15.62:0/239400269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6794101a90 msgr2=0x7f67941984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:51.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6793e00640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6794101a90 0x7f67941984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6793e00640 1 -- 172.21.15.62:0/239400269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794102450 msgr2=0x7f6794198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:51.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6793e00640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794102450 0x7f6794198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6793e00640 1 -- 172.21.15.62:0/239400269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679419d460 con 0x7f6794107aa0 2024-04-02T14:45:51.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6792a00640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794102450 0x7f6794198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:51.878 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-simple.txt 2024-04-02T14:45:51.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.881+0000 7f6793e00640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6794107aa0 0x7f679419cd30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f678c00da10 tx=0x7f678c00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:51.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.882+0000 7f678be00640 1 -- 172.21.15.62:0/239400269 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c00bd70 con 0x7f6794107aa0 2024-04-02T14:45:51.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.882+0000 7f678be00640 1 -- 172.21.15.62:0/239400269 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f678c004030 con 0x7f6794107aa0 2024-04-02T14:45:51.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.882+0000 7f679a800640 1 -- 172.21.15.62:0/239400269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679419d750 con 0x7f6794107aa0 2024-04-02T14:45:51.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.882+0000 7f678be00640 1 -- 172.21.15.62:0/239400269 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c0107e0 con 0x7f6794107aa0 2024-04-02T14:45:51.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.882+0000 7f679a800640 1 -- 172.21.15.62:0/239400269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67941a4fe0 con 0x7f6794107aa0 2024-04-02T14:45:51.880 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-spi-slot.txt 2024-04-02T14:45:51.880 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc.txt 2024-04-02T14:45:51.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.885+0000 7f678be00640 1 -- 172.21.15.62:0/239400269 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f678c0041e0 con 0x7f6794107aa0 2024-04-02T14:45:51.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.885+0000 7f679d5b7740 1 Processor -- start 2024-04-02T14:45:51.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f679d5b7740 1 -- start start 2024-04-02T14:45:51.881 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmci.txt 2024-04-02T14:45:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 0x55f68ddced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddcf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd86c0 0x55f68dddaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f679d5b7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f68dd33310 con 0x55f68ddd0030 2024-04-02T14:45:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f679d5b7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f68dcdb690 con 0x55f68ddcf350 2024-04-02T14:45:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f679d5b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f68dd2f2c0 con 0x55f68ddd86c0 2024-04-02T14:45:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f6792a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 0x55f68ddced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f6793400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddcf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f6792a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 0x55f68ddced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35748/0 (socket says 172.21.15.62:35748) 2024-04-02T14:45:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f6793400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddcf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:49000/0 (socket says 172.21.15.62:49000) 2024-04-02T14:45:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.886+0000 7f6792a00640 1 -- 172.21.15.62:0/1464373830 learned_addr learned my addr 172.21.15.62:0/1464373830 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.887+0000 7f6792a00640 1 --2- 172.21.15.62:0/1464373830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd86c0 0x55f68dddaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.887+0000 7f6793400640 1 -- 172.21.15.62:0/1464373830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd86c0 msgr2=0x55f68dddaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.887+0000 7f6793400640 1 --2- 172.21.15.62:0/1464373830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd86c0 0x55f68dddaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.887+0000 7f6793400640 1 -- 172.21.15.62:0/1464373830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 msgr2=0x55f68ddced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.887+0000 7f6793400640 1 --2- 172.21.15.62:0/1464373830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 0x55f68ddced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.887+0000 7f6793400640 1 -- 172.21.15.62:0/1464373830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68db0f7e0 con 0x55f68ddcf350 2024-04-02T14:45:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.887+0000 7f6792a00640 1 --2- 172.21.15.62:0/1464373830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 0x55f68ddced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.887+0000 7f678be00640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f67780a3da0 0x7f67780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.887+0000 7f678be00640 1 -- 172.21.15.62:0/239400269 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f678c0d0a40 con 0x7f6794107aa0 2024-04-02T14:45:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.887+0000 7f6792a00640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f67780a3da0 0x7f67780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.886 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt 2024-04-02T14:45:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.887+0000 7f6793400640 1 --2- 172.21.15.62:0/1464373830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddcf7d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6784009230 tx=0x7f678402c610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d69f06876a52a64e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.888+0000 7f6788c00640 1 -- 172.21.15.62:0/1464373830 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678402c9c0 con 0x55f68ddcf350 2024-04-02T14:45:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.888+0000 7f6788c00640 1 -- 172.21.15.62:0/1464373830 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f678402cb60 con 0x55f68ddcf350 2024-04-02T14:45:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.888+0000 7f6788c00640 1 -- 172.21.15.62:0/1464373830 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67840046c0 con 0x55f68ddcf350 2024-04-02T14:45:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.888+0000 7f679d5b7740 1 -- 172.21.15.62:0/1464373830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 msgr2=0x55f68ddcf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.888+0000 7f679d5b7740 1 --2- 172.21.15.62:0/1464373830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddcf7d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6784009230 tx=0x7f678402c610 comp rx=0 tx=0).stop 2024-04-02T14:45:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.888+0000 7f6792a00640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f67780a3da0 0x7f67780a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6780015c50 tx=0x7f6780009d00 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.889+0000 7f679d5b7740 1 -- 172.21.15.62:0/1464373830 shutdown_connections 2024-04-02T14:45:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.889+0000 7f679d5b7740 1 --2- 172.21.15.62:0/1464373830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd86c0 0x55f68dddaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.889+0000 7f679d5b7740 1 --2- 172.21.15.62:0/1464373830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddcf7d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.889+0000 7f679d5b7740 1 --2- 172.21.15.62:0/1464373830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 0x55f68ddced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.889+0000 7f679d5b7740 1 -- 172.21.15.62:0/1464373830 >> 172.21.15.62:0/1464373830 conn(0x55f68dd63270 msgr2=0x55f68ddc9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.889+0000 7f679d5b7740 1 -- 172.21.15.62:0/1464373830 shutdown_connections 2024-04-02T14:45:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.889+0000 7f679d5b7740 1 -- 172.21.15.62:0/1464373830 wait complete. 2024-04-02T14:45:51.890 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mxs-mmc.txt 2024-04-02T14:45:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.889+0000 7f679d5b7740 1 -- start start 2024-04-02T14:45:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.889+0000 7f679d5b7740 1 Processor -- start 2024-04-02T14:45:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddd5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 0x55f68ddd5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd86c0 0x55f68ddd60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f679d5b7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f68dddb4d0 con 0x55f68ddd0030 2024-04-02T14:45:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f679d5b7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f68dceea60 con 0x55f68ddcf350 2024-04-02T14:45:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f679d5b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f68dddb650 con 0x55f68ddd86c0 2024-04-02T14:45:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f679d5b7740 10 client.0 _ll_register_callbacks cb 0x55f68dcf4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f6793400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddd5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f6793400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddd5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:49010/0 (socket says 172.21.15.62:49010) 2024-04-02T14:45:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f6793400640 1 -- 172.21.15.62:0/2404997373 learned_addr learned my addr 172.21.15.62:0/2404997373 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:51.892 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt 2024-04-02T14:45:51.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f6793e00640 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd86c0 0x55f68ddd60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.890+0000 7f6792a00640 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 0x55f68ddd5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.891+0000 7f6793400640 1 -- 172.21.15.62:0/2404997373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd86c0 msgr2=0x55f68ddd60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:51.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.891+0000 7f6793400640 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd86c0 0x55f68ddd60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.891+0000 7f6793400640 1 -- 172.21.15.62:0/2404997373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 msgr2=0x55f68ddd5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:51.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.891+0000 7f6793400640 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 0x55f68ddd5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.891+0000 7f6793400640 1 -- 172.21.15.62:0/2404997373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f68db6ff70 con 0x55f68ddcf350 2024-04-02T14:45:51.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.891+0000 7f6792a00640 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 0x55f68ddd5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:51.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.891+0000 7f6793400640 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddd5630 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f678402c170 tx=0x7f678402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc1a0a76d2e54d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:51.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.891+0000 7f677d600640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:51.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.891+0000 7f677d600640 1 -- 172.21.15.62:0/2404997373 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678402c9c0 con 0x55f68ddcf350 2024-04-02T14:45:51.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.891+0000 7f677d600640 1 -- 172.21.15.62:0/2404997373 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f67840035f0 con 0x55f68ddcf350 2024-04-02T14:45:51.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.891+0000 7f679d5b7740 1 -- 172.21.15.62:0/2404997373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f678402cb60 con 0x55f68ddcf350 2024-04-02T14:45:51.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.892+0000 7f677d600640 1 -- 172.21.15.62:0/2404997373 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6784059050 con 0x55f68ddcf350 2024-04-02T14:45:51.895 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/orion-sdio.txt 2024-04-02T14:45:51.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.892+0000 7f679d5b7740 10 client.26078 fetch_fsmap learned FSMap version 29 2024-04-02T14:45:51.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.892+0000 7f679d5b7740 1 -- 172.21.15.62:0/2404997373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f68da07170 con 0x55f68ddcf350 2024-04-02T14:45:51.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.892+0000 7f677d600640 1 -- 172.21.15.62:0/2404997373 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f678402cb60 con 0x55f68ddcf350 2024-04-02T14:45:51.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.892+0000 7f677d600640 1 -- 172.21.15.62:0/2404997373 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f6784034c60 con 0x55f68ddcf350 2024-04-02T14:45:51.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.892+0000 7f679d5b7740 10 client.26078 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T14:45:51.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.892+0000 7f679d5b7740 10 client.26078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:51.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.892+0000 7f679d5b7740 20 client.26078 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:51.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.892+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f68dde8950 0x55f68ddead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.892+0000 7f679d5b7740 4 client.26078 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:51.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.892+0000 7f679d5b7740 1 -- 172.21.15.62:0/2404997373 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f68dd40850 con 0x55f68dde8950 2024-04-02T14:45:51.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.893+0000 7f6792a00640 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f68dde8950 0x55f68ddead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.893+0000 7f6792a00640 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f68dde8950 0x55f68ddead40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73e1855de67124b server_cookie=4ac9ebf6f21eedb6 in_seq=0 out_seq=0 2024-04-02T14:45:51.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.893+0000 7f677d600640 10 client.26078 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:51.898 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/pxa-mmc.txt 2024-04-02T14:45:51.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.895+0000 7f677d600640 1 -- 172.21.15.62:0/2404997373 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f68dd40850 con 0x55f68dde8950 2024-04-02T14:45:51.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.895+0000 7f677d600640 10 client.26078 handle_command_reply: tid=0 2024-04-02T14:45:51.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.895+0000 7f679d5b7740 1 client.26078 shutdown 2024-04-02T14:45:51.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.895+0000 7f679d5b7740 1 -- 172.21.15.62:0/2404997373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 msgr2=0x55f68ddd5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:51.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.895+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddd5630 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f678402c170 tx=0x7f678402bc30 comp rx=0 tx=0).stop 2024-04-02T14:45:51.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.895+0000 7f679d5b7740 1 -- 172.21.15.62:0/2404997373 shutdown_connections 2024-04-02T14:45:51.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.896+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f68dde8950 0x55f68ddead40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.896+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd86c0 0x55f68ddd60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.896+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd0030 0x55f68ddd5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.896+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2404997373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddcf350 0x55f68ddd5630 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.896+0000 7f679d5b7740 1 -- 172.21.15.62:0/2404997373 >> 172.21.15.62:0/2404997373 conn(0x55f68dd63270 msgr2=0x55f68ddcbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:51.900 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/renesas,mmcif.txt 2024-04-02T14:45:51.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.896+0000 7f677d600640 0 client.26078 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:51.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.896+0000 7f679d5b7740 1 -- 172.21.15.62:0/2404997373 shutdown_connections 2024-04-02T14:45:51.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.896+0000 7f679d5b7740 1 -- 172.21.15.62:0/2404997373 wait complete. 2024-04-02T14:45:51.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.896+0000 7f679d5b7740 20 client.26078 trim_cache size 0 max 16384 2024-04-02T14:45:51.902 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.txt 2024-04-02T14:45:51.902 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/samsung-sdhci.txt 2024-04-02T14:45:51.902 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-dove.txt 2024-04-02T14:45:51.902 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-fujitsu.txt 2024-04-02T14:45:51.903 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-msm.txt 2024-04-02T14:45:51.903 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-pxa.txt 2024-04-02T14:45:51.903 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-sirf.txt 2024-04-02T14:45:51.903 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-spear.txt 2024-04-02T14:45:51.904 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-st.txt 2024-04-02T14:45:51.906 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/socfpga-dw-mshc.txt 2024-04-02T14:45:51.906 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt 2024-04-02T14:45:51.907 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt 2024-04-02T14:45:51.908 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap-hsmmc.txt 2024-04-02T14:45:51.910 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap.txt 2024-04-02T14:45:51.911 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/tmio_mmc.txt 2024-04-02T14:45:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.917+0000 7f679d5b7740 1 Processor -- start 2024-04-02T14:45:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.917+0000 7f679d5b7740 1 -- start start 2024-04-02T14:45:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.917+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddcf350 0x55f68ddd7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.917+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd0030 0x55f68dde4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.917+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd86c0 0x55f68ddf0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.917+0000 7f679d5b7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f67941af900 con 0x55f68ddcf350 2024-04-02T14:45:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.917+0000 7f679d5b7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67941b1470 con 0x55f68ddd86c0 2024-04-02T14:45:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.917+0000 7f679d5b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67941affd0 con 0x55f68ddd0030 2024-04-02T14:45:51.913 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt 2024-04-02T14:45:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.918+0000 7f6793400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddcf350 0x55f68ddd7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.918+0000 7f6793e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd86c0 0x55f68ddf0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.918+0000 7f6793e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd86c0 0x55f68ddf0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:49022/0 (socket says 172.21.15.62:49022) 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.918+0000 7f6793e00640 1 -- 172.21.15.62:0/1844754883 learned_addr learned my addr 172.21.15.62:0/1844754883 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.918+0000 7f6792a00640 1 --2- 172.21.15.62:0/1844754883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd0030 0x55f68dde4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.918+0000 7f6793e00640 1 -- 172.21.15.62:0/1844754883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd0030 msgr2=0x55f68dde4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.918+0000 7f6793e00640 1 --2- 172.21.15.62:0/1844754883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd0030 0x55f68dde4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.918+0000 7f6793e00640 1 -- 172.21.15.62:0/1844754883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddcf350 msgr2=0x55f68ddd7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.918+0000 7f6793e00640 1 --2- 172.21.15.62:0/1844754883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddcf350 0x55f68ddd7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.918+0000 7f6793e00640 1 -- 172.21.15.62:0/1844754883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68ddce380 con 0x55f68ddd86c0 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.918+0000 7f6793e00640 1 --2- 172.21.15.62:0/1844754883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd86c0 0x55f68ddf0340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f678c09c6d0 tx=0x7f678c0bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd39827f406a18b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.919+0000 7f677fe00640 1 -- 172.21.15.62:0/1844754883 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c09c9d0 con 0x55f68ddd86c0 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.919+0000 7f677fe00640 1 -- 172.21.15.62:0/1844754883 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f678c0a0d80 con 0x55f68ddd86c0 2024-04-02T14:45:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.919+0000 7f677fe00640 1 -- 172.21.15.62:0/1844754883 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c0a51b0 con 0x55f68ddd86c0 2024-04-02T14:45:51.916 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/vt8500-sdmmc.txt 2024-04-02T14:45:51.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.919+0000 7f679d5b7740 1 -- 172.21.15.62:0/1844754883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd86c0 msgr2=0x55f68ddf0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:51.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.919+0000 7f679d5b7740 1 --2- 172.21.15.62:0/1844754883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd86c0 0x55f68ddf0340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f678c09c6d0 tx=0x7f678c0bd040 comp rx=0 tx=0).stop 2024-04-02T14:45:51.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.919+0000 7f679d5b7740 1 -- 172.21.15.62:0/1844754883 shutdown_connections 2024-04-02T14:45:51.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.919+0000 7f679d5b7740 1 --2- 172.21.15.62:0/1844754883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd86c0 0x55f68ddf0340 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.919+0000 7f679d5b7740 1 --2- 172.21.15.62:0/1844754883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddd0030 0x55f68dde4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.919+0000 7f679d5b7740 1 --2- 172.21.15.62:0/1844754883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddcf350 0x55f68ddd7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.920+0000 7f679d5b7740 1 -- 172.21.15.62:0/1844754883 >> 172.21.15.62:0/1844754883 conn(0x55f68dd63270 msgr2=0x55f68ddcba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:51.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.920+0000 7f679d5b7740 1 -- 172.21.15.62:0/1844754883 shutdown_connections 2024-04-02T14:45:51.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.920+0000 7f679d5b7740 1 -- 172.21.15.62:0/1844754883 wait complete. 2024-04-02T14:45:51.917 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/ 2024-04-02T14:45:51.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.920+0000 7f679d5b7740 1 -- start start 2024-04-02T14:45:51.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.920+0000 7f679d5b7740 1 Processor -- start 2024-04-02T14:45:51.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddcf350 0x55f68dde0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd0030 0x55f68dddd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f679d5b7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd86c0 0x55f68dddde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f679d5b7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f67941b0d70 con 0x55f68ddd86c0 2024-04-02T14:45:51.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f679d5b7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67941b06a0 con 0x55f68ddd0030 2024-04-02T14:45:51.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f679d5b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67941ad900 con 0x55f68ddcf350 2024-04-02T14:45:51.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f679d5b7740 10 client.0 _ll_register_callbacks cb 0x7f6780009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:51.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f6793400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddcf350 0x55f68dde0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f6793e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd86c0 0x55f68dddde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f6792a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd0030 0x55f68dddd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f6793400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddcf350 0x55f68dde0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:42382/0 (socket says 172.21.15.62:42382) 2024-04-02T14:45:51.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f6793400640 1 -- 172.21.15.62:0/2494519827 learned_addr learned my addr 172.21.15.62:0/2494519827 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:51.920 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/arm-versatile.txt 2024-04-02T14:45:51.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f6793400640 1 -- 172.21.15.62:0/2494519827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd0030 msgr2=0x55f68dddd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:51.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f6793400640 1 --2- 172.21.15.62:0/2494519827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd0030 0x55f68dddd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f6793400640 1 -- 172.21.15.62:0/2494519827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd86c0 msgr2=0x55f68dddde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:51.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f6793400640 1 --2- 172.21.15.62:0/2494519827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd86c0 0x55f68dddde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:51.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.921+0000 7f6793400640 1 -- 172.21.15.62:0/2494519827 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f68ddde3c0 con 0x55f68ddcf350 2024-04-02T14:45:51.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.922+0000 7f6793400640 1 --2- 172.21.15.62:0/2494519827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddcf350 0x55f68dde0b30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f678402c170 tx=0x7f6784005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=969dd96a4fe4be53 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:51.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.922+0000 7f677d600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:45:51.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.922+0000 7f677d600640 1 -- 172.21.15.62:0/2494519827 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784005d50 con 0x55f68ddcf350 2024-04-02T14:45:51.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.922+0000 7f677d600640 1 -- 172.21.15.62:0/2494519827 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f678402cbd0 con 0x55f68ddcf350 2024-04-02T14:45:51.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.922+0000 7f679d5b7740 1 -- 172.21.15.62:0/2494519827 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f678c0a0d80 con 0x55f68ddcf350 2024-04-02T14:45:51.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.922+0000 7f677d600640 1 -- 172.21.15.62:0/2494519827 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f678402cdd0 con 0x55f68ddcf350 2024-04-02T14:45:51.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.922+0000 7f679d5b7740 10 client.26053 fetch_fsmap learned FSMap version 29 2024-04-02T14:45:51.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.922+0000 7f679d5b7740 1 -- 172.21.15.62:0/2494519827 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f68de4f980 con 0x55f68ddcf350 2024-04-02T14:45:51.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.922+0000 7f677d600640 1 -- 172.21.15.62:0/2494519827 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f678c0a0d80 con 0x55f68ddcf350 2024-04-02T14:45:51.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.923+0000 7f677d600640 1 -- 172.21.15.62:0/2494519827 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f67840344e0 con 0x55f68ddcf350 2024-04-02T14:45:51.923 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-dataflash.txt 2024-04-02T14:45:51.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.923+0000 7f679d5b7740 10 client.26053 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T14:45:51.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.923+0000 7f679d5b7740 10 client.26053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:51.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.923+0000 7f679d5b7740 20 client.26053 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:51.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.923+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2494519827 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f68de50080 0x55f68de524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:51.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.923+0000 7f679d5b7740 4 client.26053 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:45:51.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.923+0000 7f679d5b7740 1 -- 172.21.15.62:0/2494519827 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f68dd3aba0 con 0x55f68de50080 2024-04-02T14:45:51.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.923+0000 7f6792a00640 1 --2- 172.21.15.62:0/2494519827 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f68de50080 0x55f68de524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:51.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.924+0000 7f677d600640 10 client.26053 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:51.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:51.924+0000 7f6792a00640 1 --2- 172.21.15.62:0/2494519827 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f68de50080 0x55f68de524a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45a36a42c0caee server_cookie=173614b8e23d01f0 in_seq=0 out_seq=0 2024-04-02T14:45:51.926 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-nand.txt 2024-04-02T14:45:51.926 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/davinci-nand.txt 2024-04-02T14:45:51.926 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/denali-nand.txt 2024-04-02T14:45:51.926 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/diskonchip.txt 2024-04-02T14:45:51.927 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/elm.txt 2024-04-02T14:45:51.928 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/flctl-nand.txt 2024-04-02T14:45:51.930 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt 2024-04-02T14:45:51.932 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-upm-nand.txt 2024-04-02T14:45:51.934 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsmc-nand.txt 2024-04-02T14:45:51.937 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpio-control-nand.txt 2024-04-02T14:45:51.940 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nand.txt 2024-04-02T14:45:51.943 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nor.txt 2024-04-02T14:45:51.946 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-onenand.txt 2024-04-02T14:45:51.949 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmi-nand.txt 2024-04-02T14:45:51.951 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/hisi504-nand.txt 2024-04-02T14:45:51.953 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-mlc.txt 2024-04-02T14:45:51.956 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-slc.txt 2024-04-02T14:45:51.958 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/m25p80.txt 2024-04-02T14:45:51.961 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mtd-physmap.txt 2024-04-02T14:45:51.964 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mxc-nand.txt 2024-04-02T14:45:51.966 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/nand.txt 2024-04-02T14:45:51.968 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/orion-nand.txt 2024-04-02T14:45:51.970 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/partition.txt 2024-04-02T14:45:51.972 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt 2024-04-02T14:45:51.973 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/spear_smi.txt 2024-04-02T14:45:51.975 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/st-fsm.txt 2024-04-02T14:45:51.977 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/sunxi-nand.txt 2024-04-02T14:45:51.979 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ 2024-04-02T14:45:51.982 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-emac.txt 2024-04-02T14:45:51.985 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-mdio.txt 2024-04-02T14:45:51.987 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.txt 2024-04-02T14:45:51.990 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/altera_tse.txt 2024-04-02T14:45:51.992 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe-phy.txt 2024-04-02T14:45:51.995 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe.txt 2024-04-02T14:45:51.998 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/apm-xgene-enet.txt 2024-04-02T14:45:52.000 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/arc_emac.txt 2024-04-02T14:45:52.003 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcm87xx.txt 2024-04-02T14:45:52.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:51 smithi204 ceph-mon[35948]: pgmap v58: 97 pgs: 97 active+clean; 30 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-04-02T14:45:52.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:51 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:45:52.006 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcmgenet.txt 2024-04-02T14:45:52.008 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-mdio-unimac.txt 2024-04-02T14:45:52.011 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-sf2.txt 2024-04-02T14:45:52.014 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-systemport.txt 2024-04-02T14:45:52.017 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/calxeda-xgmac.txt 2024-04-02T14:45:52.018 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/ 2024-04-02T14:45:52.021 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/atmel-can.txt 2024-04-02T14:45:52.024 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/c_can.txt 2024-04-02T14:45:52.026 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/cc770.txt 2024-04-02T14:45:52.028 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/fsl-flexcan.txt 2024-04-02T14:45:52.031 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/grcan.txt 2024-04-02T14:45:52.034 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/m_can.txt 2024-04-02T14:45:52.036 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt 2024-04-02T14:45:52.039 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/mpc5xxx-mscan.txt 2024-04-02T14:45:52.041 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/rcar_can.txt 2024-04-02T14:45:52.044 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/sja1000.txt 2024-04-02T14:45:52.047 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/xilinx_can.txt 2024-04-02T14:45:52.049 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mdio.txt 2024-04-02T14:45:52.052 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mix.txt 2024-04-02T14:45:52.054 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-pip.txt 2024-04-02T14:45:52.058 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cdns-emac.txt 2024-04-02T14:45:52.060 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw-phy-sel.txt 2024-04-02T14:45:52.065 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw.txt 2024-04-02T14:45:52.068 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davicom-dm9000.txt 2024-04-02T14:45:52.070 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci-mdio.txt 2024-04-02T14:45:52.072 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci_emac.txt 2024-04-02T14:45:52.074 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/ 2024-04-02T14:45:52.076 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/dsa.txt 2024-04-02T14:45:52.079 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/emac_rockchip.txt 2024-04-02T14:45:52.081 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ethernet.txt 2024-04-02T14:45:52.084 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fixed-link.txt 2024-04-02T14:45:52.086 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-fec.txt 2024-04-02T14:45:52.088 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-tsec-phy.txt 2024-04-02T14:45:52.091 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/gpmc-eth.txt 2024-04-02T14:45:52.093 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hip04-net.txt 2024-04-02T14:45:52.096 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hix5hd2-gmac.txt 2024-04-02T14:45:52.098 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/ 2024-04-02T14:45:52.101 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/at86rf230.txt 2024-04-02T14:45:52.103 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/cc2520.txt 2024-04-02T14:45:52.105 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/keystone-netcp.txt 2024-04-02T14:45:52.108 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/lpc-eth.txt 2024-04-02T14:45:52.110 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/macb.txt 2024-04-02T14:45:52.113 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt 2024-04-02T14:45:52.117 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-mdio.txt 2024-04-02T14:45:52.120 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-net.txt 2024-04-02T14:45:52.121 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pp2.txt 2024-04-02T14:45:52.123 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pxa168.txt 2024-04-02T14:45:52.125 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-gpio.txt 2024-04-02T14:45:52.127 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-gpio.txt 2024-04-02T14:45:52.129 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-mmioreg.txt 2024-04-02T14:45:52.132 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux.txt 2024-04-02T14:45:52.134 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/meson-dwmac.txt 2024-04-02T14:45:52.136 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ks8851.txt 2024-04-02T14:45:52.138 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ksz90x1.txt 2024-04-02T14:45:52.140 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel.txt 2024-04-02T14:45:52.146 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/moxa,moxart-mac.txt 2024-04-02T14:45:52.147 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/ 2024-04-02T14:45:52.149 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/pn544.txt 2024-04-02T14:45:52.151 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfca.txt 2024-04-02T14:45:52.152 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfcb.txt 2024-04-02T14:45:52.154 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/trf7970a.txt 2024-04-02T14:45:52.157 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/opencores-ethoc.txt 2024-04-02T14:45:52.159 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/phy.txt 2024-04-02T14:45:52.161 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/qca-qca7000-spi.txt 2024-04-02T14:45:52.164 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/rockchip-dwmac.txt 2024-04-02T14:45:52.166 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/samsung-sxgbe.txt 2024-04-02T14:45:52.169 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sh_eth.txt 2024-04-02T14:45:52.171 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc-lan91c111.txt 2024-04-02T14:45:52.173 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc911x.txt 2024-04-02T14:45:52.175 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/socfpga-dwmac.txt 2024-04-02T14:45:52.177 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sti-dwmac.txt 2024-04-02T14:45:52.179 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/stmmac.txt 2024-04-02T14:45:52.181 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-rhine.txt 2024-04-02T14:45:52.183 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-velocity.txt 2024-04-02T14:45:52.184 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ 2024-04-02T14:45:52.186 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/brcm,bcm43xx-fmac.txt 2024-04-02T14:45:52.188 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt 2024-04-02T14:45:52.190 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ti,wl1251.txt 2024-04-02T14:45:52.192 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/ 2024-04-02T14:45:52.194 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/nios2.txt 2024-04-02T14:45:52.197 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/timer.txt 2024-04-02T14:45:52.199 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/ 2024-04-02T14:45:52.200 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/nvidia,nvec.txt 2024-04-02T14:45:52.203 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/open-pic.txt 2024-04-02T14:45:52.209 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/ 2024-04-02T14:45:52.209 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101aw03.txt 2024-04-02T14:45:52.209 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101xtn01.txt 2024-04-02T14:45:52.212 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b116xw03.txt 2024-04-02T14:45:52.213 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133htn01.txt 2024-04-02T14:45:52.215 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133xtn01.txt 2024-04-02T14:45:52.218 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/avic,tm070ddh03.txt 2024-04-02T14:45:52.221 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wa01a.txt 2024-04-02T14:45:52.222 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wb03.txt 2024-04-02T14:45:52.224 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et057090dhu.txt 2024-04-02T14:45:52.226 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et070080dh6.txt 2024-04-02T14:45:52.228 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,etm0700g0dh6.txt 2024-04-02T14:45:52.230 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/foxlink,fl500wvr00-a0t.txt 2024-04-02T14:45:52.233 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/giantplus,gpg482739qs5.txt 2024-04-02T14:45:52.235 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hannstar,hsd070pww1.txt 2024-04-02T14:45:52.236 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hit,tx23d38vm0caa.txt 2024-04-02T14:45:52.239 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,g121i1-l01.txt 2024-04-02T14:45:52.241 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n116bge.txt 2024-04-02T14:45:52.242 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n156bge-l21.txt 2024-04-02T14:45:52.244 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,ld070wx3-sl01.txt 2024-04-02T14:45:52.246 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lh500wx1-sd03.txt 2024-04-02T14:45:52.249 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lp129qe.txt 2024-04-02T14:45:52.250 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/panasonic,vvx10f004b00.txt 2024-04-02T14:45:52.253 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ld9040.txt 2024-04-02T14:45:52.255 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ltn101nt05.txt 2024-04-02T14:45:52.257 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,s6e8aa0.txt 2024-04-02T14:45:52.259 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/sharp,lq101r1sx01.txt 2024-04-02T14:45:52.261 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/simple-panel.txt 2024-04-02T14:45:52.263 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ 2024-04-02T14:45:52.267 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/83xx-512x-pci.txt 2024-04-02T14:45:52.267 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/designware-pcie.txt 2024-04-02T14:45:52.269 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt 2024-04-02T14:45:52.270 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,pci.txt 2024-04-02T14:45:52.273 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/host-generic-pci.txt 2024-04-02T14:45:52.275 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/layerscape-pci.txt 2024-04-02T14:45:52.277 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/mvebu-pci.txt 2024-04-02T14:45:52.279 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt 2024-04-02T14:45:52.281 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-keystone.txt 2024-04-02T14:45:52.283 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-rcar-gen2.txt 2024-04-02T14:45:52.285 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci.txt 2024-04-02T14:45:52.287 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ralink,rt3883-pci.txt 2024-04-02T14:45:52.289 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/rcar-pci.txt 2024-04-02T14:45:52.292 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/samsung,exynos5440-pcie.txt 2024-04-02T14:45:52.294 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/spear13xx-pcie.txt 2024-04-02T14:45:52.295 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ti-pci.txt 2024-04-02T14:45:52.298 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/v3-v360epc-pci.txt 2024-04-02T14:45:52.300 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/versatile.txt 2024-04-02T14:45:52.302 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xgene-pci.txt 2024-04-02T14:45:52.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xilinx-pcie.txt 2024-04-02T14:45:52.306 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ 2024-04-02T14:45:52.308 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/apm-xgene-phy.txt 2024-04-02T14:45:52.310 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/bcm-phy.txt 2024-04-02T14:45:52.312 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-sata-phy.txt 2024-04-02T14:45:52.314 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-usb-phy.txt 2024-04-02T14:45:52.316 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/hix5hd2-phy.txt 2024-04-02T14:45:52.318 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-bindings.txt 2024-04-02T14:45:52.320 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy28lp.txt 2024-04-02T14:45:52.322 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy365x.txt 2024-04-02T14:45:52.325 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-mvebu.txt 2024-04-02T14:45:52.326 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih407-usb.txt 2024-04-02T14:45:52.328 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt 2024-04-02T14:45:52.331 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-apq8064-sata-phy.txt 2024-04-02T14:45:52.333 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-dwc3-usb-phy.txt 2024-04-02T14:45:52.335 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-ipq806x-sata-phy.txt 2024-04-02T14:45:52.337 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt 2024-04-02T14:45:52.339 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rockchip-usb-phy.txt 2024-04-02T14:45:52.341 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/samsung-phy.txt 2024-04-02T14:45:52.343 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/st-spear-miphy.txt 2024-04-02T14:45:52.345 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/sun4i-usb-phy.txt 2024-04-02T14:45:52.347 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ti-phy.txt 2024-04-02T14:45:52.349 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ 2024-04-02T14:45:52.351 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/abilis,tb10x-iomux.txt 2024-04-02T14:45:52.352 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt 2024-04-02T14:45:52.355 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/atmel,at91-pinctrl.txt 2024-04-02T14:45:52.357 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm11351-pinctrl.txt 2024-04-02T14:45:52.359 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm2835-gpio.txt 2024-04-02T14:45:52.361 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx-pinctrl.txt 2024-04-02T14:45:52.363 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx25-pinctrl.txt 2024-04-02T14:45:52.365 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx27-pinctrl.txt 2024-04-02T14:45:52.367 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx35-pinctrl.txt 2024-04-02T14:45:52.370 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx51-pinctrl.txt 2024-04-02T14:45:52.372 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx53-pinctrl.txt 2024-04-02T14:45:52.373 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6dl-pinctrl.txt 2024-04-02T14:45:52.376 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6q-pinctrl.txt 2024-04-02T14:45:52.378 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sl-pinctrl.txt 2024-04-02T14:45:52.380 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sx-pinctrl.txt 2024-04-02T14:45:52.381 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,mxs-pinctrl.txt 2024-04-02T14:45:52.383 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,vf610-pinctrl.txt 2024-04-02T14:45:52.385 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pdc-pinctrl.txt 2024-04-02T14:45:52.387 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pinctrl.txt 2024-04-02T14:45:52.389 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,falcon-pinumx.txt 2024-04-02T14:45:52.391 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,xway-pinumx.txt 2024-04-02T14:45:52.393 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-370-pinctrl.txt 2024-04-02T14:45:52.394 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-375-pinctrl.txt 2024-04-02T14:45:52.396 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-38x-pinctrl.txt 2024-04-02T14:45:52.398 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-xp-pinctrl.txt 2024-04-02T14:45:52.400 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,dove-pinctrl.txt 2024-04-02T14:45:52.402 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,kirkwood-pinctrl.txt 2024-04-02T14:45:52.404 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,mvebu-pinctrl.txt 2024-04-02T14:45:52.406 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,orion-pinctrl.txt 2024-04-02T14:45:52.408 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/meson,pinctrl.txt 2024-04-02T14:45:52.410 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra114-pinmux.txt 2024-04-02T14:45:52.412 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-pinmux.txt 2024-04-02T14:45:52.414 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt 2024-04-02T14:45:52.416 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra20-pinmux.txt 2024-04-02T14:45:52.418 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra30-pinmux.txt 2024-04-02T14:45:52.420 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt 2024-04-02T14:45:52.421 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 2024-04-02T14:45:52.423 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-single.txt 2024-04-02T14:45:52.425 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-sirf.txt 2024-04-02T14:45:52.427 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-st.txt 2024-04-02T14:45:52.429 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-vt8500.txt 2024-04-02T14:45:52.431 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl_spear.txt 2024-04-02T14:45:52.433 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8064-pinctrl.txt 2024-04-02T14:45:52.436 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8084-pinctrl.txt 2024-04-02T14:45:52.437 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,ipq8064-pinctrl.txt 2024-04-02T14:45:52.439 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8916-pinctrl.txt 2024-04-02T14:45:52.441 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8960-pinctrl.txt 2024-04-02T14:45:52.444 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8974-pinctrl.txt 2024-04-02T14:45:52.446 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt 2024-04-02T14:45:52.448 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt 2024-04-02T14:45:52.450 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt 2024-04-02T14:45:52.452 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt 2024-04-02T14:45:52.454 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt 2024-04-02T14:45:52.457 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,abx500.txt 2024-04-02T14:45:52.459 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt 2024-04-02T14:45:52.461 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ti,omap-pinctrl.txt 2024-04-02T14:45:52.463 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/xlnx,zynq-pinctrl.txt 2024-04-02T14:45:52.464 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ 2024-04-02T14:45:52.467 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/bq2415x.txt 2024-04-02T14:45:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.473+0000 7f677d600640 1 -- 172.21.15.62:0/2494519827 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f68dd3aba0 con 0x55f68de50080 2024-04-02T14:45:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.473+0000 7f677d600640 10 client.26053 handle_command_reply: tid=0 2024-04-02T14:45:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.473+0000 7f679d5b7740 1 client.26053 shutdown 2024-04-02T14:45:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.473+0000 7f679d5b7740 1 -- 172.21.15.62:0/2494519827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddcf350 msgr2=0x55f68dde0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.473+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2494519827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddcf350 0x55f68dde0b30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f678402c170 tx=0x7f6784005be0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.474+0000 7f679d5b7740 1 -- 172.21.15.62:0/2494519827 shutdown_connections 2024-04-02T14:45:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.474+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2494519827 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f68de50080 0x55f68de524a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.474+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2494519827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68ddd86c0 0x55f68dddde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.474+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2494519827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f68ddd0030 0x55f68dddd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.474+0000 7f679d5b7740 1 --2- 172.21.15.62:0/2494519827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f68ddcf350 0x55f68dde0b30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.474+0000 7f679d5b7740 1 -- 172.21.15.62:0/2494519827 >> 172.21.15.62:0/2494519827 conn(0x55f68dd63270 msgr2=0x55f68ddcafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.474+0000 7f677d600640 0 client.26053 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:52.470 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/isp1704.txt 2024-04-02T14:45:52.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.474+0000 7f679d5b7740 1 -- 172.21.15.62:0/2494519827 shutdown_connections 2024-04-02T14:45:52.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.475+0000 7f679d5b7740 1 -- 172.21.15.62:0/2494519827 wait complete. 2024-04-02T14:45:52.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.475+0000 7f679d5b7740 20 client.26053 trim_cache size 0 max 16384 2024-04-02T14:45:52.471 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:45:52.471 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-02T14:45:52.471 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "bc3dbfe7-3363-43cf-858c-341b5ce1a3e6", 2024-04-02T14:45:52.471 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-02T14:45:52.471 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:45:52.472 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ltc2941.txt 2024-04-02T14:45:52.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.476+0000 7f679a800640 1 -- 172.21.15.62:0/239400269 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f67780a3da0 msgr2=0x7f67780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.476+0000 7f679a800640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f67780a3da0 0x7f67780a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6780015c50 tx=0x7f6780009d00 comp rx=0 tx=0).stop 2024-04-02T14:45:52.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.476+0000 7f679a800640 1 -- 172.21.15.62:0/239400269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6794107aa0 msgr2=0x7f679419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.476+0000 7f679a800640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6794107aa0 0x7f679419cd30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f678c00da10 tx=0x7f678c00ded0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.477+0000 7f679a800640 1 -- 172.21.15.62:0/239400269 shutdown_connections 2024-04-02T14:45:52.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.477+0000 7f679a800640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f67780a3da0 0x7f67780a6260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.477+0000 7f679a800640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6794107aa0 0x7f679419cd30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.477+0000 7f679a800640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794102450 0x7f6794198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.477+0000 7f679a800640 1 --2- 172.21.15.62:0/239400269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6794101a90 0x7f67941984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.477+0000 7f679a800640 1 -- 172.21.15.62:0/239400269 >> 172.21.15.62:0/239400269 conn(0x7f6794078f00 msgr2=0x7f6794104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:52.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.477+0000 7f679a800640 1 -- 172.21.15.62:0/239400269 shutdown_connections 2024-04-02T14:45:52.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.477+0000 7f679a800640 1 -- 172.21.15.62:0/239400269 wait complete. 2024-04-02T14:45:52.474 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/opp.txt 2024-04-02T14:45:52.475 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power-controller.txt 2024-04-02T14:45:52.478 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power_domain.txt 2024-04-02T14:45:52.480 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/renesas,sysc-rmobile.txt 2024-04-02T14:45:52.482 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ 2024-04-02T14:45:52.484 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/keystone-reset.txt 2024-04-02T14:45:52.486 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ltc2952-poweroff.txt 2024-04-02T14:45:52.487 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/st-reset.txt 2024-04-02T14:45:52.489 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/syscon-reboot.txt 2024-04-02T14:45:52.491 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rockchip-io-domain.txt 2024-04-02T14:45:52.493 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rx51-battery.txt 2024-04-02T14:45:52.494 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/twl-charger.txt 2024-04-02T14:45:52.497 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ 2024-04-02T14:45:52.498 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/ 2024-04-02T14:45:52.500 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/btemp.txt 2024-04-02T14:45:52.502 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/chargalg.txt 2024-04-02T14:45:52.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:45:52.503 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:45:52.504 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/charger.txt 2024-04-02T14:45:52.512 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/fg.txt 2024-04-02T14:45:52.512 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/axxia-reset.txt 2024-04-02T14:45:52.512 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/charger-manager.txt 2024-04-02T14:45:52.512 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/gpio-charger.txt 2024-04-02T14:45:52.512 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/imx-snvs-poweroff.txt 2024-04-02T14:45:52.514 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/lp8727_charger.txt 2024-04-02T14:45:52.515 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max17042_battery.txt 2024-04-02T14:45:52.516 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max8925_batter.txt 2024-04-02T14:45:52.518 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/msm-poweroff.txt 2024-04-02T14:45:52.522 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/olpc_battery.txt 2024-04-02T14:45:52.523 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/power_supply.txt 2024-04-02T14:45:52.525 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/qnap-poweroff.txt 2024-04-02T14:45:52.527 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/restart-poweroff.txt 2024-04-02T14:45:52.529 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/sbs_sbs-battery.txt 2024-04-02T14:45:52.530 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ti,bq24735.txt 2024-04-02T14:45:52.532 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/tps65090.txt 2024-04-02T14:45:52.536 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/ 2024-04-02T14:45:52.538 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ 2024-04-02T14:45:52.539 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt 2024-04-02T14:45:52.541 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/cpm.txt 2024-04-02T14:45:52.543 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/emac.txt 2024-04-02T14:45:52.544 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt 2024-04-02T14:45:52.546 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ndfc.txt 2024-04-02T14:45:52.547 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ppc440spe-adma.txt 2024-04-02T14:45:52.551 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/reboot.txt 2024-04-02T14:45:52.553 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ 2024-04-02T14:45:52.555 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/board.txt 2024-04-02T14:45:52.557 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cache_sram.txt 2024-04-02T14:45:52.558 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt 2024-04-02T14:45:52.561 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/ 2024-04-02T14:45:52.563 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm.txt 2024-04-02T14:45:52.566 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/ 2024-04-02T14:45:52.567 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/brg.txt 2024-04-02T14:45:52.569 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/i2c.txt 2024-04-02T14:45:52.571 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/pic.txt 2024-04-02T14:45:52.573 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/usb.txt 2024-04-02T14:45:52.577 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/gpio.txt 2024-04-02T14:45:52.579 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/network.txt 2024-04-02T14:45:52.580 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe.txt 2024-04-02T14:45:52.582 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ 2024-04-02T14:45:52.583 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/firmware.txt 2024-04-02T14:45:52.584 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/par_io.txt 2024-04-02T14:45:52.586 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/pincfg.txt 2024-04-02T14:45:52.588 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ucc.txt 2024-04-02T14:45:52.589 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/usb.txt 2024-04-02T14:45:52.591 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/serial.txt 2024-04-02T14:45:52.593 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt 2024-04-02T14:45:52.595 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dcsr.txt 2024-04-02T14:45:52.597 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/diu.txt 2024-04-02T14:45:52.599 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dma.txt 2024-04-02T14:45:52.601 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ecm.txt 2024-04-02T14:45:52.602 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/fman.txt 2024-04-02T14:45:52.604 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/gtm.txt 2024-04-02T14:45:52.605 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/guts.txt 2024-04-02T14:45:52.607 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/interlaken-lac.txt 2024-04-02T14:45:52.608 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/l2cache.txt 2024-04-02T14:45:52.610 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/lbc.txt 2024-04-02T14:45:52.612 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcm.txt 2024-04-02T14:45:52.613 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcu-mpc8349emitx.txt 2024-04-02T14:45:52.615 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mem-ctrlr.txt 2024-04-02T14:45:52.616 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5121-psc.txt 2024-04-02T14:45:52.618 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5200.txt 2024-04-02T14:45:52.620 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt 2024-04-02T14:45:52.622 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-timer.txt 2024-04-02T14:45:52.624 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic.txt 2024-04-02T14:45:52.626 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/msi-pic.txt 2024-04-02T14:45:52.628 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt 2024-04-02T14:45:52.630 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pmc.txt 2024-04-02T14:45:52.632 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/raideng.txt 2024-04-02T14:45:52.633 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio-rmu.txt 2024-04-02T14:45:52.636 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio.txt 2024-04-02T14:45:52.638 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/ 2024-04-02T14:45:52.640 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/gamecube.txt 2024-04-02T14:45:52.643 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/wii.txt 2024-04-02T14:45:52.645 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/ 2024-04-02T14:45:52.647 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/pps-gpio.txt 2024-04-02T14:45:52.650 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ 2024-04-02T14:45:52.651 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-hlcdc-pwm.txt 2024-04-02T14:45:52.653 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-pwm.txt 2024-04-02T14:45:52.655 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-tcb-pwm.txt 2024-04-02T14:45:52.656 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/bcm-kona-pwm.txt 2024-04-02T14:45:52.658 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/cirrus,clps711x-pwm.txt 2024-04-02T14:45:52.660 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/img-pwm.txt 2024-04-02T14:45:52.661 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/imx-pwm.txt 2024-04-02T14:45:52.663 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/lpc32xx-pwm.txt 2024-04-02T14:45:52.665 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/mxs-pwm.txt 2024-04-02T14:45:52.667 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nvidia,tegra20-pwm.txt 2024-04-02T14:45:52.668 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nxp,pca9685-pwm.txt 2024-04-02T14:45:52.670 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-bcm2835.txt 2024-04-02T14:45:52.671 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-fsl-ftm.txt 2024-04-02T14:45:52.673 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-lp3943.txt 2024-04-02T14:45:52.675 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-rockchip.txt 2024-04-02T14:45:52.676 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-samsung.txt 2024-04-02T14:45:52.678 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-st.txt 2024-04-02T14:45:52.680 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-sun4i.txt 2024-04-02T14:45:52.682 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiecap.txt 2024-04-02T14:45:52.683 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiehrpwm.txt 2024-04-02T14:45:52.685 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tipwmss.txt 2024-04-02T14:45:52.687 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm.txt 2024-04-02T14:45:52.689 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pxa-pwm.txt 2024-04-02T14:45:52.690 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/renesas,tpu-pwm.txt 2024-04-02T14:45:52.692 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/spear-pwm.txt 2024-04-02T14:45:52.694 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwm.txt 2024-04-02T14:45:52.696 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwmled.txt 2024-04-02T14:45:52.698 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/vt8500-pwm.txt 2024-04-02T14:45:52.700 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/ 2024-04-02T14:45:52.703 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/regmap.txt 2024-04-02T14:45:52.705 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ 2024-04-02T14:45:52.707 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm800.txt 2024-04-02T14:45:52.709 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm860x.txt 2024-04-02T14:45:52.711 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/act8865-regulator.txt 2024-04-02T14:45:52.712 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/anatop-regulator.txt 2024-04-02T14:45:52.714 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/as3722-regulator.txt 2024-04-02T14:45:52.715 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9210.txt 2024-04-02T14:45:52.718 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9211.txt 2024-04-02T14:45:52.719 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fan53555.txt 2024-04-02T14:45:52.721 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fixed-regulator.txt 2024-04-02T14:45:52.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.727+0000 7f6716200640 1 -- 172.21.15.62:0/2844347236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6710109050 msgr2=0x7f6710109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.727+0000 7f6716200640 1 --2- 172.21.15.62:0/2844347236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6710109050 0x7f6710109430 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6700009900 tx=0x7f670002bc30 comp rx=0 tx=0).stop 2024-04-02T14:45:52.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.727+0000 7f6716200640 1 -- 172.21.15.62:0/2844347236 shutdown_connections 2024-04-02T14:45:52.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.728+0000 7f6716200640 1 --2- 172.21.15.62:0/2844347236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710116c00 0x7f6710118ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.728+0000 7f6716200640 1 --2- 172.21.15.62:0/2844347236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6710109a00 0x7f67101166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.728+0000 7f6716200640 1 --2- 172.21.15.62:0/2844347236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6710109050 0x7f6710109430 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.728+0000 7f6716200640 1 -- 172.21.15.62:0/2844347236 >> 172.21.15.62:0/2844347236 conn(0x7f6710072f50 msgr2=0x7f6710073360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:52.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.728+0000 7f6716200640 1 -- 172.21.15.62:0/2844347236 shutdown_connections 2024-04-02T14:45:52.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.728+0000 7f6716200640 1 -- 172.21.15.62:0/2844347236 wait complete. 2024-04-02T14:45:52.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.729+0000 7f6716200640 1 Processor -- start 2024-04-02T14:45:52.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.729+0000 7f6716200640 1 -- start start 2024-04-02T14:45:52.724 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/gpio-regulator.txt 2024-04-02T14:45:52.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f6716200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6710109050 0x7f67101a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f6716200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6710109a00 0x7f67101a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f6716200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710116c00 0x7f67101a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f6716200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f671011c990 con 0x7f6710109050 2024-04-02T14:45:52.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f6716200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f671011c810 con 0x7f6710109a00 2024-04-02T14:45:52.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f6716200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f671011cb10 con 0x7f6710116c00 2024-04-02T14:45:52.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f670ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6710109a00 0x7f67101a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f670f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6710109050 0x7f67101a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f670ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6710109a00 0x7f67101a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:56276/0 (socket says 172.21.15.62:56276) 2024-04-02T14:45:52.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f670ea00640 1 -- 172.21.15.62:0/2488289905 learned_addr learned my addr 172.21.15.62:0/2488289905 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:52.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f670fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710116c00 0x7f67101a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f670ea00640 1 -- 172.21.15.62:0/2488289905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710116c00 msgr2=0x7f67101a79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f670ea00640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710116c00 0x7f67101a79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f670ea00640 1 -- 172.21.15.62:0/2488289905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6710109050 msgr2=0x7f67101a2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f670ea00640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6710109050 0x7f67101a2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f670ea00640 1 -- 172.21.15.62:0/2488289905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67101a80d0 con 0x7f6710109a00 2024-04-02T14:45:52.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.730+0000 7f670fe00640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710116c00 0x7f67101a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:52.728 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/isl9305.txt 2024-04-02T14:45:52.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.731+0000 7f670ea00640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6710109a00 0x7f67101a3410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00e9f0 tx=0x7f66fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:52.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.731+0000 7f6707e00640 1 -- 172.21.15.62:0/2488289905 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc00cd80 con 0x7f6710109a00 2024-04-02T14:45:52.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.731+0000 7f6716200640 1 -- 172.21.15.62:0/2488289905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67101a83c0 con 0x7f6710109a00 2024-04-02T14:45:52.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.731+0000 7f6707e00640 1 -- 172.21.15.62:0/2488289905 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66fc010930 con 0x7f6710109a00 2024-04-02T14:45:52.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.731+0000 7f6707e00640 1 -- 172.21.15.62:0/2488289905 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc018b00 con 0x7f6710109a00 2024-04-02T14:45:52.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.731+0000 7f6716200640 1 -- 172.21.15.62:0/2488289905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67101afc50 con 0x7f6710109a00 2024-04-02T14:45:52.730 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/lp872x.txt 2024-04-02T14:45:52.730 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ltc3589.txt 2024-04-02T14:45:52.730 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max1586-regulator.txt 2024-04-02T14:45:52.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.735+0000 7f6707e00640 1 -- 172.21.15.62:0/2488289905 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f66fc018ca0 con 0x7f6710109a00 2024-04-02T14:45:52.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.735+0000 7f6718e40740 1 Processor -- start 2024-04-02T14:45:52.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.735+0000 7f6718e40740 1 -- start start 2024-04-02T14:45:52.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f6718e40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4457b0 0x5568fd445b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f6718e40740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4460d0 0x5568fd44e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f6718e40740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd44ecb0 0x5568fd4510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f6718e40740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5568fd3a9770 con 0x5568fd4460d0 2024-04-02T14:45:52.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f6718e40740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5568fd351700 con 0x5568fd44ecb0 2024-04-02T14:45:52.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f6718e40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5568fd3a5720 con 0x5568fd4457b0 2024-04-02T14:45:52.733 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max77802.txt 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f670fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4457b0 0x5568fd445b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f670fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4457b0 0x5568fd445b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55592/0 (socket says 172.21.15.62:55592) 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f670fe00640 1 -- 172.21.15.62:0/1921345299 learned_addr learned my addr 172.21.15.62:0/1921345299 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f670ea00640 1 --2- 172.21.15.62:0/1921345299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd44ecb0 0x5568fd4510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f670f400640 1 --2- 172.21.15.62:0/1921345299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4460d0 0x5568fd44e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f670fe00640 1 -- 172.21.15.62:0/1921345299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd44ecb0 msgr2=0x5568fd4510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.736+0000 7f670fe00640 1 --2- 172.21.15.62:0/1921345299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd44ecb0 0x5568fd4510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.737+0000 7f670fe00640 1 -- 172.21.15.62:0/1921345299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4460d0 msgr2=0x5568fd44e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.737+0000 7f670fe00640 1 --2- 172.21.15.62:0/1921345299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4460d0 0x5568fd44e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.737+0000 7f670fe00640 1 -- 172.21.15.62:0/1921345299 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568fd1877d0 con 0x5568fd4457b0 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.737+0000 7f6707e00640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66f40a3d20 0x7f66f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.737+0000 7f670fe00640 1 --2- 172.21.15.62:0/1921345299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4457b0 0x5568fd445b90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6708011790 tx=0x7f6708011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26796d5b420fd811 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.737+0000 7f6707e00640 1 -- 172.21.15.62:0/2488289905 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f66fc0d0b90 con 0x7f6710109a00 2024-04-02T14:45:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.737+0000 7f670f400640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66f40a3d20 0x7f66f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.737+0000 7f66fbe00640 1 -- 172.21.15.62:0/1921345299 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708003c30 con 0x5568fd4457b0 2024-04-02T14:45:52.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.737+0000 7f66fbe00640 1 -- 172.21.15.62:0/1921345299 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f670800fe20 con 0x5568fd4457b0 2024-04-02T14:45:52.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.737+0000 7f66fbe00640 1 -- 172.21.15.62:0/1921345299 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67080193e0 con 0x5568fd4457b0 2024-04-02T14:45:52.736 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8660.txt 2024-04-02T14:45:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.738+0000 7f6718e40740 1 -- 172.21.15.62:0/1921345299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4457b0 msgr2=0x5568fd445b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.738+0000 7f670f400640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66f40a3d20 0x7f66f40a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6700006010 tx=0x7f6700005e40 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.738+0000 7f6718e40740 1 --2- 172.21.15.62:0/1921345299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4457b0 0x5568fd445b90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6708011790 tx=0x7f6708011c50 comp rx=0 tx=0).stop 2024-04-02T14:45:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.738+0000 7f6718e40740 1 -- 172.21.15.62:0/1921345299 shutdown_connections 2024-04-02T14:45:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.738+0000 7f6718e40740 1 --2- 172.21.15.62:0/1921345299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd44ecb0 0x5568fd4510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.738+0000 7f6718e40740 1 --2- 172.21.15.62:0/1921345299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4460d0 0x5568fd44e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.738+0000 7f6718e40740 1 --2- 172.21.15.62:0/1921345299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4457b0 0x5568fd445b90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.738+0000 7f6718e40740 1 -- 172.21.15.62:0/1921345299 >> 172.21.15.62:0/1921345299 conn(0x5568fd43fee0 msgr2=0x5568fd4402c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.738+0000 7f6718e40740 1 -- 172.21.15.62:0/1921345299 shutdown_connections 2024-04-02T14:45:52.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.738+0000 7f6718e40740 1 -- 172.21.15.62:0/1921345299 wait complete. 2024-04-02T14:45:52.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.739+0000 7f6718e40740 1 -- start start 2024-04-02T14:45:52.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.739+0000 7f6718e40740 1 Processor -- start 2024-04-02T14:45:52.738 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8907.txt 2024-04-02T14:45:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f6718e40740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4457b0 0x5568fd44bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f6718e40740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4460d0 0x5568fd44c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f6718e40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd44ecb0 0x5568fd44c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f6718e40740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5568fd451aa0 con 0x5568fd4460d0 2024-04-02T14:45:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f6718e40740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5568fd364ac0 con 0x5568fd4457b0 2024-04-02T14:45:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f6718e40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5568fd451c20 con 0x5568fd44ecb0 2024-04-02T14:45:52.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f6718e40740 10 client.0 _ll_register_callbacks cb 0x5568fd367180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:52.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f670f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4457b0 0x5568fd44bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f670ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4460d0 0x5568fd44c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f670fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd44ecb0 0x5568fd44c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f670f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4457b0 0x5568fd44bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:56298/0 (socket says 172.21.15.62:56298) 2024-04-02T14:45:52.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f670f400640 1 -- 172.21.15.62:0/2930730252 learned_addr learned my addr 172.21.15.62:0/2930730252 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:52.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f670f400640 1 -- 172.21.15.62:0/2930730252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd44ecb0 msgr2=0x5568fd44c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:52.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f670f400640 1 --2- 172.21.15.62:0/2930730252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd44ecb0 0x5568fd44c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f670f400640 1 -- 172.21.15.62:0/2930730252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4460d0 msgr2=0x5568fd44c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:52.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f670f400640 1 --2- 172.21.15.62:0/2930730252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4460d0 0x5568fd44c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.740+0000 7f670f400640 1 -- 172.21.15.62:0/2930730252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568fd0bd3f0 con 0x5568fd4457b0 2024-04-02T14:45:52.741 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8925-regulator.txt 2024-04-02T14:45:52.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.741+0000 7f670f400640 1 --2- 172.21.15.62:0/2930730252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4457b0 0x5568fd44bd50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6700035000 tx=0x7f6700009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=618394928513201 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:52.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.741+0000 7f66f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:52.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.741+0000 7f66f8c00640 1 -- 172.21.15.62:0/2930730252 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670004f020 con 0x5568fd4457b0 2024-04-02T14:45:52.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.741+0000 7f66f8c00640 1 -- 172.21.15.62:0/2930730252 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f6700005040 con 0x5568fd4457b0 2024-04-02T14:45:52.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.741+0000 7f6718e40740 1 -- 172.21.15.62:0/2930730252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f670800fe20 con 0x5568fd4457b0 2024-04-02T14:45:52.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.741+0000 7f66f8c00640 1 -- 172.21.15.62:0/2930730252 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f670005a050 con 0x5568fd4457b0 2024-04-02T14:45:52.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.742+0000 7f6718e40740 10 client.26096 fetch_fsmap learned FSMap version 29 2024-04-02T14:45:52.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.742+0000 7f6718e40740 1 -- 172.21.15.62:0/2930730252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5568fd07d0f0 con 0x5568fd4457b0 2024-04-02T14:45:52.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.742+0000 7f66f8c00640 1 -- 172.21.15.62:0/2930730252 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f670800fe20 con 0x5568fd4457b0 2024-04-02T14:45:52.742 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8952.txt 2024-04-02T14:45:52.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.742+0000 7f66f8c00640 1 -- 172.21.15.62:0/2930730252 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f67000035f0 con 0x5568fd4457b0 2024-04-02T14:45:52.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.742+0000 7f6718e40740 10 client.26096 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T14:45:52.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.742+0000 7f6718e40740 10 client.26096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:52.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.742+0000 7f6718e40740 20 client.26096 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:52.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.742+0000 7f6718e40740 1 --2- 172.21.15.62:0/2930730252 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5568fd45edd0 0x5568fd461250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.742+0000 7f6718e40740 4 client.26096 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:52.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.742+0000 7f6718e40740 1 -- 172.21.15.62:0/2930730252 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568fd34a630 con 0x5568fd45edd0 2024-04-02T14:45:52.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.743+0000 7f670ea00640 1 --2- 172.21.15.62:0/2930730252 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5568fd45edd0 0x5568fd461250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.743+0000 7f670ea00640 1 --2- 172.21.15.62:0/2930730252 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5568fd45edd0 0x5568fd461250 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10ebbacd84bc819 server_cookie=34e89170cda7973c in_seq=0 out_seq=0 2024-04-02T14:45:52.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.743+0000 7f66f8c00640 10 client.26096 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:52.744 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8973-regulator.txt 2024-04-02T14:45:52.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f66f8c00640 1 -- 172.21.15.62:0/2930730252 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568fd34a630 con 0x5568fd45edd0 2024-04-02T14:45:52.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f66f8c00640 10 client.26096 handle_command_reply: tid=0 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 1 client.26096 shutdown 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 1 -- 172.21.15.62:0/2930730252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4457b0 msgr2=0x5568fd44bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 1 --2- 172.21.15.62:0/2930730252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4457b0 0x5568fd44bd50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6700035000 tx=0x7f6700009260 comp rx=0 tx=0).stop 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 1 -- 172.21.15.62:0/2930730252 shutdown_connections 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 1 --2- 172.21.15.62:0/2930730252 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5568fd45edd0 0x5568fd461250 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 1 --2- 172.21.15.62:0/2930730252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd44ecb0 0x5568fd44c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 1 --2- 172.21.15.62:0/2930730252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4460d0 0x5568fd44c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 1 --2- 172.21.15.62:0/2930730252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4457b0 0x5568fd44bd50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 1 -- 172.21.15.62:0/2930730252 >> 172.21.15.62:0/2930730252 conn(0x5568fd43fee0 msgr2=0x5568fd441c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f66f8c00640 0 client.26096 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 1 -- 172.21.15.62:0/2930730252 shutdown_connections 2024-04-02T14:45:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 1 -- 172.21.15.62:0/2930730252 wait complete. 2024-04-02T14:45:52.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.745+0000 7f6718e40740 20 client.26096 trim_cache size 0 max 16384 2024-04-02T14:45:52.747 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8997-regulator.txt 2024-04-02T14:45:52.747 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/mt6397-regulator.txt 2024-04-02T14:45:52.747 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/palmas-pmic.txt 2024-04-02T14:45:52.747 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pbias-regulator.txt 2024-04-02T14:45:52.748 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pfuze100.txt 2024-04-02T14:45:52.748 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pwm-regulator.txt 2024-04-02T14:45:52.749 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/regulator.txt 2024-04-02T14:45:52.750 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/s5m8767-regulator.txt 2024-04-02T14:45:52.751 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/sky81452-regulator.txt 2024-04-02T14:45:52.752 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ti-abb-regulator.txt 2024-04-02T14:45:52.753 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps51632-regulator.txt 2024-04-02T14:45:52.755 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps62360-regulator.txt 2024-04-02T14:45:52.756 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65090.txt 2024-04-02T14:45:52.757 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65217.txt 2024-04-02T14:45:52.759 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65218.txt 2024-04-02T14:45:52.760 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps6586x.txt 2024-04-02T14:45:52.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.766+0000 7f6718e40740 1 Processor -- start 2024-04-02T14:45:52.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.766+0000 7f6718e40740 1 -- start start 2024-04-02T14:45:52.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.766+0000 7f6718e40740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 0x5568fd44e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.766+0000 7f6718e40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4460d0 0x5568fd44ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.766+0000 7f6718e40740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd44ecb0 0x5568fd45a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.766+0000 7f6718e40740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f67101b8180 con 0x5568fd4457b0 2024-04-02T14:45:52.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.766+0000 7f6718e40740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67101b9cf0 con 0x5568fd44ecb0 2024-04-02T14:45:52.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.766+0000 7f6718e40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67101b8850 con 0x5568fd4460d0 2024-04-02T14:45:52.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 0x5568fd44e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4460d0 0x5568fd44ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4460d0 0x5568fd44ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:55608/0 (socket says 172.21.15.62:55608) 2024-04-02T14:45:52.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670ea00640 1 -- 172.21.15.62:0/2233283502 learned_addr learned my addr 172.21.15.62:0/2233283502 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:52.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670fe00640 1 --2- 172.21.15.62:0/2233283502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd44ecb0 0x5568fd45a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.763 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/twl-regulator.txt 2024-04-02T14:45:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670f400640 1 -- 172.21.15.62:0/2233283502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4460d0 msgr2=0x5568fd44ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670f400640 1 --2- 172.21.15.62:0/2233283502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4460d0 0x5568fd44ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670f400640 1 -- 172.21.15.62:0/2233283502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd44ecb0 msgr2=0x5568fd45a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670f400640 1 --2- 172.21.15.62:0/2233283502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd44ecb0 0x5568fd45a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670f400640 1 -- 172.21.15.62:0/2233283502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568fd4448a0 con 0x5568fd4457b0 2024-04-02T14:45:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670ea00640 1 --2- 172.21.15.62:0/2233283502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4460d0 0x5568fd44ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.767+0000 7f670f400640 1 --2- 172.21.15.62:0/2233283502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 0x5568fd44e480 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6700009230 tx=0x7f67000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ef1ab3d9cb4411 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.768+0000 7f66fb400640 1 -- 172.21.15.62:0/2233283502 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700063070 con 0x5568fd4457b0 2024-04-02T14:45:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.768+0000 7f66fb400640 1 -- 172.21.15.62:0/2233283502 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f670002c7f0 con 0x5568fd4457b0 2024-04-02T14:45:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.768+0000 7f66fb400640 1 -- 172.21.15.62:0/2233283502 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670004f040 con 0x5568fd4457b0 2024-04-02T14:45:52.766 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/vexpress.txt 2024-04-02T14:45:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.769+0000 7f6718e40740 1 -- 172.21.15.62:0/2233283502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 msgr2=0x5568fd44e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.769+0000 7f6718e40740 1 --2- 172.21.15.62:0/2233283502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 0x5568fd44e480 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6700009230 tx=0x7f67000057f0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.769+0000 7f6718e40740 1 -- 172.21.15.62:0/2233283502 shutdown_connections 2024-04-02T14:45:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.769+0000 7f6718e40740 1 --2- 172.21.15.62:0/2233283502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd44ecb0 0x5568fd45a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.769+0000 7f6718e40740 1 --2- 172.21.15.62:0/2233283502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd4460d0 0x5568fd44ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.769+0000 7f6718e40740 1 --2- 172.21.15.62:0/2233283502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 0x5568fd44e480 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.769+0000 7f6718e40740 1 -- 172.21.15.62:0/2233283502 >> 172.21.15.62:0/2233283502 conn(0x5568fd43fee0 msgr2=0x5568fd441ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:52.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.769+0000 7f6718e40740 1 -- 172.21.15.62:0/2233283502 shutdown_connections 2024-04-02T14:45:52.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.769+0000 7f6718e40740 1 -- 172.21.15.62:0/2233283502 wait complete. 2024-04-02T14:45:52.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.769+0000 7f6718e40740 1 -- start start 2024-04-02T14:45:52.768 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/ 2024-04-02T14:45:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f6718e40740 1 Processor -- start 2024-04-02T14:45:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f6718e40740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 0x5568fd452920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f6718e40740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4460d0 0x5568fd452e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f6718e40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd44ecb0 0x5568fd4533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f6718e40740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f67101b95f0 con 0x5568fd4457b0 2024-04-02T14:45:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f6718e40740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67101b8f20 con 0x5568fd4460d0 2024-04-02T14:45:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f6718e40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67101b6160 con 0x5568fd44ecb0 2024-04-02T14:45:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f6718e40740 10 client.0 _ll_register_callbacks cb 0x7f67080093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f670ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4460d0 0x5568fd452e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f670f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 0x5568fd452920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f670f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 0x5568fd452920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53884/0 (socket says 172.21.15.62:53884) 2024-04-02T14:45:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f670ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4460d0 0x5568fd452e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:56314/0 (socket says 172.21.15.62:56314) 2024-04-02T14:45:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f670fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd44ecb0 0x5568fd4533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.770+0000 7f670f400640 1 -- 172.21.15.62:0/1976589586 learned_addr learned my addr 172.21.15.62:0/1976589586 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:52.770 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt 2024-04-02T14:45:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.771+0000 7f670ea00640 1 -- 172.21.15.62:0/1976589586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd44ecb0 msgr2=0x5568fd4533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.771+0000 7f670ea00640 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd44ecb0 0x5568fd4533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.771+0000 7f670ea00640 1 -- 172.21.15.62:0/1976589586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 msgr2=0x5568fd452920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.771+0000 7f670ea00640 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 0x5568fd452920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.771+0000 7f670ea00640 1 -- 172.21.15.62:0/1976589586 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568fd4c43e0 con 0x5568fd4460d0 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.771+0000 7f670fe00640 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd44ecb0 0x5568fd4533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.771+0000 7f670ea00640 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4460d0 0x5568fd452e60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00c6a0 tx=0x7f66fc0bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f3ec9b7e96141f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.771+0000 7f66f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.771+0000 7f66f8c00640 1 -- 172.21.15.62:0/1976589586 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0d5040 con 0x5568fd4460d0 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.771+0000 7f66f8c00640 1 -- 172.21.15.62:0/1976589586 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f66fc0bd0b0 con 0x5568fd4460d0 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.771+0000 7f6718e40740 1 -- 172.21.15.62:0/1976589586 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f670002c7f0 con 0x5568fd4460d0 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f66f8c00640 1 -- 172.21.15.62:0/1976589586 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f66fc099240 con 0x5568fd4460d0 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f6718e40740 10 client.26102 fetch_fsmap learned FSMap version 29 2024-04-02T14:45:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f6718e40740 1 -- 172.21.15.62:0/1976589586 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5568fd4c57a0 con 0x5568fd4460d0 2024-04-02T14:45:52.773 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/ 2024-04-02T14:45:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f66f8c00640 1 -- 172.21.15.62:0/1976589586 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f670002c7f0 con 0x5568fd4460d0 2024-04-02T14:45:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f66f8c00640 1 -- 172.21.15.62:0/1976589586 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f66fc0b8080 con 0x5568fd4460d0 2024-04-02T14:45:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f6718e40740 10 client.26102 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T14:45:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f6718e40740 10 client.26102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f6718e40740 20 client.26102 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f6718e40740 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5568fd4c5ef0 0x5568fd4c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f6718e40740 4 client.26102 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:45:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f6718e40740 1 -- 172.21.15.62:0/1976589586 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5568fd3b6cb0 con 0x5568fd4c5ef0 2024-04-02T14:45:52.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.772+0000 7f670f400640 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5568fd4c5ef0 0x5568fd4c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:52.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.773+0000 7f670f400640 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5568fd4c5ef0 0x5568fd4c8310 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b9c6412a26c1b79 server_cookie=59147407b74b0449 in_seq=0 out_seq=0 2024-04-02T14:45:52.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.773+0000 7f66f8c00640 10 client.26102 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:52.776 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/allwinner,sunxi-clock-reset.txt 2024-04-02T14:45:52.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f66f8c00640 1 -- 172.21.15.62:0/1976589586 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5568fd3b6cb0 con 0x5568fd4c5ef0 2024-04-02T14:45:52.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f66f8c00640 10 client.26102 handle_command_reply: tid=0 2024-04-02T14:45:52.776 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/fsl,imx-src.txt 2024-04-02T14:45:52.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f6718e40740 1 client.26102 shutdown 2024-04-02T14:45:52.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f6718e40740 1 -- 172.21.15.62:0/1976589586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4460d0 msgr2=0x5568fd452e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f6718e40740 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4460d0 0x5568fd452e60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00c6a0 tx=0x7f66fc0bd040 comp rx=0 tx=0).stop 2024-04-02T14:45:52.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f6718e40740 1 -- 172.21.15.62:0/1976589586 shutdown_connections 2024-04-02T14:45:52.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f6718e40740 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5568fd4c5ef0 0x5568fd4c8310 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f6718e40740 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568fd44ecb0 0x5568fd4533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f66f8c00640 0 client.26102 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:52.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f6718e40740 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5568fd4460d0 0x5568fd452e60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f6718e40740 1 --2- 172.21.15.62:0/1976589586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568fd4457b0 0x5568fd452920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.775+0000 7f6718e40740 1 -- 172.21.15.62:0/1976589586 >> 172.21.15.62:0/1976589586 conn(0x5568fd43fee0 msgr2=0x5568fd44bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:52.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.776+0000 7f6718e40740 1 -- 172.21.15.62:0/1976589586 shutdown_connections 2024-04-02T14:45:52.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.776+0000 7f6718e40740 1 -- 172.21.15.62:0/1976589586 wait complete. 2024-04-02T14:45:52.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.776+0000 7f6718e40740 20 client.26102 trim_cache size 0 max 16384 2024-04-02T14:45:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:45:52.779 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "scrub active (173 inodes in the stack)", 2024-04-02T14:45:52.779 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-02T14:45:52.779 INFO:teuthology.orchestra.run.smithi062.stdout: "bc3dbfe7-3363-43cf-858c-341b5ce1a3e6": { 2024-04-02T14:45:52.779 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-04-02T14:45:52.779 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "bc3dbfe7-3363-43cf-858c-341b5ce1a3e6", 2024-04-02T14:45:52.779 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-04-02T14:45:52.779 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:45:52.779 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:45:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:45:52.780 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/reset.txt 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.776+0000 7f6716200640 1 -- 172.21.15.62:0/2488289905 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66f40a3d20 msgr2=0x7f66f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.776+0000 7f6716200640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66f40a3d20 0x7f66f40a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6700006010 tx=0x7f6700005e40 comp rx=0 tx=0).stop 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.776+0000 7f6716200640 1 -- 172.21.15.62:0/2488289905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6710109a00 msgr2=0x7f67101a3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.776+0000 7f6716200640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6710109a00 0x7f67101a3410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00e9f0 tx=0x7f66fc00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.777+0000 7f6716200640 1 -- 172.21.15.62:0/2488289905 shutdown_connections 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.777+0000 7f6716200640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66f40a3d20 0x7f66f40a61e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.777+0000 7f6716200640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710116c00 0x7f67101a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.777+0000 7f6716200640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6710109a00 0x7f67101a3410 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.777+0000 7f6716200640 1 --2- 172.21.15.62:0/2488289905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6710109050 0x7f67101a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.777+0000 7f6716200640 1 -- 172.21.15.62:0/2488289905 >> 172.21.15.62:0/2488289905 conn(0x7f6710072f50 msgr2=0x7f67101181c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.777+0000 7f6716200640 1 -- 172.21.15.62:0/2488289905 shutdown_connections 2024-04-02T14:45:52.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:45:52.777+0000 7f6716200640 1 -- 172.21.15.62:0/2488289905 wait complete. 2024-04-02T14:45:52.782 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/sirf,rstc.txt 2024-04-02T14:45:52.782 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/socfpga-reset.txt 2024-04-02T14:45:52.782 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-picophyreset.txt 2024-04-02T14:45:52.783 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-powerdown.txt 2024-04-02T14:45:52.783 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-softreset.txt 2024-04-02T14:45:52.783 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/resource-names.txt 2024-04-02T14:45:52.783 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/ 2024-04-02T14:45:52.783 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/apm,rng.txt 2024-04-02T14:45:52.784 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/brcm,bcm2835.txt 2024-04-02T14:45:52.786 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/qcom,prng.txt 2024-04-02T14:45:52.787 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/ 2024-04-02T14:45:52.788 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/armada-380-rtc.txt 2024-04-02T14:45:52.789 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91rm9200-rtc.txt 2024-04-02T14:45:52.791 INFO:tasks.cephfs.filesystem:scrub status for tag:bc3dbfe7-3363-43cf-858c-341b5ce1a3e6 - {'path': '/', 'tag': 'bc3dbfe7-3363-43cf-858c-341b5ce1a3e6', 'options': 'recursive,force'} 2024-04-02T14:45:52.791 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91sam9-rtc.txt 2024-04-02T14:45:52.792 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dallas,ds1339.txt 2024-04-02T14:45:52.793 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dw-apb.txt 2024-04-02T14:45:52.794 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt 2024-04-02T14:45:52.796 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/imxdi-rtc.txt 2024-04-02T14:45:52.797 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/isil,isl12057.txt 2024-04-02T14:45:52.799 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/lpc32xx-rtc.txt 2024-04-02T14:45:52.801 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/maxim,ds1742.txt 2024-04-02T14:45:52.802 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/moxa,moxart-rtc.txt 2024-04-02T14:45:52.803 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt 2024-04-02T14:45:52.805 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nxp,rtc-2123.txt 2024-04-02T14:45:52.807 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/olpc-xo1-rtc.txt 2024-04-02T14:45:52.808 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/orion-rtc.txt 2024-04-02T14:45:52.809 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/pxa-rtc.txt 2024-04-02T14:45:52.811 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-cmos.txt 2024-04-02T14:45:52.813 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-omap.txt 2024-04-02T14:45:52.814 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-opal.txt 2024-04-02T14:45:52.816 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-palmas.txt 2024-04-02T14:45:52.818 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/s3c-rtc.txt 2024-04-02T14:45:52.820 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt 2024-04-02T14:45:52.822 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/snvs-rtc.txt 2024-04-02T14:45:52.824 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/spear-rtc.txt 2024-04-02T14:45:52.826 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/stmp3xxx-rtc.txt 2024-04-02T14:45:52.827 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sun6i-rtc.txt 2024-04-02T14:45:52.830 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sunxi-rtc.txt 2024-04-02T14:45:52.832 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/twl-rtc.txt 2024-04-02T14:45:52.835 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/via,vt8500-rtc.txt 2024-04-02T14:45:52.837 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/xgene-rtc.txt 2024-04-02T14:45:52.839 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/ 2024-04-02T14:45:52.841 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/ 2024-04-02T14:45:52.842 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/st33zp24-i2c.txt 2024-04-02T14:45:52.844 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/ 2024-04-02T14:45:52.846 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/8250.txt 2024-04-02T14:45:52.847 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_jtaguart.txt 2024-04-02T14:45:52.849 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_uart.txt 2024-04-02T14:45:52.851 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/arc-uart.txt 2024-04-02T14:45:52.854 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/atmel-usart.txt 2024-04-02T14:45:52.856 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/axis,etraxfs-uart.txt 2024-04-02T14:45:52.858 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/bcm63xx-uart.txt 2024-04-02T14:45:52.860 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cavium-uart.txt 2024-04-02T14:45:52.862 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cdns,uart.txt 2024-04-02T14:45:52.864 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cirrus,clps711x-uart.txt 2024-04-02T14:45:52.866 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/digicolor-usart.txt 2024-04-02T14:45:52.868 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/efm32-uart.txt 2024-04-02T14:45:52.870 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt 2024-04-02T14:45:52.872 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-lpuart.txt 2024-04-02T14:45:52.875 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt 2024-04-02T14:45:52.876 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/lantiq_asc.txt 2024-04-02T14:45:52.879 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/maxim,max310x.txt 2024-04-02T14:45:52.880 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl,pxa-ssp.txt 2024-04-02T14:45:52.883 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl-serial.txt 2024-04-02T14:45:52.885 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mtk-uart.txt 2024-04-02T14:45:52.887 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nvidia,tegra20-hsuart.txt 2024-04-02T14:45:52.889 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp,sc16is7xx.txt 2024-04-02T14:45:52.890 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp-lpc32xx-hsuart.txt 2024-04-02T14:45:52.892 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/omap_serial.txt 2024-04-02T14:45:52.894 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/pl011.txt 2024-04-02T14:45:52.896 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qca,ar9330-uart.txt 2024-04-02T14:45:52.898 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uart.txt 2024-04-02T14:45:52.900 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uartdm.txt 2024-04-02T14:45:52.903 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/renesas,sci-serial.txt 2024-04-02T14:45:52.905 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/rs485.txt 2024-04-02T14:45:52.907 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/samsung_uart.txt 2024-04-02T14:45:52.908 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sirf-uart.txt 2024-04-02T14:45:52.912 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.txt 2024-04-02T14:45:52.914 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sprd-uart.txt 2024-04-02T14:45:52.916 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/st-asc.txt 2024-04-02T14:45:52.918 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/vt8500-uart.txt 2024-04-02T14:45:52.920 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/ 2024-04-02T14:45:52.922 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/allwinner,sun4i-ps2.txt 2024-04-02T14:45:52.924 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/altera_ps2.txt 2024-04-02T14:45:52.927 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/olpc,ap-sp.txt 2024-04-02T14:45:52.929 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/snps-arc_ps2.txt 2024-04-02T14:45:52.931 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ 2024-04-02T14:45:52.933 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/ 2024-04-02T14:45:52.937 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman-portals.txt 2024-04-02T14:45:52.937 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman.txt 2024-04-02T14:45:52.939 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman-portals.txt 2024-04-02T14:45:52.940 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman.txt 2024-04-02T14:45:52.942 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/ 2024-04-02T14:45:52.943 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/qcom,gsbi.txt 2024-04-02T14:45:52.945 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/ 2024-04-02T14:45:52.947 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-dma.txt 2024-04-02T14:45:52.949 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt 2024-04-02T14:45:52.951 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ 2024-04-02T14:45:52.953 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,adau1701.txt 2024-04-02T14:45:52.955 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-i2s.txt 2024-04-02T14:45:52.956 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-spdif-tx.txt 2024-04-02T14:45:52.958 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,ssm2602.txt 2024-04-02T14:45:52.960 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4104.txt 2024-04-02T14:45:52.962 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4554.c 2024-04-02T14:45:52.963 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4642.txt 2024-04-02T14:45:52.965 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak5386.txt 2024-04-02T14:45:52.967 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5623.txt 2024-04-02T14:45:52.969 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5632.txt 2024-04-02T14:45:52.971 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/armada-370db-audio.txt 2024-04-02T14:45:52.973 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/arndale.txt 2024-04-02T14:45:52.974 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-at91sam9g20ek-wm8731-audio.txt 2024-04-02T14:45:52.976 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-sam9x5-wm8731-audio.txt 2024-04-02T14:45:52.978 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-wm8904.txt 2024-04-02T14:45:52.980 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel_ac97c.txt 2024-04-02T14:45:52.982 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/bcm2835-i2s.txt 2024-04-02T14:45:52.985 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cdns,xtfpga-i2s.txt 2024-04-02T14:45:52.987 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs35l32.txt 2024-04-02T14:45:52.988 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4265.txt 2024-04-02T14:45:52.990 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4270.txt 2024-04-02T14:45:52.992 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4271.txt 2024-04-02T14:45:52.994 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l52.txt 2024-04-02T14:45:52.996 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l56.txt 2024-04-02T14:45:52.998 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l73.txt 2024-04-02T14:45:53.000 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42xx8.txt 2024-04-02T14:45:53.001 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/da9055.txt 2024-04-02T14:45:53.004 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-evm-audio.txt 2024-04-02T14:45:53.005 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-mcasp-audio.txt 2024-04-02T14:45:53.007 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/designware-i2s.txt 2024-04-02T14:45:53.009 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/es8328.txt 2024-04-02T14:45:53.010 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/eukrea-tlv320.txt 2024-04-02T14:45:53.012 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,asrc.txt 2024-04-02T14:45:53.014 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,esai.txt 2024-04-02T14:45:53.016 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,spdif.txt 2024-04-02T14:45:53.018 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,ssi.txt 2024-04-02T14:45:53.021 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-asoc-card.txt 2024-04-02T14:45:53.023 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-sai.txt 2024-04-02T14:45:53.025 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/hdmi.txt 2024-04-02T14:45:53.026 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-es8328.txt 2024-04-02T14:45:53.028 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-sgtl5000.txt 2024-04-02T14:45:53.031 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-spdif.txt 2024-04-02T14:45:53.033 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-wm8962.txt 2024-04-02T14:45:53.035 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audmux.txt 2024-04-02T14:45:53.038 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ingenic,jz4740-i2s.txt 2024-04-02T14:45:53.040 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98090.txt 2024-04-02T14:45:53.042 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98095.txt 2024-04-02T14:45:53.044 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98357a.txt 2024-04-02T14:45:53.046 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa-ssp.txt 2024-04-02T14:45:53.047 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa2xx-pcm.txt 2024-04-02T14:45:53.050 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mvebu-audio.txt 2024-04-02T14:45:53.052 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-audio-sgtl5000.txt 2024-04-02T14:45:53.054 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-saif.txt 2024-04-02T14:45:53.055 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nokia,rx51.txt 2024-04-02T14:45:53.057 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-alc5632.txt 2024-04-02T14:45:53.059 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-max98090.txt 2024-04-02T14:45:53.064 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5640.txt 2024-04-02T14:45:53.066 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5677.txt 2024-04-02T14:45:53.071 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-trimslice.txt 2024-04-02T14:45:53.073 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8753.txt 2024-04-02T14:45:53.074 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8903.txt 2024-04-02T14:45:53.076 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm9712.txt 2024-04-02T14:45:53.078 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-ac97.txt 2024-04-02T14:45:53.079 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-das.txt 2024-04-02T14:45:53.081 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-i2s.txt 2024-04-02T14:45:53.083 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-ahub.txt 2024-04-02T14:45:53.084 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt 2024-04-02T14:45:53.088 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-i2s.txt 2024-04-02T14:45:53.090 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-abe-twl6040.txt 2024-04-02T14:45:53.091 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-dmic.txt 2024-04-02T14:45:53.093 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcbsp.txt 2024-04-02T14:45:53.095 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcpdm.txt 2024-04-02T14:45:53.096 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-twl4030.txt 2024-04-02T14:45:53.098 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm1792a.txt 2024-04-02T14:45:53.100 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm512x.txt 2024-04-02T14:45:53.102 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,fsi.txt 2024-04-02T14:45:53.104 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,rsnd.txt 2024-04-02T14:45:53.106 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rockchip-i2s.txt 2024-04-02T14:45:53.107 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5631.txt 2024-04-02T14:45:53.109 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5640.txt 2024-04-02T14:45:53.111 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5677.txt 2024-04-02T14:45:53.113 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,odroidx2-max98090.txt 2024-04-02T14:45:53.115 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,smdk-wm8994.txt 2024-04-02T14:45:53.116 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung-i2s.txt 2024-04-02T14:45:53.118 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sgtl5000.txt 2024-04-02T14:45:53.120 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/simple-card.txt 2024-04-02T14:45:53.121 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-codec.txt 2024-04-02T14:45:53.123 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-port.txt 2024-04-02T14:45:53.125 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio.txt 2024-04-02T14:45:53.127 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-usp.txt 2024-04-02T14:45:53.129 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/snow.txt 2024-04-02T14:45:53.130 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/soc-ac97link.txt 2024-04-02T14:45:53.132 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-receiver.txt 2024-04-02T14:45:53.134 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-transmitter.txt 2024-04-02T14:45:53.135 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm2518.txt 2024-04-02T14:45:53.137 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm4567.txt 2024-04-02T14:45:53.139 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta32x.txt 2024-04-02T14:45:53.141 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta350.txt 2024-04-02T14:45:53.142 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tas2552.txt 2024-04-02T14:45:53.144 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tdm-slot.txt 2024-04-02T14:45:53.146 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,pcm1681.txt 2024-04-02T14:45:53.147 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,tas5086.txt 2024-04-02T14:45:53.149 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic31xx.txt 2024-04-02T14:45:53.150 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic32x4.txt 2024-04-02T14:45:53.152 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic3x.txt 2024-04-02T14:45:53.154 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tpa6130a2.txt 2024-04-02T14:45:53.156 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ts3a227e.txt 2024-04-02T14:45:53.157 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-mop500.txt 2024-04-02T14:45:53.159 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-msp.txt 2024-04-02T14:45:53.161 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/widgets.txt 2024-04-02T14:45:53.162 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8510.txt 2024-04-02T14:45:53.164 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8523.txt 2024-04-02T14:45:53.165 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8580.txt 2024-04-02T14:45:53.167 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8711.txt 2024-04-02T14:45:53.169 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8728.txt 2024-04-02T14:45:53.170 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8731.txt 2024-04-02T14:45:53.172 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8737.txt 2024-04-02T14:45:53.173 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8741.txt 2024-04-02T14:45:53.175 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8750.txt 2024-04-02T14:45:53.177 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8753.txt 2024-04-02T14:45:53.178 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8770.txt 2024-04-02T14:45:53.180 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8776.txt 2024-04-02T14:45:53.182 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8804.txt 2024-04-02T14:45:53.183 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8903.txt 2024-04-02T14:45:53.185 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8904.txt 2024-04-02T14:45:53.186 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8960.txt 2024-04-02T14:45:53.188 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8962.txt 2024-04-02T14:45:53.190 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8994.txt 2024-04-02T14:45:53.191 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ 2024-04-02T14:45:53.193 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/brcm,bcm2835-spi.txt 2024-04-02T14:45:53.194 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/efm32-spi.txt 2024-04-02T14:45:53.196 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-imx-cspi.txt 2024-04-02T14:45:53.198 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-spi.txt 2024-04-02T14:45:53.199 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/mxs-spi.txt 2024-04-02T14:45:53.201 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra114-spi.txt 2024-04-02T14:45:53.203 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-sflash.txt 2024-04-02T14:45:53.204 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-slink.txt 2024-04-02T14:45:53.206 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/omap-spi.txt 2024-04-02T14:45:53.207 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/qcom,spi-qup.txt 2024-04-02T14:45:53.209 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-hspi.txt 2024-04-02T14:45:53.211 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-msiof.txt 2024-04-02T14:45:53.213 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/snps,dw-apb-ssi.txt 2024-04-02T14:45:53.214 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-bus.txt 2024-04-02T14:45:53.216 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-cadence.txt 2024-04-02T14:45:53.218 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-davinci.txt 2024-04-02T14:45:53.219 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-dw.txt 2024-04-02T14:45:53.221 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt 2024-04-02T14:45:53.223 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-gpio.txt 2024-04-02T14:45:53.224 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-img-spfi.txt 2024-04-02T14:45:53.226 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-meson.txt 2024-04-02T14:45:53.228 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-octeon.txt 2024-04-02T14:45:53.229 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-orion.txt 2024-04-02T14:45:53.231 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rockchip.txt 2024-04-02T14:45:53.233 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rspi.txt 2024-04-02T14:45:53.234 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-samsung.txt 2024-04-02T14:45:53.236 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sc18is602.txt 2024-04-02T14:45:53.238 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sirf.txt 2024-04-02T14:45:53.240 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-st-ssc.txt 2024-04-02T14:45:53.241 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun4i.txt 2024-04-02T14:45:53.243 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun6i.txt 2024-04-02T14:45:53.244 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-xtensa-xtfpga.txt 2024-04-02T14:45:53.246 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_altera.txt 2024-04-02T14:45:53.248 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_atmel.txt 2024-04-02T14:45:53.250 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_oc_tiny.txt 2024-04-02T14:45:53.251 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_pl022.txt 2024-04-02T14:45:53.253 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ti_qspi.txt 2024-04-02T14:45:53.255 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/ 2024-04-02T14:45:53.257 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/qcom,spmi-pmic-arb.txt 2024-04-02T14:45:53.259 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/spmi.txt 2024-04-02T14:45:53.261 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/ 2024-04-02T14:45:53.262 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/ 2024-04-02T14:45:53.264 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/ 2024-04-02T14:45:53.265 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/lpc32xx-adc.txt 2024-04-02T14:45:53.267 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt 2024-04-02T14:45:53.269 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/spear-adc.txt 2024-04-02T14:45:53.270 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/submitting-patches.txt 2024-04-02T14:45:53.272 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ 2024-04-02T14:45:53.274 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/armada-thermal.txt 2024-04-02T14:45:53.276 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/db8500-thermal.txt 2024-04-02T14:45:53.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/dove-thermal.txt 2024-04-02T14:45:53.279 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/exynos-thermal.txt 2024-04-02T14:45:53.281 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/imx-thermal.txt 2024-04-02T14:45:53.283 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/kirkwood-thermal.txt 2024-04-02T14:45:53.284 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rcar-thermal.txt 2024-04-02T14:45:53.286 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rockchip-thermal.txt 2024-04-02T14:45:53.288 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/spear-thermal.txt 2024-04-02T14:45:53.290 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/st-thermal.txt 2024-04-02T14:45:53.291 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt 2024-04-02T14:45:53.293 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/thermal.txt 2024-04-02T14:45:53.295 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ti_soc_thermal.txt 2024-04-02T14:45:53.297 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ 2024-04-02T14:45:53.298 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun4i-timer.txt 2024-04-02T14:45:53.300 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun5i-a13-hstimer.txt 2024-04-02T14:45:53.302 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/amlogic,meson6-timer.txt 2024-04-02T14:45:53.303 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/arm,sp804.txt 2024-04-02T14:45:53.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/brcm,bcm2835-system-timer.txt 2024-04-02T14:45:53.307 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cadence,ttc-timer.txt 2024-04-02T14:45:53.309 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cirrus,clps711x-timer.txt 2024-04-02T14:45:53.311 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/digicolor-timer.txt 2024-04-02T14:45:53.313 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/energymicro,efm32-timer.txt 2024-04-02T14:45:53.315 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,ftm-timer.txt 2024-04-02T14:45:53.316 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt 2024-04-02T14:45:53.318 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/lsi,zevio-timer.txt 2024-04-02T14:45:53.320 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,armada-370-xp-timer.txt 2024-04-02T14:45:53.322 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,orion-timer.txt 2024-04-02T14:45:53.324 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/mediatek,mtk-timer.txt 2024-04-02T14:45:53.326 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/moxa,moxart-timer.txt 2024-04-02T14:45:53.327 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra20-timer.txt 2024-04-02T14:45:53.329 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra30-timer.txt 2024-04-02T14:45:53.331 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,cmt.txt 2024-04-02T14:45:53.333 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,mtu2.txt 2024-04-02T14:45:53.334 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,tmu.txt 2024-04-02T14:45:53.336 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/rockchip,rk3288-timer.txt 2024-04-02T14:45:53.338 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/samsung,exynos4210-mct.txt 2024-04-02T14:45:53.340 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/stericsson-u300-apptimer.txt 2024-04-02T14:45:53.342 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ti,keystone-timer.txt 2024-04-02T14:45:53.344 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ 2024-04-02T14:45:53.346 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ufshcd-pltfrm.txt 2024-04-02T14:45:53.349 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/unittest.txt 2024-04-02T14:45:53.350 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ 2024-04-02T14:45:53.351 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/am33xx-usb.txt 2024-04-02T14:45:53.353 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/atmel-usb.txt 2024-04-02T14:45:53.355 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-imx.txt 2024-04-02T14:45:53.357 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-qcom.txt 2024-04-02T14:45:53.358 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-usb2.txt 2024-04-02T14:45:53.360 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-zevio.txt 2024-04-02T14:45:53.362 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc2.txt 2024-04-02T14:45:53.364 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3-st.txt 2024-04-02T14:45:53.366 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3.txt 2024-04-02T14:45:53.368 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-omap.txt 2024-04-02T14:45:53.370 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-orion.txt 2024-04-02T14:45:53.371 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-st.txt 2024-04-02T14:45:53.374 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/exynos-usb.txt 2024-04-02T14:45:53.376 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/fsl-usb.txt 2024-04-02T14:45:53.377 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/generic.txt 2024-04-02T14:45:53.379 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/gr-udc.txt 2024-04-02T14:45:53.381 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/isp1301.txt 2024-04-02T14:45:53.383 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-phy.txt 2024-04-02T14:45:53.385 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-usb.txt 2024-04-02T14:45:53.387 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/lpc32xx-udc.txt 2024-04-02T14:45:53.388 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/msm-hsusb.txt 2024-04-02T14:45:53.390 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/mxs-phy.txt 2024-04-02T14:45:53.392 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-ehci.txt 2024-04-02T14:45:53.394 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-usb-phy.txt 2024-04-02T14:45:53.396 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-nxp.txt 2024-04-02T14:45:53.397 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-omap3.txt 2024-04-02T14:45:53.399 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-st.txt 2024-04-02T14:45:53.401 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/omap-usb.txt 2024-04-02T14:45:53.403 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/pxa-usb.txt 2024-04-02T14:45:53.404 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/qcom,dwc3.txt 2024-04-02T14:45:53.406 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/renesas_usbhs.txt 2024-04-02T14:45:53.408 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-hsotg.txt 2024-04-02T14:45:53.410 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-usbphy.txt 2024-04-02T14:45:53.411 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/spear-usb.txt 2024-04-02T14:45:53.413 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/twlxxxx-usb.txt 2024-04-02T14:45:53.415 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/udc-xilinx.txt 2024-04-02T14:45:53.417 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ehci.txt 2024-04-02T14:45:53.419 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-nop-xceiv.txt 2024-04-02T14:45:53.421 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ohci.txt 2024-04-02T14:45:53.422 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-uhci.txt 2024-04-02T14:45:53.424 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-xhci.txt 2024-04-02T14:45:53.425 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb3503.txt 2024-04-02T14:45:53.427 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usbmisc-imx.txt 2024-04-02T14:45:53.429 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ux500-usb.txt 2024-04-02T14:45:53.431 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/vendor-prefixes.txt 2024-04-02T14:45:53.433 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ 2024-04-02T14:45:53.434 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7123.txt 2024-04-02T14:45:53.436 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7511.txt 2024-04-02T14:45:53.437 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/analog-tv-connector.txt 2024-04-02T14:45:53.439 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/arm,pl11x.txt 2024-04-02T14:45:53.441 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/atmel,lcdc.txt 2024-04-02T14:45:53.443 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/ 2024-04-02T14:45:53.444 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/88pm860x.txt 2024-04-02T14:45:53.447 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/gpio-backlight.txt 2024-04-02T14:45:53.448 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/lp855x.txt 2024-04-02T14:45:53.449 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/max8925-backlight.txt 2024-04-02T14:45:53.451 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/pwm-backlight.txt 2024-04-02T14:45:53.453 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/tps65217-backlight.txt 2024-04-02T14:45:53.455 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ 2024-04-02T14:45:53.456 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ps8622.txt 2024-04-02T14:45:53.458 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ptn3460.txt 2024-04-02T14:45:53.460 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/cirrus,clps711x-fb.txt 2024-04-02T14:45:53.462 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/display-timing.txt 2024-04-02T14:45:53.463 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dvi-connector.txt 2024-04-02T14:45:53.465 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dw_hdmi-rockchip.txt 2024-04-02T14:45:53.467 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos7-decon.txt 2024-04-02T14:45:53.469 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dp.txt 2024-04-02T14:45:53.472 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dsim.txt 2024-04-02T14:45:53.474 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmi.txt 2024-04-02T14:45:53.477 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiddc.txt 2024-04-02T14:45:53.480 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiphy.txt 2024-04-02T14:45:53.483 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_mixer.txt 2024-04-02T14:45:53.485 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/fsl,imx-fb.txt 2024-04-02T14:45:53.487 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/hdmi-connector.txt 2024-04-02T14:45:53.488 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/lgphilips,lb035q02.txt 2024-04-02T14:45:53.490 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dpi.txt 2024-04-02T14:45:53.492 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dsi-cm.txt 2024-04-02T14:45:53.494 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/renesas,du.txt 2024-04-02T14:45:53.496 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-drm.txt 2024-04-02T14:45:53.731 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-vop.txt 2024-04-02T14:45:53.732 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/samsung-fimd.txt 2024-04-02T14:45:53.734 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sharp,ls037v7dw01.txt 2024-04-02T14:45:53.736 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer-sunxi.txt 2024-04-02T14:45:53.737 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer.txt 2024-04-02T14:45:53.739 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sony,acx565akm.txt 2024-04-02T14:45:53.741 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1289fb.txt 2024-04-02T14:45:53.743 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1307fb.txt 2024-04-02T14:45:53.744 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/thine,thc63lvdm83d 2024-04-02T14:45:53.746 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,dra7-dss.txt 2024-04-02T14:45:53.748 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap-dss.txt 2024-04-02T14:45:53.750 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap2-dss.txt 2024-04-02T14:45:53.753 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap3-dss.txt 2024-04-02T14:45:53.753 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap4-dss.txt 2024-04-02T14:45:53.755 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap5-dss.txt 2024-04-02T14:45:53.757 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,opa362.txt 2024-04-02T14:45:53.759 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tfp410.txt 2024-04-02T14:45:53.760 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tpd12s015.txt 2024-04-02T14:45:53.762 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/toppoly,td028ttec1.txt 2024-04-02T14:45:53.764 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/tpo,td043mtea1.txt 2024-04-02T14:45:53.766 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/vga-connector.txt 2024-04-02T14:45:53.767 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/via,vt8500-fb.txt 2024-04-02T14:45:53.769 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,prizm-ge-rops.txt 2024-04-02T14:45:53.771 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,wm8505-fb.txt 2024-04-02T14:45:53.773 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/ 2024-04-02T14:45:53.774 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/mmio.txt 2024-04-02T14:45:53.776 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/ 2024-04-02T14:45:53.777 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/fsl-imx-owire.txt 2024-04-02T14:45:53.779 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/omap-hdq.txt 2024-04-02T14:45:53.781 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/w1-gpio.txt 2024-04-02T14:45:53.783 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ 2024-04-02T14:45:53.784 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-at91rm9200-wdt.txt 2024-04-02T14:45:53.786 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-wdt.txt 2024-04-02T14:45:53.787 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/brcm,bcm2835-pm-wdog.txt 2024-04-02T14:45:53.789 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/cadence-wdt.txt 2024-04-02T14:45:53.791 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/davinci-wdt.txt 2024-04-02T14:45:53.793 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/dw_wdt.txt 2024-04-02T14:45:53.794 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/fsl-imx-wdt.txt 2024-04-02T14:45:53.796 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/gpio-wdt.txt 2024-04-02T14:45:53.798 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/imgpdc-wdt.txt 2024-04-02T14:45:53.799 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ingenic,jz4740-wdt.txt 2024-04-02T14:45:53.801 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/marvel.txt 2024-04-02T14:45:53.803 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/men-a021-wdt.txt 2024-04-02T14:45:53.804 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/meson6-wdt.txt 2024-04-02T14:45:53.806 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/moxa,moxart-watchdog.txt 2024-04-02T14:45:53.808 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/mtk-wdt.txt 2024-04-02T14:45:53.809 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/of-xilinx-wdt.txt 2024-04-02T14:45:53.811 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/omap-wdt.txt 2024-04-02T14:45:53.813 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/pnx4008-wdt.txt 2024-04-02T14:45:53.814 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qca-ar7130-wdt.txt 2024-04-02T14:45:53.816 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qcom-wdt.txt 2024-04-02T14:45:53.818 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/rt2880-wdt.txt 2024-04-02T14:45:53.819 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/samsung-wdt.txt 2024-04-02T14:45:53.821 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sirfsoc_wdt.txt 2024-04-02T14:45:53.823 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/stericsson-coh901327.txt 2024-04-02T14:45:53.824 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sunxi-wdt.txt 2024-04-02T14:45:53.826 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/twl4030-wdt.txt 2024-04-02T14:45:53.828 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ 2024-04-02T14:45:53.829 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ce4100.txt 2024-04-02T14:45:53.831 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/interrupt.txt 2024-04-02T14:45:53.833 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/timer.txt 2024-04-02T14:45:53.834 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/xilinx.txt 2024-04-02T14:45:53.836 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/ 2024-04-02T14:45:53.838 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/xillybus.txt 2024-04-02T14:45:53.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:53 smithi062 ceph-mon[32116]: pgmap v59: 97 pgs: 97 active+clean; 56 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.6 MiB/s wr, 31 op/s 2024-04-02T14:45:53.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:53 smithi062 ceph-mon[32116]: scrub queued for path: / 2024-04-02T14:45:53.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:53 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:53.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:53 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:45:53.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:53 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:45:53.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:53 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:53.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:53 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:53.840 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/booting-without-of.txt 2024-04-02T14:45:53.842 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/changesets.txt 2024-04-02T14:45:53.844 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/dynamic-resolution-notes.txt 2024-04-02T14:45:53.846 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/of_selftest.txt 2024-04-02T14:45:53.847 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/overlay-notes.txt 2024-04-02T14:45:53.849 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/todo.txt 2024-04-02T14:45:53.850 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/devicetree/usage-model.txt 2024-04-02T14:45:53.852 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/digsig.txt 2024-04-02T14:45:53.854 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dma-buf-sharing.txt 2024-04-02T14:45:53.856 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dmaengine/ 2024-04-02T14:45:53.857 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dmaengine/00-INDEX 2024-04-02T14:45:53.859 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dmaengine/client.txt 2024-04-02T14:45:53.860 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dmaengine/dmatest.txt 2024-04-02T14:45:53.862 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dmaengine/provider.txt 2024-04-02T14:45:53.864 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dontdiff 2024-04-02T14:45:53.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:53 smithi113 ceph-mon[36135]: pgmap v59: 97 pgs: 97 active+clean; 56 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.6 MiB/s wr, 31 op/s 2024-04-02T14:45:53.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:53 smithi113 ceph-mon[36135]: scrub queued for path: / 2024-04-02T14:45:53.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:53 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:53.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:53 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:45:53.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:53 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:45:53.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:53 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:53.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:53 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:53.866 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/driver-model/ 2024-04-02T14:45:53.867 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/driver-model/binding.txt 2024-04-02T14:45:53.869 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/driver-model/bus.txt 2024-04-02T14:45:53.870 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/driver-model/class.txt 2024-04-02T14:45:53.872 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/driver-model/design-patterns.txt 2024-04-02T14:45:53.873 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/driver-model/device.txt 2024-04-02T14:45:53.875 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/driver-model/devres.txt 2024-04-02T14:45:53.877 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/driver-model/driver.txt 2024-04-02T14:45:53.878 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/driver-model/overview.txt 2024-04-02T14:45:53.880 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/driver-model/platform.txt 2024-04-02T14:45:53.882 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/driver-model/porting.txt 2024-04-02T14:45:53.883 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/ 2024-04-02T14:45:53.885 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/README.dvb-usb 2024-04-02T14:45:53.887 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/avermedia.txt 2024-04-02T14:45:53.888 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/bt8xx.txt 2024-04-02T14:45:53.890 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/cards.txt 2024-04-02T14:45:53.892 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/ci.txt 2024-04-02T14:45:53.893 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/contributors.txt 2024-04-02T14:45:53.895 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/faq.txt 2024-04-02T14:45:53.897 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/get_dvb_firmware 2024-04-02T14:45:53.898 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/lmedm04.txt 2024-04-02T14:45:53.900 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/opera-firmware.txt 2024-04-02T14:45:53.902 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/readme.txt 2024-04-02T14:45:53.903 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/technisat.txt 2024-04-02T14:45:53.905 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/ttusb-dec.txt 2024-04-02T14:45:53.907 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dvb/udev.txt 2024-04-02T14:45:53.908 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/dynamic-debug-howto.txt 2024-04-02T14:45:53.910 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/early-userspace/ 2024-04-02T14:45:53.912 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/early-userspace/README 2024-04-02T14:45:53.913 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/early-userspace/buffer-format.txt 2024-04-02T14:45:53.916 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/edac.txt 2024-04-02T14:45:53.917 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/efi-stub.txt 2024-04-02T14:45:53.919 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/eisa.txt 2024-04-02T14:45:53.921 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/email-clients.txt 2024-04-02T14:45:53.922 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/extcon/ 2024-04-02T14:45:53.924 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/extcon/porting-android-switch-class 2024-04-02T14:45:53.925 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fault-injection/ 2024-04-02T14:45:53.927 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fault-injection/fault-injection.txt 2024-04-02T14:45:53.929 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fault-injection/notifier-error-inject.txt 2024-04-02T14:45:53.930 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fault-injection/provoke-crashes.txt 2024-04-02T14:45:53.932 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/ 2024-04-02T14:45:53.933 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/00-INDEX 2024-04-02T14:45:53.935 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/api.txt 2024-04-02T14:45:53.937 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/arkfb.txt 2024-04-02T14:45:53.938 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/aty128fb.txt 2024-04-02T14:45:53.940 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/cirrusfb.txt 2024-04-02T14:45:53.942 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/cmap_xfbdev.txt 2024-04-02T14:45:53.944 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/deferred_io.txt 2024-04-02T14:45:53.946 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/efifb.txt 2024-04-02T14:45:53.947 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/ep93xx-fb.txt 2024-04-02T14:45:53.949 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/fbcon.txt 2024-04-02T14:45:53.950 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/framebuffer.txt 2024-04-02T14:45:53.952 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/gxfb.txt 2024-04-02T14:45:53.953 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/intel810.txt 2024-04-02T14:45:53.957 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/intelfb.txt 2024-04-02T14:45:53.957 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/internals.txt 2024-04-02T14:45:53.958 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/lxfb.txt 2024-04-02T14:45:53.960 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/matroxfb.txt 2024-04-02T14:45:53.962 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/metronomefb.txt 2024-04-02T14:45:53.963 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/modedb.txt 2024-04-02T14:45:53.965 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/pvr2fb.txt 2024-04-02T14:45:53.967 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/pxafb.txt 2024-04-02T14:45:53.969 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/s3fb.txt 2024-04-02T14:45:53.970 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/sa1100fb.txt 2024-04-02T14:45:53.972 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/sh7760fb.txt 2024-04-02T14:45:53.974 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/sisfb.txt 2024-04-02T14:45:53.976 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/sm501.txt 2024-04-02T14:45:53.977 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/sstfb.txt 2024-04-02T14:45:53.979 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/tgafb.txt 2024-04-02T14:45:53.980 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/tridentfb.txt 2024-04-02T14:45:53.982 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/udlfb.txt 2024-04-02T14:45:53.984 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/uvesafb.txt 2024-04-02T14:45:53.985 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/vesafb.txt 2024-04-02T14:45:53.987 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/viafb.modes 2024-04-02T14:45:53.989 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/viafb.txt 2024-04-02T14:45:53.991 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fb/vt8623fb.txt 2024-04-02T14:45:53.993 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ 2024-04-02T14:45:53.994 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/.gitignore 2024-04-02T14:45:53.996 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/00-INDEX 2024-04-02T14:45:53.997 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/9p.txt 2024-04-02T14:45:53.999 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/Locking 2024-04-02T14:45:54.001 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/Makefile 2024-04-02T14:45:54.002 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/adfs.txt 2024-04-02T14:45:54.004 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/affs.txt 2024-04-02T14:45:54.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:53 smithi204 ceph-mon[35948]: pgmap v59: 97 pgs: 97 active+clean; 56 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.6 MiB/s wr, 31 op/s 2024-04-02T14:45:54.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:53 smithi204 ceph-mon[35948]: scrub queued for path: / 2024-04-02T14:45:54.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:53 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:54.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:53 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:45:54.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:53 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:45:54.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:53 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:54.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:53 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:45:54.006 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/afs.txt 2024-04-02T14:45:54.007 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/autofs4-mount-control.txt 2024-04-02T14:45:54.009 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/autofs4.txt 2024-04-02T14:45:54.011 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/automount-support.txt 2024-04-02T14:45:54.013 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/befs.txt 2024-04-02T14:45:54.014 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/bfs.txt 2024-04-02T14:45:54.016 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/btrfs.txt 2024-04-02T14:45:54.018 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/caching/ 2024-04-02T14:45:54.019 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/caching/backend-api.txt 2024-04-02T14:45:54.021 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/caching/cachefiles.txt 2024-04-02T14:45:54.022 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/caching/fscache.txt 2024-04-02T14:45:54.024 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/caching/netfs-api.txt 2024-04-02T14:45:54.026 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/caching/object.txt 2024-04-02T14:45:54.028 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/caching/operations.txt 2024-04-02T14:45:54.030 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ceph.txt 2024-04-02T14:45:54.031 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/cifs/ 2024-04-02T14:45:54.033 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/cifs/AUTHORS 2024-04-02T14:45:54.034 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/cifs/CHANGES 2024-04-02T14:45:54.036 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/cifs/README 2024-04-02T14:45:54.038 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/cifs/TODO 2024-04-02T14:45:54.040 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/cifs/cifs.txt 2024-04-02T14:45:54.041 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/cifs/winucase_convert.pl 2024-04-02T14:45:54.043 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/coda.txt 2024-04-02T14:45:54.045 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/configfs/ 2024-04-02T14:45:54.047 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/configfs/Makefile 2024-04-02T14:45:54.048 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs.txt 2024-04-02T14:45:54.051 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_explicit.c 2024-04-02T14:45:54.052 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_macros.c 2024-04-02T14:45:54.054 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/cramfs.txt 2024-04-02T14:45:54.056 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/dax.txt 2024-04-02T14:45:54.058 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/debugfs.txt 2024-04-02T14:45:54.060 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/devpts.txt 2024-04-02T14:45:54.062 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/directory-locking 2024-04-02T14:45:54.063 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/dlmfs.txt 2024-04-02T14:45:54.065 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/dnotify.txt 2024-04-02T14:45:54.067 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/dnotify_test.c 2024-04-02T14:45:54.068 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ecryptfs.txt 2024-04-02T14:45:54.070 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/efivarfs.txt 2024-04-02T14:45:54.071 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/exofs.txt 2024-04-02T14:45:54.073 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ext2.txt 2024-04-02T14:45:54.075 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ext3.txt 2024-04-02T14:45:54.077 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ext4.txt 2024-04-02T14:45:54.078 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/f2fs.txt 2024-04-02T14:45:54.080 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/fiemap.txt 2024-04-02T14:45:54.082 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/files.txt 2024-04-02T14:45:54.084 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/fuse.txt 2024-04-02T14:45:54.085 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/gfs2-glocks.txt 2024-04-02T14:45:54.087 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/gfs2-uevents.txt 2024-04-02T14:45:54.088 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/gfs2.txt 2024-04-02T14:45:54.090 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/hfs.txt 2024-04-02T14:45:54.092 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/hfsplus.txt 2024-04-02T14:45:54.094 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/hpfs.txt 2024-04-02T14:45:54.096 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/inotify.txt 2024-04-02T14:45:54.097 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/isofs.txt 2024-04-02T14:45:54.099 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/jfs.txt 2024-04-02T14:45:54.101 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/locks.txt 2024-04-02T14:45:54.102 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/logfs.txt 2024-04-02T14:45:54.104 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/mandatory-locking.txt 2024-04-02T14:45:54.106 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ncpfs.txt 2024-04-02T14:45:54.108 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/ 2024-04-02T14:45:54.110 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/00-INDEX 2024-04-02T14:45:54.112 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/Exporting 2024-04-02T14:45:54.113 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/fault_injection.txt 2024-04-02T14:45:54.115 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/idmapper.txt 2024-04-02T14:45:54.116 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/knfsd-stats.txt 2024-04-02T14:45:54.118 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs-rdma.txt 2024-04-02T14:45:54.120 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs.txt 2024-04-02T14:45:54.121 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs41-server.txt 2024-04-02T14:45:54.123 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsd-admin-interfaces.txt 2024-04-02T14:45:54.125 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsroot.txt 2024-04-02T14:45:54.126 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs-block-server.txt 2024-04-02T14:45:54.128 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs.txt 2024-04-02T14:45:54.130 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-cache.txt 2024-04-02T14:45:54.131 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-server-gss.txt 2024-04-02T14:45:54.133 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/nilfs2.txt 2024-04-02T14:45:54.135 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ntfs.txt 2024-04-02T14:45:54.137 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ocfs2.txt 2024-04-02T14:45:54.139 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/omfs.txt 2024-04-02T14:45:54.140 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/overlayfs.txt 2024-04-02T14:45:54.142 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/path-lookup.txt 2024-04-02T14:45:54.144 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/ 2024-04-02T14:45:54.145 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/design_notes.txt 2024-04-02T14:45:54.147 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/info.txt 2024-04-02T14:45:54.149 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/network_protocol.txt 2024-04-02T14:45:54.150 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/porting 2024-04-02T14:45:54.152 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/proc.txt 2024-04-02T14:45:54.155 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/qnx6.txt 2024-04-02T14:45:54.157 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/quota.txt 2024-04-02T14:45:54.158 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ramfs-rootfs-initramfs.txt 2024-04-02T14:45:54.162 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/relay.txt 2024-04-02T14:45:54.162 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/romfs.txt 2024-04-02T14:45:54.164 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/seq_file.txt 2024-04-02T14:45:54.165 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/sharedsubtree.txt 2024-04-02T14:45:54.168 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/spufs.txt 2024-04-02T14:45:54.169 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/squashfs.txt 2024-04-02T14:45:54.171 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/sysfs-pci.txt 2024-04-02T14:45:54.173 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/sysfs-tagging.txt 2024-04-02T14:45:54.174 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/sysfs.txt 2024-04-02T14:45:54.176 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/sysv-fs.txt 2024-04-02T14:45:54.177 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/tmpfs.txt 2024-04-02T14:45:54.179 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ubifs.txt 2024-04-02T14:45:54.181 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/udf.txt 2024-04-02T14:45:54.182 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/ufs.txt 2024-04-02T14:45:54.184 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/vfat.txt 2024-04-02T14:45:54.186 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/vfs.txt 2024-04-02T14:45:54.188 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/xfs-delayed-logging-design.txt 2024-04-02T14:45:54.190 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/xfs-self-describing-metadata.txt 2024-04-02T14:45:54.191 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/filesystems/xfs.txt 2024-04-02T14:45:54.193 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/firmware_class/ 2024-04-02T14:45:54.195 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/firmware_class/README 2024-04-02T14:45:54.196 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/firmware_class/hotplug-script 2024-04-02T14:45:54.198 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/flexible-arrays.txt 2024-04-02T14:45:54.200 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/ 2024-04-02T14:45:54.201 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/00-INDEX 2024-04-02T14:45:54.202 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/API.txt 2024-04-02T14:45:54.204 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/FMC-and-SDB.txt 2024-04-02T14:45:54.206 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/carrier.txt 2024-04-02T14:45:54.208 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/fmc-chardev.txt 2024-04-02T14:45:54.209 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/fmc-fakedev.txt 2024-04-02T14:45:54.211 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/fmc-trivial.txt 2024-04-02T14:45:54.212 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/fmc-write-eeprom.txt 2024-04-02T14:45:54.214 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/identifiers.txt 2024-04-02T14:45:54.216 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/mezzanine.txt 2024-04-02T14:45:54.217 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/fmc/parameters.txt 2024-04-02T14:45:54.219 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/frv/ 2024-04-02T14:45:54.221 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/frv/README.txt 2024-04-02T14:45:54.222 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/frv/atomic-ops.txt 2024-04-02T14:45:54.224 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/frv/booting.txt 2024-04-02T14:45:54.226 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/frv/clock.txt 2024-04-02T14:45:54.227 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/frv/configuring.txt 2024-04-02T14:45:54.229 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/frv/features.txt 2024-04-02T14:45:54.231 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/frv/gdbinit 2024-04-02T14:45:54.233 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/frv/gdbstub.txt 2024-04-02T14:45:54.234 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/frv/kernel-ABI.txt 2024-04-02T14:45:54.236 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/frv/mmu-layout.txt 2024-04-02T14:45:54.238 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/futex-requeue-pi.txt 2024-04-02T14:45:54.240 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/gcov.txt 2024-04-02T14:45:54.241 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/gdb-kernel-debugging.txt 2024-04-02T14:45:54.243 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/gpio/ 2024-04-02T14:45:54.245 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/gpio/00-INDEX 2024-04-02T14:45:54.246 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/gpio/board.txt 2024-04-02T14:45:54.248 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/gpio/consumer.txt 2024-04-02T14:45:54.250 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/gpio/driver.txt 2024-04-02T14:45:54.252 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/gpio/gpio-legacy.txt 2024-04-02T14:45:54.253 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/gpio/gpio.txt 2024-04-02T14:45:54.255 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/gpio/sysfs.txt 2024-04-02T14:45:54.257 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hid/ 2024-04-02T14:45:54.260 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hid/hid-sensor.txt 2024-04-02T14:45:54.261 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hid/hid-transport.txt 2024-04-02T14:45:54.263 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hid/hiddev.txt 2024-04-02T14:45:54.265 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hid/hidraw.txt 2024-04-02T14:45:54.267 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hid/uhid.txt 2024-04-02T14:45:54.268 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/highuid.txt 2024-04-02T14:45:54.270 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hsi.txt 2024-04-02T14:45:54.272 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hw_random.txt 2024-04-02T14:45:54.274 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ 2024-04-02T14:45:54.275 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ab8500 2024-04-02T14:45:54.277 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/abituguru 2024-04-02T14:45:54.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/abituguru-datasheet 2024-04-02T14:45:54.281 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/abituguru3 2024-04-02T14:45:54.282 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/abx500 2024-04-02T14:45:54.283 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/acpi_power_meter 2024-04-02T14:45:54.285 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ad7314 2024-04-02T14:45:54.287 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adc128d818 2024-04-02T14:45:54.288 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adm1021 2024-04-02T14:45:54.290 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adm1025 2024-04-02T14:45:54.292 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adm1026 2024-04-02T14:45:54.294 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adm1031 2024-04-02T14:45:54.295 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adm1275 2024-04-02T14:45:54.297 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adm9240 2024-04-02T14:45:54.299 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ads1015 2024-04-02T14:45:54.301 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ads7828 2024-04-02T14:45:54.302 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adt7410 2024-04-02T14:45:54.304 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adt7411 2024-04-02T14:45:54.306 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adt7462 2024-04-02T14:45:54.307 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adt7470 2024-04-02T14:45:54.309 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/adt7475 2024-04-02T14:45:54.311 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/amc6821 2024-04-02T14:45:54.312 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/asb100 2024-04-02T14:45:54.314 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/asc7621 2024-04-02T14:45:54.316 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/coretemp 2024-04-02T14:45:54.318 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/da9052 2024-04-02T14:45:54.319 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/da9055 2024-04-02T14:45:54.321 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/dme1737 2024-04-02T14:45:54.323 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ds1621 2024-04-02T14:45:54.324 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ds620 2024-04-02T14:45:54.326 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/emc1403 2024-04-02T14:45:54.328 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/emc2103 2024-04-02T14:45:54.329 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/emc6w201 2024-04-02T14:45:54.331 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/f71805f 2024-04-02T14:45:54.333 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/f71882fg 2024-04-02T14:45:54.335 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/fam15h_power 2024-04-02T14:45:54.336 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/g760a 2024-04-02T14:45:54.338 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/g762 2024-04-02T14:45:54.341 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/gl518sm 2024-04-02T14:45:54.342 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/hih6130 2024-04-02T14:45:54.343 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/htu21 2024-04-02T14:45:54.345 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/hwmon-kernel-api.txt 2024-04-02T14:45:54.347 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ibmaem 2024-04-02T14:45:54.348 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ibmpowernv 2024-04-02T14:45:54.350 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ina209 2024-04-02T14:45:54.352 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ina2xx 2024-04-02T14:45:54.354 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/it87 2024-04-02T14:45:54.356 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/jc42 2024-04-02T14:45:54.358 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/k10temp 2024-04-02T14:45:54.359 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/k8temp 2024-04-02T14:45:54.361 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lineage-pem 2024-04-02T14:45:54.363 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm25066 2024-04-02T14:45:54.365 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm63 2024-04-02T14:45:54.367 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm70 2024-04-02T14:45:54.368 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm73 2024-04-02T14:45:54.370 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm75 2024-04-02T14:45:54.372 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm77 2024-04-02T14:45:54.373 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm78 2024-04-02T14:45:54.375 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm80 2024-04-02T14:45:54.377 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm83 2024-04-02T14:45:54.379 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm85 2024-04-02T14:45:54.381 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm87 2024-04-02T14:45:54.382 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm90 2024-04-02T14:45:54.384 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm92 2024-04-02T14:45:54.386 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm93 2024-04-02T14:45:54.388 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm95234 2024-04-02T14:45:54.389 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/lm95245 2024-04-02T14:45:54.391 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ltc2945 2024-04-02T14:45:54.393 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ltc2978 2024-04-02T14:45:54.394 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ltc4151 2024-04-02T14:45:54.396 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ltc4215 2024-04-02T14:45:54.398 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ltc4245 2024-04-02T14:45:54.399 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ltc4260 2024-04-02T14:45:54.401 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ltc4261 2024-04-02T14:45:54.403 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/max16064 2024-04-02T14:45:54.404 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/max16065 2024-04-02T14:45:54.406 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/max1619 2024-04-02T14:45:54.408 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/max1668 2024-04-02T14:45:54.409 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/max197 2024-04-02T14:45:54.411 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/max34440 2024-04-02T14:45:54.413 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/max6639 2024-04-02T14:45:54.415 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/max6642 2024-04-02T14:45:54.416 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/max6650 2024-04-02T14:45:54.418 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/max6697 2024-04-02T14:45:54.420 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/max8688 2024-04-02T14:45:54.421 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/mc13783-adc 2024-04-02T14:45:54.423 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/mcp3021 2024-04-02T14:45:54.425 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/menf21bmc 2024-04-02T14:45:54.427 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/nct6683 2024-04-02T14:45:54.428 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/nct6775 2024-04-02T14:45:54.430 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/nct7802 2024-04-02T14:45:54.432 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ntc_thermistor 2024-04-02T14:45:54.433 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/pc87360 2024-04-02T14:45:54.435 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/pc87427 2024-04-02T14:45:54.437 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/pcf8591 2024-04-02T14:45:54.438 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/pmbus 2024-04-02T14:45:54.440 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/pmbus-core 2024-04-02T14:45:54.442 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/powr1220 2024-04-02T14:45:54.443 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/pwm-fan 2024-04-02T14:45:54.445 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/sch5627 2024-04-02T14:45:54.447 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/sch5636 2024-04-02T14:45:54.449 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/sht15 2024-04-02T14:45:54.451 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/sht21 2024-04-02T14:45:54.452 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/shtc1 2024-04-02T14:45:54.454 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/sis5595 2024-04-02T14:45:54.455 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/smm665 2024-04-02T14:45:54.457 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/smsc47b397 2024-04-02T14:45:54.459 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/smsc47m1 2024-04-02T14:45:54.461 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/smsc47m192 2024-04-02T14:45:54.462 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/submitting-patches 2024-04-02T14:45:54.464 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/sysfs-interface 2024-04-02T14:45:54.466 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/thmc50 2024-04-02T14:45:54.467 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/tmp102 2024-04-02T14:45:54.469 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/tmp103 2024-04-02T14:45:54.471 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/tmp401 2024-04-02T14:45:54.473 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/tmp421 2024-04-02T14:45:54.475 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/tps40422 2024-04-02T14:45:54.476 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/twl4030-madc-hwmon 2024-04-02T14:45:54.478 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ucd9000 2024-04-02T14:45:54.479 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/ucd9200 2024-04-02T14:45:54.481 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/userspace-tools 2024-04-02T14:45:54.483 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/vexpress 2024-04-02T14:45:54.485 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/via686a 2024-04-02T14:45:54.486 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/vt1211 2024-04-02T14:45:54.489 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/w83627ehf 2024-04-02T14:45:54.490 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/w83627hf 2024-04-02T14:45:54.491 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/w83781d 2024-04-02T14:45:54.493 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/w83791d 2024-04-02T14:45:54.495 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/w83792d 2024-04-02T14:45:54.497 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/w83793 2024-04-02T14:45:54.498 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/w83795 2024-04-02T14:45:54.500 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/w83l785ts 2024-04-02T14:45:54.502 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/w83l786ng 2024-04-02T14:45:54.503 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/wm831x 2024-04-02T14:45:54.505 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/wm8350 2024-04-02T14:45:54.507 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwmon/zl6100 2024-04-02T14:45:54.508 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/hwspinlock.txt 2024-04-02T14:45:54.510 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/ 2024-04-02T14:45:54.512 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/ 2024-04-02T14:45:54.513 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1535 2024-04-02T14:45:54.515 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1563 2024-04-02T14:45:54.517 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali15x3 2024-04-02T14:45:54.518 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd756 2024-04-02T14:45:54.520 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd8111 2024-04-02T14:45:54.521 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-diolan-u2c 2024-04-02T14:45:54.523 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-i801 2024-04-02T14:45:54.525 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ismt 2024-04-02T14:45:54.526 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-nforce2 2024-04-02T14:45:54.528 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ocores 2024-04-02T14:45:54.530 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport 2024-04-02T14:45:54.532 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport-light 2024-04-02T14:45:54.533 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-pca-isa 2024-04-02T14:45:54.535 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-piix4 2024-04-02T14:45:54.537 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis5595 2024-04-02T14:45:54.538 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis630 2024-04-02T14:45:54.540 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis96x 2024-04-02T14:45:54.542 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-taos-evm 2024-04-02T14:45:54.545 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-via 2024-04-02T14:45:54.547 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-viapro 2024-04-02T14:45:54.549 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/busses/scx200_acb 2024-04-02T14:45:54.551 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/dev-interface 2024-04-02T14:45:54.553 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/fault-codes 2024-04-02T14:45:54.555 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/functionality 2024-04-02T14:45:54.557 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/i2c-protocol 2024-04-02T14:45:54.560 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/i2c-stub 2024-04-02T14:45:54.562 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/instantiating-devices 2024-04-02T14:45:54.563 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/muxes/ 2024-04-02T14:45:54.565 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/muxes/i2c-mux-gpio 2024-04-02T14:45:54.566 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/old-module-parameters 2024-04-02T14:45:54.569 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/smbus-protocol 2024-04-02T14:45:54.570 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/summary 2024-04-02T14:45:54.572 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/ten-bit-addresses 2024-04-02T14:45:54.573 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/upgrading-clients 2024-04-02T14:45:54.576 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2c/writing-clients 2024-04-02T14:45:54.576 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2o/ 2024-04-02T14:45:54.578 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2o/README 2024-04-02T14:45:54.580 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/i2o/ioctl 2024-04-02T14:45:54.581 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/ 2024-04-02T14:45:54.583 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/.gitignore 2024-04-02T14:45:54.584 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/IRQ-redir.txt 2024-04-02T14:45:54.586 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/Makefile 2024-04-02T14:45:54.588 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/README 2024-04-02T14:45:54.589 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/aliasing-test.c 2024-04-02T14:45:54.800 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/aliasing.txt 2024-04-02T14:45:54.801 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/efirtc.txt 2024-04-02T14:45:54.803 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/err_inject.txt 2024-04-02T14:45:54.805 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/fsys.txt 2024-04-02T14:45:54.807 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/mca.txt 2024-04-02T14:45:54.809 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/serial.txt 2024-04-02T14:45:54.811 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ia64/xen.txt 2024-04-02T14:45:54.812 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ide/ 2024-04-02T14:45:54.814 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ide/00-INDEX 2024-04-02T14:45:54.816 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-cd.1994-2004 2024-04-02T14:45:54.818 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-floppy.1996-2002 2024-04-02T14:45:54.819 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-tape.1995-2002 2024-04-02T14:45:54.821 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ide/ide-tape.txt 2024-04-02T14:45:54.823 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ide/ide.txt 2024-04-02T14:45:54.825 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ide/warm-plug-howto.txt 2024-04-02T14:45:54.827 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/infiniband/ 2024-04-02T14:45:54.828 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/infiniband/core_locking.txt 2024-04-02T14:45:54.830 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/infiniband/ipoib.txt 2024-04-02T14:45:54.831 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/infiniband/sysfs.txt 2024-04-02T14:45:54.833 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/infiniband/user_mad.txt 2024-04-02T14:45:54.835 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/infiniband/user_verbs.txt 2024-04-02T14:45:54.836 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/init.txt 2024-04-02T14:45:54.838 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/initrd.txt 2024-04-02T14:45:54.840 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/ 2024-04-02T14:45:54.841 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/alps.txt 2024-04-02T14:45:54.843 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/amijoy.txt 2024-04-02T14:45:54.845 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/appletouch.txt 2024-04-02T14:45:54.846 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/atarikbd.txt 2024-04-02T14:45:54.848 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/bcm5974.txt 2024-04-02T14:45:54.849 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/cd32.txt 2024-04-02T14:45:54.851 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/cma3000_d0x.txt 2024-04-02T14:45:54.853 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/cs461x.txt 2024-04-02T14:45:54.854 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/edt-ft5x06.txt 2024-04-02T14:45:54.856 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/elantech.txt 2024-04-02T14:45:54.858 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/event-codes.txt 2024-04-02T14:45:54.859 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/ff.txt 2024-04-02T14:45:54.861 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/gamepad.txt 2024-04-02T14:45:54.863 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/gameport-programming.txt 2024-04-02T14:45:54.864 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/gpio-tilt.txt 2024-04-02T14:45:54.866 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/iforce-protocol.txt 2024-04-02T14:45:54.867 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/input-programming.txt 2024-04-02T14:45:54.869 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/input.txt 2024-04-02T14:45:54.871 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/interactive.fig 2024-04-02T14:45:54.874 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/joystick-api.txt 2024-04-02T14:45:54.874 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/joystick-parport.txt 2024-04-02T14:45:54.876 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/joystick.txt 2024-04-02T14:45:54.878 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/multi-touch-protocol.txt 2024-04-02T14:45:54.880 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/notifier.txt 2024-04-02T14:45:54.881 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/ntrig.txt 2024-04-02T14:45:54.883 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/rotary-encoder.txt 2024-04-02T14:45:54.885 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/sentelic.txt 2024-04-02T14:45:54.887 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/shape.fig 2024-04-02T14:45:54.888 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/walkera0701.txt 2024-04-02T14:45:54.890 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/xpad.txt 2024-04-02T14:45:54.892 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/input/yealink.txt 2024-04-02T14:45:54.893 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/intel_txt.txt 2024-04-02T14:45:54.895 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/io-mapping.txt 2024-04-02T14:45:54.897 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/io_ordering.txt 2024-04-02T14:45:54.899 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ioctl/ 2024-04-02T14:45:54.900 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ioctl/00-INDEX 2024-04-02T14:45:54.902 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ioctl/botching-up-ioctls.txt 2024-04-02T14:45:54.904 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ioctl/cdrom.txt 2024-04-02T14:45:54.905 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ioctl/hdio.txt 2024-04-02T14:45:54.907 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ioctl/ioctl-decoding.txt 2024-04-02T14:45:54.909 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ioctl/ioctl-number.txt 2024-04-02T14:45:54.911 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/iostats.txt 2024-04-02T14:45:54.913 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/irqflags-tracing.txt 2024-04-02T14:45:54.915 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isapnp.txt 2024-04-02T14:45:54.917 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/ 2024-04-02T14:45:54.918 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/00-INDEX 2024-04-02T14:45:54.920 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/CREDITS 2024-04-02T14:45:54.922 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/HiSax.cert 2024-04-02T14:45:54.923 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/INTERFACE 2024-04-02T14:45:54.925 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.CAPI 2024-04-02T14:45:54.927 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.fax 2024-04-02T14:45:54.929 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README 2024-04-02T14:45:54.931 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.FAQ 2024-04-02T14:45:54.932 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.HiSax 2024-04-02T14:45:54.934 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.act2000 2024-04-02T14:45:54.936 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.audio 2024-04-02T14:45:54.938 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.avmb1 2024-04-02T14:45:54.940 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.concap 2024-04-02T14:45:54.941 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.diversion 2024-04-02T14:45:54.943 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.fax 2024-04-02T14:45:54.944 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.gigaset 2024-04-02T14:45:54.946 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.hfc-pci 2024-04-02T14:45:54.948 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.hysdn 2024-04-02T14:45:54.950 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.icn 2024-04-02T14:45:54.952 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.mISDN 2024-04-02T14:45:54.954 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.pcbit 2024-04-02T14:45:54.956 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.sc 2024-04-02T14:45:54.958 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.syncppp 2024-04-02T14:45:54.959 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/README.x25 2024-04-02T14:45:54.961 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/isdn/syncPPP.FAQ 2024-04-02T14:45:54.963 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ja_JP/ 2024-04-02T14:45:54.965 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ja_JP/HOWTO 2024-04-02T14:45:54.966 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ja_JP/SubmitChecklist 2024-04-02T14:45:54.968 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ja_JP/SubmittingPatches 2024-04-02T14:45:54.970 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ja_JP/stable_api_nonsense.txt 2024-04-02T14:45:54.972 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ja_JP/stable_kernel_rules.txt 2024-04-02T14:45:54.975 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/java.txt 2024-04-02T14:45:54.976 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kasan.txt 2024-04-02T14:45:54.977 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kbuild/ 2024-04-02T14:45:54.982 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kbuild/00-INDEX 2024-04-02T14:45:54.982 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kbuild/headers_install.txt 2024-04-02T14:45:54.982 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kbuild/kbuild.txt 2024-04-02T14:45:54.984 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kbuild/kconfig-language.txt 2024-04-02T14:45:54.986 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kbuild/kconfig.txt 2024-04-02T14:45:54.987 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kbuild/makefiles.txt 2024-04-02T14:45:54.989 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kbuild/modules.txt 2024-04-02T14:45:54.992 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kdump/ 2024-04-02T14:45:54.995 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kdump/gdbmacros.txt 2024-04-02T14:45:54.995 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kdump/kdump.txt 2024-04-02T14:45:54.996 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kernel-doc-nano-HOWTO.txt 2024-04-02T14:45:54.998 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kernel-docs.txt 2024-04-02T14:45:55.000 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kernel-parameters.txt 2024-04-02T14:45:55.006 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kernel-per-CPU-kthreads.txt 2024-04-02T14:45:55.007 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kmemcheck.txt 2024-04-02T14:45:55.009 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kmemleak.txt 2024-04-02T14:45:55.012 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ko_KR/ 2024-04-02T14:45:55.013 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ko_KR/HOWTO 2024-04-02T14:45:55.014 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ko_KR/stable_api_nonsense.txt 2024-04-02T14:45:55.016 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kobject.txt 2024-04-02T14:45:55.018 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kprobes.txt 2024-04-02T14:45:55.021 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kref.txt 2024-04-02T14:45:55.022 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/kselftest.txt 2024-04-02T14:45:55.024 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/ 2024-04-02T14:45:55.025 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/.gitignore 2024-04-02T14:45:55.027 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/00-INDEX 2024-04-02T14:45:55.029 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/Makefile 2024-04-02T14:45:55.030 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/asus-laptop.txt 2024-04-02T14:45:55.032 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/disk-shock-protection.txt 2024-04-02T14:45:55.033 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/dslm.c 2024-04-02T14:45:55.035 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/freefall.c 2024-04-02T14:45:55.037 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/laptop-mode.txt 2024-04-02T14:45:55.038 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/sony-laptop.txt 2024-04-02T14:45:55.040 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/sonypi.txt 2024-04-02T14:45:55.041 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/thinkpad-acpi.txt 2024-04-02T14:45:55.044 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/laptops/toshiba_haps.txt 2024-04-02T14:45:55.045 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ldm.txt 2024-04-02T14:45:55.047 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/ 2024-04-02T14:45:55.049 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/00-INDEX 2024-04-02T14:45:55.050 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/leds-blinkm.txt 2024-04-02T14:45:55.052 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/leds-class.txt 2024-04-02T14:45:55.054 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/leds-lm3556.txt 2024-04-02T14:45:55.056 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/leds-lp3944.txt 2024-04-02T14:45:55.058 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/leds-lp5521.txt 2024-04-02T14:45:55.060 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/leds-lp5523.txt 2024-04-02T14:45:55.062 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/leds-lp5562.txt 2024-04-02T14:45:55.066 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/leds-lp55xx.txt 2024-04-02T14:45:55.067 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/ledtrig-oneshot.txt 2024-04-02T14:45:55.069 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/leds/ledtrig-transient.txt 2024-04-02T14:45:55.071 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/local_ops.txt 2024-04-02T14:45:55.073 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/locking/ 2024-04-02T14:45:55.074 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/locking/00-INDEX 2024-04-02T14:45:55.076 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/locking/lglock.txt 2024-04-02T14:45:55.078 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/locking/lockdep-design.txt 2024-04-02T14:45:55.079 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/locking/lockstat.txt 2024-04-02T14:45:55.081 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/locking/locktorture.txt 2024-04-02T14:45:55.083 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/locking/mutex-design.txt 2024-04-02T14:45:55.084 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/locking/rt-mutex-design.txt 2024-04-02T14:45:55.086 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/locking/rt-mutex.txt 2024-04-02T14:45:55.088 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/locking/spinlocks.txt 2024-04-02T14:45:55.090 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/locking/ww-mutex-design.txt 2024-04-02T14:45:55.091 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/lockup-watchdogs.txt 2024-04-02T14:45:55.093 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/logo.gif 2024-04-02T14:45:55.095 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/logo.txt 2024-04-02T14:45:55.097 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/lzo.txt 2024-04-02T14:45:55.098 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/m68k/ 2024-04-02T14:45:55.100 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/m68k/00-INDEX 2024-04-02T14:45:55.102 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/m68k/README.buddha 2024-04-02T14:45:55.103 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/m68k/kernel-options.txt 2024-04-02T14:45:55.105 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/magic-number.txt 2024-04-02T14:45:55.107 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mailbox.txt 2024-04-02T14:45:55.109 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/md.txt 2024-04-02T14:45:55.111 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/media-framework.txt 2024-04-02T14:45:55.114 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/memory-barriers.txt 2024-04-02T14:45:55.218 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/memory-devices/ 2024-04-02T14:45:55.220 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/memory-devices/ti-emif.txt 2024-04-02T14:45:55.222 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/memory-hotplug.txt 2024-04-02T14:45:55.225 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/metag/ 2024-04-02T14:45:55.226 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/metag/00-INDEX 2024-04-02T14:45:55.229 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/metag/kernel-ABI.txt 2024-04-02T14:45:55.231 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mic/ 2024-04-02T14:45:55.233 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mic/Makefile 2024-04-02T14:45:55.235 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mic/mic_overview.txt 2024-04-02T14:45:55.237 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mic/mpssd/ 2024-04-02T14:45:55.239 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mic/mpssd/.gitignore 2024-04-02T14:45:55.241 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mic/mpssd/Makefile 2024-04-02T14:45:55.243 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mic/mpssd/micctrl 2024-04-02T14:45:55.245 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mic/mpssd/mpss 2024-04-02T14:45:55.248 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.c 2024-04-02T14:45:55.250 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.h 2024-04-02T14:45:55.252 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mic/mpssd/sysfs.c 2024-04-02T14:45:55.255 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mips/ 2024-04-02T14:45:55.257 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mips/00-INDEX 2024-04-02T14:45:55.260 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mips/AU1xxx_IDE.README 2024-04-02T14:45:55.261 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/ 2024-04-02T14:45:55.263 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/Makefile 2024-04-02T14:45:55.265 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/ad525x_dpot.txt 2024-04-02T14:45:55.266 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/apds990x.txt 2024-04-02T14:45:55.268 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/bh1770glc.txt 2024-04-02T14:45:55.270 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/c2port.txt 2024-04-02T14:45:55.271 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/eeprom 2024-04-02T14:45:55.273 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/ics932s401 2024-04-02T14:45:55.275 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/isl29003 2024-04-02T14:45:55.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/lis3lv02d 2024-04-02T14:45:55.278 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/max6875 2024-04-02T14:45:55.281 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/mei/ 2024-04-02T14:45:55.282 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/mei/.gitignore 2024-04-02T14:45:55.283 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/mei/Makefile 2024-04-02T14:45:55.285 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/mei/TODO 2024-04-02T14:45:55.286 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-amt-version.c 2024-04-02T14:45:55.288 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-client-bus.txt 2024-04-02T14:45:55.289 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei.txt 2024-04-02T14:45:55.291 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/misc-devices/spear-pcie-gadget.txt 2024-04-02T14:45:55.293 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mmc/ 2024-04-02T14:45:55.295 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mmc/00-INDEX 2024-04-02T14:45:55.296 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mmc/mmc-async-req.txt 2024-04-02T14:45:55.298 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-attrs.txt 2024-04-02T14:45:55.299 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-parts.txt 2024-04-02T14:45:55.300 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mn10300/ 2024-04-02T14:45:55.302 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mn10300/ABI.txt 2024-04-02T14:45:55.304 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mn10300/compartmentalisation.txt 2024-04-02T14:45:55.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/module-signing.txt 2024-04-02T14:45:55.307 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mono.txt 2024-04-02T14:45:55.309 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mtd/ 2024-04-02T14:45:55.310 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mtd/nand/ 2024-04-02T14:45:55.311 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt 2024-04-02T14:45:55.313 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mtd/nand_ecc.txt 2024-04-02T14:45:55.315 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/mtd/spi-nor.txt 2024-04-02T14:45:55.317 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/namespaces/ 2024-04-02T14:45:55.318 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/namespaces/compatibility-list.txt 2024-04-02T14:45:55.320 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/namespaces/resource-control.txt 2024-04-02T14:45:55.321 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/netlabel/ 2024-04-02T14:45:55.323 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/netlabel/00-INDEX 2024-04-02T14:45:55.324 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/netlabel/cipso_ipv4.txt 2024-04-02T14:45:55.326 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/netlabel/draft-ietf-cipso-ipsecurity-01.txt 2024-04-02T14:45:55.328 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/netlabel/introduction.txt 2024-04-02T14:45:55.329 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/netlabel/lsm_interface.txt 2024-04-02T14:45:55.331 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ 2024-04-02T14:45:55.332 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/00-INDEX 2024-04-02T14:45:55.334 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/3c509.txt 2024-04-02T14:45:55.336 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/6pack.txt 2024-04-02T14:45:55.337 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/LICENSE.qla3xxx 2024-04-02T14:45:55.339 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlcnic 2024-04-02T14:45:55.341 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlge 2024-04-02T14:45:55.343 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/Makefile 2024-04-02T14:45:55.344 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/PLIP.txt 2024-04-02T14:45:55.346 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/README.ipw2100 2024-04-02T14:45:55.347 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/README.ipw2200 2024-04-02T14:45:55.349 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/README.sb1000 2024-04-02T14:45:55.351 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/alias.txt 2024-04-02T14:45:55.353 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/altera_tse.txt 2024-04-02T14:45:55.354 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/arcnet-hardware.txt 2024-04-02T14:45:55.358 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/arcnet.txt 2024-04-02T14:45:55.367 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/atm.txt 2024-04-02T14:45:55.376 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ax25.txt 2024-04-02T14:45:55.380 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/batman-adv.txt 2024-04-02T14:45:55.384 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/baycom.txt 2024-04-02T14:45:55.386 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/bonding.txt 2024-04-02T14:45:55.391 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/bridge.txt 2024-04-02T14:45:55.393 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/caif/ 2024-04-02T14:45:55.394 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/caif/Linux-CAIF.txt 2024-04-02T14:45:55.397 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/caif/README 2024-04-02T14:45:55.399 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/caif/spi_porting.txt 2024-04-02T14:45:55.401 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/can.txt 2024-04-02T14:45:55.403 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/cdc_mbim.txt 2024-04-02T14:45:55.405 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/cops.txt 2024-04-02T14:45:55.407 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/cs89x0.txt 2024-04-02T14:45:55.409 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/cxacru-cf.py 2024-04-02T14:45:55.412 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/cxacru.txt 2024-04-02T14:45:55.413 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/cxgb.txt 2024-04-02T14:45:55.416 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/dccp.txt 2024-04-02T14:45:55.418 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/dctcp.txt 2024-04-02T14:45:55.420 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/de4x5.txt 2024-04-02T14:45:55.422 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/decnet.txt 2024-04-02T14:45:55.424 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/dl2k.txt 2024-04-02T14:45:55.426 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/dm9000.txt 2024-04-02T14:45:55.428 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/dmfe.txt 2024-04-02T14:45:55.431 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/dns_resolver.txt 2024-04-02T14:45:55.433 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/driver.txt 2024-04-02T14:45:55.436 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/e100.txt 2024-04-02T14:45:55.438 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/e1000.txt 2024-04-02T14:45:55.441 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/e1000e.txt 2024-04-02T14:45:55.443 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/eql.txt 2024-04-02T14:45:55.446 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/fib_trie.txt 2024-04-02T14:45:55.448 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/filter.txt 2024-04-02T14:45:55.451 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/fore200e.txt 2024-04-02T14:45:55.453 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/framerelay.txt 2024-04-02T14:45:55.455 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/gen_stats.txt 2024-04-02T14:45:55.456 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/generic-hdlc.txt 2024-04-02T14:45:55.459 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/generic_netlink.txt 2024-04-02T14:45:55.460 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/gianfar.txt 2024-04-02T14:45:55.462 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/i40e.txt 2024-04-02T14:45:55.464 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/i40evf.txt 2024-04-02T14:45:55.466 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ieee802154.txt 2024-04-02T14:45:55.467 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/igb.txt 2024-04-02T14:45:55.469 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/igbvf.txt 2024-04-02T14:45:55.471 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ip-sysctl.txt 2024-04-02T14:45:55.474 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ip_dynaddr.txt 2024-04-02T14:45:55.475 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ipddp.txt 2024-04-02T14:45:55.477 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/iphase.txt 2024-04-02T14:45:55.479 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ipsec.txt 2024-04-02T14:45:55.481 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ipv6.txt 2024-04-02T14:45:55.483 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ipvlan.txt 2024-04-02T14:45:55.484 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ipvs-sysctl.txt 2024-04-02T14:45:55.486 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/irda.txt 2024-04-02T14:45:55.488 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ixgb.txt 2024-04-02T14:45:55.490 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ixgbe.txt 2024-04-02T14:45:55.492 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ixgbevf.txt 2024-04-02T14:45:55.494 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/l2tp.txt 2024-04-02T14:45:55.495 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/lapb-module.txt 2024-04-02T14:45:55.497 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ltpc.txt 2024-04-02T14:45:55.499 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/mac80211-auth-assoc-deauth.txt 2024-04-02T14:45:55.500 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/mac80211-injection.txt 2024-04-02T14:45:55.502 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/ 2024-04-02T14:45:55.503 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/README 2024-04-02T14:45:55.505 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/hostapd.conf 2024-04-02T14:45:55.507 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/wpa_supplicant.conf 2024-04-02T14:45:55.508 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/multiqueue.txt 2024-04-02T14:45:55.510 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/netconsole.txt 2024-04-02T14:45:55.512 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/netdev-FAQ.txt 2024-04-02T14:45:55.514 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/netdev-features.txt 2024-04-02T14:45:55.516 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/netdevices.txt 2024-04-02T14:45:55.518 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/netif-msg.txt 2024-04-02T14:45:55.519 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/netlink_mmap.txt 2024-04-02T14:45:55.521 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/nf_conntrack-sysctl.txt 2024-04-02T14:45:55.523 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/nfc.txt 2024-04-02T14:45:55.525 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/openvswitch.txt 2024-04-02T14:45:55.526 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/operstates.txt 2024-04-02T14:45:55.528 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/packet_mmap.txt 2024-04-02T14:45:55.530 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/phonet.txt 2024-04-02T14:45:55.532 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/phy.txt 2024-04-02T14:45:55.534 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/pktgen.txt 2024-04-02T14:45:55.535 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/policy-routing.txt 2024-04-02T14:45:55.537 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ppp_generic.txt 2024-04-02T14:45:55.539 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/proc_net_tcp.txt 2024-04-02T14:45:55.541 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/radiotap-headers.txt 2024-04-02T14:45:55.542 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/ray_cs.txt 2024-04-02T14:45:55.544 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/rds.txt 2024-04-02T14:45:55.603 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/regulatory.txt 2024-04-02T14:45:55.605 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/rxrpc.txt 2024-04-02T14:45:55.607 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/s2io.txt 2024-04-02T14:45:55.610 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/scaling.txt 2024-04-02T14:45:55.625 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/sctp.txt 2024-04-02T14:45:55.649 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/secid.txt 2024-04-02T14:45:55.660 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/skfp.txt 2024-04-02T14:45:55.662 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/smc9.txt 2024-04-02T14:45:55.665 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/spider_net.txt 2024-04-02T14:45:55.668 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/stmmac.txt 2024-04-02T14:45:55.670 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/switchdev.txt 2024-04-02T14:45:55.674 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/tc-actions-env-rules.txt 2024-04-02T14:45:55.676 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/tcp-thin.txt 2024-04-02T14:45:55.680 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/tcp.txt 2024-04-02T14:45:55.684 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/team.txt 2024-04-02T14:45:55.686 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/timestamping.txt 2024-04-02T14:45:55.691 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/timestamping/ 2024-04-02T14:45:55.694 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/timestamping/.gitignore 2024-04-02T14:45:55.696 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/timestamping/Makefile 2024-04-02T14:45:55.699 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c 2024-04-02T14:45:55.718 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/timestamping/timestamping.c 2024-04-02T14:45:55.720 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c 2024-04-02T14:45:55.722 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/tlan.txt 2024-04-02T14:45:55.724 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/tproxy.txt 2024-04-02T14:45:55.726 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/tuntap.txt 2024-04-02T14:45:55.728 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/udplite.txt 2024-04-02T14:45:55.730 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/vortex.txt 2024-04-02T14:45:55.731 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/vxge.txt 2024-04-02T14:45:55.733 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/vxlan.txt 2024-04-02T14:45:55.735 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/x25-iface.txt 2024-04-02T14:45:55.737 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/x25.txt 2024-04-02T14:45:55.739 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/xfrm_proc.txt 2024-04-02T14:45:55.741 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/xfrm_sync.txt 2024-04-02T14:45:55.743 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/xfrm_sysctl.txt 2024-04-02T14:45:55.745 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/networking/z8530drv.txt 2024-04-02T14:45:55.747 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/nfc/ 2024-04-02T14:45:55.748 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/nfc/nfc-hci.txt 2024-04-02T14:45:55.750 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/nfc/nfc-pn544.txt 2024-04-02T14:45:55.752 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/nios2/ 2024-04-02T14:45:55.753 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/nios2/README 2024-04-02T14:45:55.755 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/nommu-mmap.txt 2024-04-02T14:45:55.757 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/numastat.txt 2024-04-02T14:45:55.759 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/oops-tracing.txt 2024-04-02T14:45:55.761 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/padata.txt 2024-04-02T14:45:55.763 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/parisc/ 2024-04-02T14:45:55.764 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/parisc/00-INDEX 2024-04-02T14:45:55.766 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/parisc/debugging 2024-04-02T14:45:55.767 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/parisc/registers 2024-04-02T14:45:55.769 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/parport-lowlevel.txt 2024-04-02T14:45:55.772 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/parport.txt 2024-04-02T14:45:55.773 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pcmcia/ 2024-04-02T14:45:55.775 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pcmcia/.gitignore 2024-04-02T14:45:55.776 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pcmcia/Makefile 2024-04-02T14:45:55.778 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pcmcia/crc32hash.c 2024-04-02T14:45:55.780 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pcmcia/devicetable.txt 2024-04-02T14:45:55.781 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pcmcia/driver-changes.txt 2024-04-02T14:45:55.783 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pcmcia/driver.txt 2024-04-02T14:45:55.785 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pcmcia/locking.txt 2024-04-02T14:45:55.787 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/percpu-rw-semaphore.txt 2024-04-02T14:45:55.788 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/phy.txt 2024-04-02T14:45:55.790 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/phy/ 2024-04-02T14:45:55.792 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/phy/samsung-usb2.txt 2024-04-02T14:45:55.794 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pi-futex.txt 2024-04-02T14:45:55.797 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pinctrl.txt 2024-04-02T14:45:55.797 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/platform/ 2024-04-02T14:45:55.799 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/platform/x86-laptop-drivers.txt 2024-04-02T14:45:55.802 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pnp.txt 2024-04-02T14:45:55.803 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/ 2024-04-02T14:45:55.804 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/00-INDEX 2024-04-02T14:45:55.806 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/apm-acpi.txt 2024-04-02T14:45:55.808 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/basic-pm-debugging.txt 2024-04-02T14:45:55.810 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/charger-manager.txt 2024-04-02T14:45:55.811 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/devices.txt 2024-04-02T14:45:55.814 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/drivers-testing.txt 2024-04-02T14:45:55.816 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/freezing-of-tasks.txt 2024-04-02T14:45:55.818 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/interface.txt 2024-04-02T14:45:55.820 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/notifiers.txt 2024-04-02T14:45:55.821 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/opp.txt 2024-04-02T14:45:55.823 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/pci.txt 2024-04-02T14:45:55.826 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/pm_qos_interface.txt 2024-04-02T14:45:55.828 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/power_supply_class.txt 2024-04-02T14:45:55.830 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/powercap/ 2024-04-02T14:45:55.831 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/powercap/powercap.txt 2024-04-02T14:45:55.833 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/regulator/ 2024-04-02T14:45:55.835 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/regulator/consumer.txt 2024-04-02T14:45:55.837 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/regulator/design.txt 2024-04-02T14:45:55.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:55 smithi062 ceph-mon[32116]: pgmap v60: 97 pgs: 97 active+clean; 56 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.6 MiB/s wr, 27 op/s 2024-04-02T14:45:55.839 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/regulator/machine.txt 2024-04-02T14:45:55.840 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/regulator/overview.txt 2024-04-02T14:45:55.842 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/regulator/regulator.txt 2024-04-02T14:45:55.844 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/runtime_pm.txt 2024-04-02T14:45:55.846 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/s2ram.txt 2024-04-02T14:45:55.848 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/states.txt 2024-04-02T14:45:55.850 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/suspend-and-cpuhotplug.txt 2024-04-02T14:45:55.851 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/suspend-and-interrupts.txt 2024-04-02T14:45:55.853 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/swsusp-and-swap-files.txt 2024-04-02T14:45:55.855 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/swsusp-dmcrypt.txt 2024-04-02T14:45:55.857 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/swsusp.txt 2024-04-02T14:45:55.859 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/tricks.txt 2024-04-02T14:45:55.861 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/userland-swsusp.txt 2024-04-02T14:45:55.862 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/power/video.txt 2024-04-02T14:45:55.864 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/ 2024-04-02T14:45:55.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:55 smithi113 ceph-mon[36135]: pgmap v60: 97 pgs: 97 active+clean; 56 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.6 MiB/s wr, 27 op/s 2024-04-02T14:45:55.866 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/00-INDEX 2024-04-02T14:45:55.867 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/bootwrapper.txt 2024-04-02T14:45:55.869 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/cpu_families.txt 2024-04-02T14:45:55.871 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/cpu_features.txt 2024-04-02T14:45:55.873 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/cxl.txt 2024-04-02T14:45:55.875 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/eeh-pci-error-recovery.txt 2024-04-02T14:45:55.876 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/firmware-assisted-dump.txt 2024-04-02T14:45:55.879 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/hvcs.txt 2024-04-02T14:45:55.881 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/mpc52xx.txt 2024-04-02T14:45:55.883 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/pmu-ebb.txt 2024-04-02T14:45:55.885 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/ptrace.txt 2024-04-02T14:45:55.887 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/qe_firmware.txt 2024-04-02T14:45:55.889 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/powerpc/transactional_memory.txt 2024-04-02T14:45:55.890 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pps/ 2024-04-02T14:45:55.895 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pps/pps.txt 2024-04-02T14:45:55.895 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/prctl/ 2024-04-02T14:45:55.896 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/prctl/.gitignore 2024-04-02T14:45:55.898 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/prctl/Makefile 2024-04-02T14:45:55.899 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-ctxt-sw-stress-test.c 2024-04-02T14:45:55.901 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-on-off-stress-test.c 2024-04-02T14:45:55.903 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-test.c 2024-04-02T14:45:55.904 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/prctl/no_new_privs.txt 2024-04-02T14:45:55.906 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/prctl/seccomp_filter.txt 2024-04-02T14:45:55.908 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/preempt-locking.txt 2024-04-02T14:45:55.910 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/printk-formats.txt 2024-04-02T14:45:55.912 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pti/ 2024-04-02T14:45:55.913 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pti/pti_intel_mid.txt 2024-04-02T14:45:55.915 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ptp/ 2024-04-02T14:45:55.917 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ptp/.gitignore 2024-04-02T14:45:55.918 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ptp/Makefile 2024-04-02T14:45:55.920 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ptp/ptp.txt 2024-04-02T14:45:55.922 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ptp/testptp.c 2024-04-02T14:45:55.924 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ptp/testptp.mk 2024-04-02T14:45:55.925 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/pwm.txt 2024-04-02T14:45:55.927 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/ramoops.txt 2024-04-02T14:45:55.929 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/rapidio/ 2024-04-02T14:45:55.930 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/rapidio/rapidio.txt 2024-04-02T14:45:55.932 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/rapidio/sysfs.txt 2024-04-02T14:45:55.935 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/rapidio/tsi721.txt 2024-04-02T14:45:55.937 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/rbtree.txt 2024-04-02T14:45:55.941 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/remoteproc.txt 2024-04-02T14:45:55.942 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/rfkill.txt 2024-04-02T14:45:55.944 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/robust-futex-ABI.txt 2024-04-02T14:45:55.946 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/robust-futexes.txt 2024-04-02T14:45:55.949 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/rpmsg.txt 2024-04-02T14:45:55.951 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/rtc.txt 2024-04-02T14:45:55.954 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/ 2024-04-02T14:45:55.956 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/00-INDEX 2024-04-02T14:45:55.958 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/3270.ChangeLog 2024-04-02T14:45:55.960 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/3270.txt 2024-04-02T14:45:55.962 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/CommonIO 2024-04-02T14:45:55.964 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/DASD 2024-04-02T14:45:55.966 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/Debugging390.txt 2024-04-02T14:45:55.969 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/cds.txt 2024-04-02T14:45:55.971 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/config3270.sh 2024-04-02T14:45:55.973 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/driver-model.txt 2024-04-02T14:45:55.975 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/kvm.txt 2024-04-02T14:45:55.977 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/monreader.txt 2024-04-02T14:45:55.980 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/qeth.txt 2024-04-02T14:45:55.981 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/s390dbf.txt 2024-04-02T14:45:55.984 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/s390/zfcpdump.txt 2024-04-02T14:45:55.985 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scheduler/ 2024-04-02T14:45:55.987 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scheduler/00-INDEX 2024-04-02T14:45:55.990 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scheduler/completion.txt 2024-04-02T14:45:55.992 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scheduler/sched-arch.txt 2024-04-02T14:45:55.994 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scheduler/sched-bwc.txt 2024-04-02T14:45:55.996 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scheduler/sched-deadline.txt 2024-04-02T14:45:55.999 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scheduler/sched-design-CFS.txt 2024-04-02T14:45:56.002 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scheduler/sched-domains.txt 2024-04-02T14:45:56.003 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scheduler/sched-nice-design.txt 2024-04-02T14:45:56.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:55 smithi204 ceph-mon[35948]: pgmap v60: 97 pgs: 97 active+clean; 56 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.6 MiB/s wr, 27 op/s 2024-04-02T14:45:56.005 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scheduler/sched-rt-group.txt 2024-04-02T14:45:56.007 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scheduler/sched-stats.txt 2024-04-02T14:45:56.009 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ 2024-04-02T14:45:56.011 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/00-INDEX 2024-04-02T14:45:56.013 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/53c700.txt 2024-04-02T14:45:56.015 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/BusLogic.txt 2024-04-02T14:45:56.017 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.1992-1997 2024-04-02T14:45:56.019 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.arcmsr 2024-04-02T14:45:56.021 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ips 2024-04-02T14:45:56.023 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.lpfc 2024-04-02T14:45:56.027 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid 2024-04-02T14:45:56.029 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid_sas 2024-04-02T14:45:56.031 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ncr53c8xx 2024-04-02T14:45:56.033 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx 2024-04-02T14:45:56.035 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx_2 2024-04-02T14:45:56.037 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/FlashPoint.txt 2024-04-02T14:45:56.040 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/LICENSE.FlashPoint 2024-04-02T14:45:56.041 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla2xxx 2024-04-02T14:45:56.043 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla4xxx 2024-04-02T14:45:56.044 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/Mylex.txt 2024-04-02T14:45:56.046 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/NinjaSCSI.txt 2024-04-02T14:45:56.048 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/aacraid.txt 2024-04-02T14:45:56.049 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/advansys.txt 2024-04-02T14:45:56.051 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/aha152x.txt 2024-04-02T14:45:56.053 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/aic79xx.txt 2024-04-02T14:45:56.055 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/aic7xxx.txt 2024-04-02T14:45:56.057 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/arcmsr_spec.txt 2024-04-02T14:45:56.060 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/bfa.txt 2024-04-02T14:45:56.061 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/bnx2fc.txt 2024-04-02T14:45:56.064 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/cxgb3i.txt 2024-04-02T14:45:56.066 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/dc395x.txt 2024-04-02T14:45:56.068 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/dpti.txt 2024-04-02T14:45:56.070 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/dtc3x80.txt 2024-04-02T14:45:56.072 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/g_NCR5380.txt 2024-04-02T14:45:56.073 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/hpsa.txt 2024-04-02T14:45:56.075 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/hptiop.txt 2024-04-02T14:45:56.076 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/in2000.txt 2024-04-02T14:45:56.078 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/libsas.txt 2024-04-02T14:45:56.080 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/link_power_management_policy.txt 2024-04-02T14:45:56.081 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/lpfc.txt 2024-04-02T14:45:56.084 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/megaraid.txt 2024-04-02T14:45:56.085 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ncr53c8xx.txt 2024-04-02T14:45:56.087 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/osd.txt 2024-04-02T14:45:56.089 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/osst.txt 2024-04-02T14:45:56.091 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ppa.txt 2024-04-02T14:45:56.092 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/qlogicfas.txt 2024-04-02T14:45:56.094 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/scsi-changer.txt 2024-04-02T14:45:56.096 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/scsi-generic.txt 2024-04-02T14:45:56.097 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/scsi-parameters.txt 2024-04-02T14:45:56.099 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/scsi.txt 2024-04-02T14:45:56.101 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/scsi_eh.txt 2024-04-02T14:45:56.103 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/scsi_fc_transport.txt 2024-04-02T14:45:56.104 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/scsi_mid_low_api.txt 2024-04-02T14:45:56.107 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/ 2024-04-02T14:45:56.108 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/Makefile 2024-04-02T14:45:56.109 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/rport_state_diagram.dot 2024-04-02T14:45:56.111 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/st.txt 2024-04-02T14:45:56.113 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/sym53c500_cs.txt 2024-04-02T14:45:56.115 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/sym53c8xx_2.txt 2024-04-02T14:45:56.117 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/tmscsim.txt 2024-04-02T14:45:56.119 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/ufs.txt 2024-04-02T14:45:56.120 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/scsi/wd719x.txt 2024-04-02T14:45:56.122 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/ 2024-04-02T14:45:56.124 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/00-INDEX 2024-04-02T14:45:56.125 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/IMA-templates.txt 2024-04-02T14:45:56.127 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/LSM.txt 2024-04-02T14:45:56.129 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/SELinux.txt 2024-04-02T14:45:56.131 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/Smack.txt 2024-04-02T14:45:56.132 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/Yama.txt 2024-04-02T14:45:56.134 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/apparmor.txt 2024-04-02T14:45:56.136 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/credentials.txt 2024-04-02T14:45:56.138 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/keys-ecryptfs.txt 2024-04-02T14:45:56.140 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/keys-request-key.txt 2024-04-02T14:45:56.142 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/keys-trusted-encrypted.txt 2024-04-02T14:45:56.143 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/keys.txt 2024-04-02T14:45:56.146 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/security/tomoyo.txt 2024-04-02T14:45:56.149 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/serial-console.txt 2024-04-02T14:45:56.151 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/serial/ 2024-04-02T14:45:56.152 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/serial/00-INDEX 2024-04-02T14:45:56.154 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/serial/README.cycladesZ 2024-04-02T14:45:56.156 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/serial/driver 2024-04-02T14:45:56.157 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/serial/moxa-smartio 2024-04-02T14:45:56.159 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/serial/n_gsm.txt 2024-04-02T14:45:56.161 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/serial/rocket.txt 2024-04-02T14:45:56.163 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/serial/serial-rs485.txt 2024-04-02T14:45:56.165 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/serial/tty.txt 2024-04-02T14:45:56.166 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sgi-ioc4.txt 2024-04-02T14:45:56.168 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sh/ 2024-04-02T14:45:56.170 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sh/new-machine.txt 2024-04-02T14:45:56.172 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sh/register-banks.txt 2024-04-02T14:45:56.173 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/smsc_ece1099.txt 2024-04-02T14:45:56.175 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/ 2024-04-02T14:45:56.177 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/ 2024-04-02T14:45:56.178 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/ALSA-Configuration.txt 2024-04-02T14:45:56.180 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/Audigy-mixer.txt 2024-04-02T14:45:56.182 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/Audiophile-Usb.txt 2024-04-02T14:45:56.184 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/Bt87x.txt 2024-04-02T14:45:56.185 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/CMIPCI.txt 2024-04-02T14:45:56.187 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/Channel-Mapping-API.txt 2024-04-02T14:45:56.189 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/ControlNames.txt 2024-04-02T14:45:56.190 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Controls.txt 2024-04-02T14:45:56.192 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Models.txt 2024-04-02T14:45:56.194 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio.txt 2024-04-02T14:45:56.196 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/Joystick.txt 2024-04-02T14:45:56.198 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/MIXART.txt 2024-04-02T14:45:56.199 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/OSS-Emulation.txt 2024-04-02T14:45:56.201 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/Procfile.txt 2024-04-02T14:45:56.203 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/README.maya44 2024-04-02T14:45:56.205 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/SB-Live-mixer.txt 2024-04-02T14:45:56.210 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/VIA82xx-mixer.txt 2024-04-02T14:45:56.210 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/alsa-parameters.txt 2024-04-02T14:45:56.210 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/compress_offload.txt 2024-04-02T14:45:56.212 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/emu10k1-jack.txt 2024-04-02T14:45:56.213 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/hda_codec.txt 2024-04-02T14:45:56.215 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/hdspm.txt 2024-04-02T14:45:56.217 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/powersave.txt 2024-04-02T14:45:56.218 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/seq_oss.html 2024-04-02T14:45:56.220 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/serial-u16550.txt 2024-04-02T14:45:56.222 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/soc/ 2024-04-02T14:45:56.224 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt 2024-04-02T14:45:56.225 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt 2024-04-02T14:45:56.227 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt 2024-04-02T14:45:56.228 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/soc/codec.txt 2024-04-02T14:45:56.230 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt 2024-04-02T14:45:56.232 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/soc/jack.txt 2024-04-02T14:45:56.233 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/soc/machine.txt 2024-04-02T14:45:56.235 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/soc/overview.txt 2024-04-02T14:45:56.237 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/soc/platform.txt 2024-04-02T14:45:56.238 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt 2024-04-02T14:45:56.240 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/ 2024-04-02T14:45:56.241 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/ALS 2024-04-02T14:45:56.243 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16 2024-04-02T14:45:56.245 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/CMI8330 2024-04-02T14:45:56.246 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/ESS 2024-04-02T14:45:56.248 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/ESS1868 2024-04-02T14:45:56.249 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/Introduction 2024-04-02T14:45:56.251 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/MultiSound 2024-04-02T14:45:56.253 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/OPL3 2024-04-02T14:45:56.255 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/Opti 2024-04-02T14:45:56.256 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/PAS16 2024-04-02T14:45:56.258 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/PSS 2024-04-02T14:45:56.260 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/PSS-updates 2024-04-02T14:45:56.261 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/README.OSS 2024-04-02T14:45:56.263 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/README.modules 2024-04-02T14:45:56.265 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/README.ymfsb 2024-04-02T14:45:56.269 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/SoundPro 2024-04-02T14:45:56.269 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/Soundblaster 2024-04-02T14:45:56.271 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/Tropez+ 2024-04-02T14:45:56.272 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/VIBRA16 2024-04-02T14:45:56.275 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/WaveArtist 2024-04-02T14:45:56.276 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/btaudio 2024-04-02T14:45:56.279 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/mwave 2024-04-02T14:45:56.280 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/oss-parameters.txt 2024-04-02T14:45:56.281 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sound/oss/ultrasound 2024-04-02T14:45:56.283 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sparse.txt 2024-04-02T14:45:56.284 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/ 2024-04-02T14:45:56.287 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/.gitignore 2024-04-02T14:45:56.288 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/00-INDEX 2024-04-02T14:45:56.289 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/Makefile 2024-04-02T14:45:56.291 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/butterfly 2024-04-02T14:45:56.292 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/ep93xx_spi 2024-04-02T14:45:56.294 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/pxa2xx 2024-04-02T14:45:56.296 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/spi-lm70llp 2024-04-02T14:45:56.298 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/spi-sc18is602 2024-04-02T14:45:56.299 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/spi-summary 2024-04-02T14:45:56.301 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/spidev 2024-04-02T14:45:56.303 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/spidev_fdx.c 2024-04-02T14:45:56.305 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/spi/spidev_test.c 2024-04-02T14:45:56.306 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/stable_api_nonsense.txt 2024-04-02T14:45:56.308 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/stable_kernel_rules.txt 2024-04-02T14:45:56.310 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/static-keys.txt 2024-04-02T14:45:56.311 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/svga.txt 2024-04-02T14:45:56.313 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sysctl/ 2024-04-02T14:45:56.315 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sysctl/00-INDEX 2024-04-02T14:45:56.317 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sysctl/README 2024-04-02T14:45:56.318 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sysctl/abi.txt 2024-04-02T14:45:56.320 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sysctl/fs.txt 2024-04-02T14:45:56.322 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sysctl/kernel.txt 2024-04-02T14:45:56.325 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sysctl/net.txt 2024-04-02T14:45:56.326 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sysctl/sunrpc.txt 2024-04-02T14:45:56.328 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sysctl/vm.txt 2024-04-02T14:45:56.330 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sysfs-rules.txt 2024-04-02T14:45:56.331 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/sysrq.txt 2024-04-02T14:45:56.334 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/target/ 2024-04-02T14:45:56.339 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.py 2024-04-02T14:45:56.339 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.txt 2024-04-02T14:45:56.339 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/target/tcmu-design.txt 2024-04-02T14:45:56.340 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/thermal/ 2024-04-02T14:45:56.342 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/thermal/cpu-cooling-api.txt 2024-04-02T14:45:56.344 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal 2024-04-02T14:45:56.345 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal_emulation 2024-04-02T14:45:56.347 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/thermal/intel_powerclamp.txt 2024-04-02T14:45:56.349 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/thermal/nouveau_thermal 2024-04-02T14:45:56.351 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/thermal/sysfs-api.txt 2024-04-02T14:45:56.352 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/thermal/x86_pkg_temperature_thermal 2024-04-02T14:45:56.354 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/this_cpu_ops.txt 2024-04-02T14:45:56.356 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/ 2024-04-02T14:45:56.358 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/.gitignore 2024-04-02T14:45:56.360 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/00-INDEX 2024-04-02T14:45:56.361 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/Makefile 2024-04-02T14:45:56.363 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/NO_HZ.txt 2024-04-02T14:45:56.365 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/highres.txt 2024-04-02T14:45:56.366 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/hpet.txt 2024-04-02T14:45:56.368 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/hpet_example.c 2024-04-02T14:45:56.370 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/hrtimers.txt 2024-04-02T14:45:56.372 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/timekeeping.txt 2024-04-02T14:45:56.374 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/timer_stats.txt 2024-04-02T14:45:56.376 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/timers/timers-howto.txt 2024-04-02T14:45:56.377 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/tpm/ 2024-04-02T14:45:56.379 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/tpm/xen-tpmfront.txt 2024-04-02T14:45:56.381 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/ 2024-04-02T14:45:56.382 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/coresight.txt 2024-04-02T14:45:56.384 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/events-kmem.txt 2024-04-02T14:45:56.386 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/events-nmi.txt 2024-04-02T14:45:56.387 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/events-power.txt 2024-04-02T14:45:56.389 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/events.txt 2024-04-02T14:45:56.391 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/ftrace-design.txt 2024-04-02T14:45:56.393 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/ftrace.txt 2024-04-02T14:45:56.396 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/function-graph-fold.vim 2024-04-02T14:45:56.398 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/kprobetrace.txt 2024-04-02T14:45:56.400 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/mmiotrace.txt 2024-04-02T14:45:56.401 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/postprocess/ 2024-04-02T14:45:56.403 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-pagealloc-postprocess.pl 2024-04-02T14:45:56.404 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-vmscan-postprocess.pl 2024-04-02T14:45:56.406 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/ring-buffer-design.txt 2024-04-02T14:45:56.408 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/tracepoint-analysis.txt 2024-04-02T14:45:56.410 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/tracepoints.txt 2024-04-02T14:45:56.414 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/trace/uprobetracer.txt 2024-04-02T14:45:56.414 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/unaligned-memory-access.txt 2024-04-02T14:45:56.415 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/unicode.txt 2024-04-02T14:45:56.417 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/unshare.txt 2024-04-02T14:45:56.419 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/ 2024-04-02T14:45:56.420 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/CREDITS 2024-04-02T14:45:56.422 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/URB.txt 2024-04-02T14:45:56.424 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/WUSB-Design-overview.txt 2024-04-02T14:45:56.425 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/acm.txt 2024-04-02T14:45:56.427 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/anchors.txt 2024-04-02T14:45:56.429 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/authorization.txt 2024-04-02T14:45:56.430 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/bulk-streams.txt 2024-04-02T14:45:56.432 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/callbacks.txt 2024-04-02T14:45:56.433 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/chipidea.txt 2024-04-02T14:45:56.435 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/dma.txt 2024-04-02T14:45:56.437 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/dwc3.txt 2024-04-02T14:45:56.438 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/ehci.txt 2024-04-02T14:45:56.440 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/error-codes.txt 2024-04-02T14:45:56.441 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/functionfs.txt 2024-04-02T14:45:56.443 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/gadget-testing.txt 2024-04-02T14:45:56.446 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/gadget_configfs.txt 2024-04-02T14:45:56.448 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/gadget_hid.txt 2024-04-02T14:45:56.450 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/gadget_multi.txt 2024-04-02T14:45:56.451 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/gadget_printer.txt 2024-04-02T14:45:56.453 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/gadget_serial.txt 2024-04-02T14:45:56.454 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/hotplug.txt 2024-04-02T14:45:56.456 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/iuu_phoenix.txt 2024-04-02T14:45:56.457 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/linux-cdc-acm.inf 2024-04-02T14:45:56.459 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/linux.inf 2024-04-02T14:45:56.461 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/mass-storage.txt 2024-04-02T14:45:56.463 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/misc_usbsevseg.txt 2024-04-02T14:45:56.464 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/mtouchusb.txt 2024-04-02T14:45:56.466 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/ohci.txt 2024-04-02T14:45:56.468 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/persist.txt 2024-04-02T14:45:56.470 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/power-management.txt 2024-04-02T14:45:56.472 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/proc_usb_info.txt 2024-04-02T14:45:56.474 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/rio.txt 2024-04-02T14:45:56.475 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/usb-help.txt 2024-04-02T14:45:56.477 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/usb-serial.txt 2024-04-02T14:45:56.479 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/usbmon.txt 2024-04-02T14:45:56.481 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/usb/wusb-cbaf 2024-04-02T14:45:56.483 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vDSO/ 2024-04-02T14:45:56.484 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vDSO/.gitignore 2024-04-02T14:45:56.486 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vDSO/Makefile 2024-04-02T14:45:56.487 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vDSO/parse_vdso.c 2024-04-02T14:45:56.489 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vDSO/vdso_standalone_test_x86.c 2024-04-02T14:45:56.490 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vDSO/vdso_test.c 2024-04-02T14:45:56.492 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vfio.txt 2024-04-02T14:45:56.494 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vgaarbiter.txt 2024-04-02T14:45:56.496 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video-output.txt 2024-04-02T14:45:56.497 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/ 2024-04-02T14:45:56.499 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/4CCs.txt 2024-04-02T14:45:56.500 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/API.html 2024-04-02T14:45:56.502 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.au0828 2024-04-02T14:45:56.503 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.bttv 2024-04-02T14:45:56.505 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx23885 2024-04-02T14:45:56.506 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx88 2024-04-02T14:45:56.508 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.em28xx 2024-04-02T14:45:56.510 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.ivtv 2024-04-02T14:45:56.511 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7134 2024-04-02T14:45:56.513 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7164 2024-04-02T14:45:56.514 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tm6000 2024-04-02T14:45:56.516 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tuner 2024-04-02T14:45:56.518 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.usbvision 2024-04-02T14:45:56.519 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/Makefile 2024-04-02T14:45:56.521 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/README.cpia2 2024-04-02T14:45:56.523 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/README.cx88 2024-04-02T14:45:56.524 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/README.davinci-vpbe 2024-04-02T14:45:56.526 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/README.ir 2024-04-02T14:45:56.528 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/README.ivtv 2024-04-02T14:45:56.530 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/README.pvrusb2 2024-04-02T14:45:56.531 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/README.saa7134 2024-04-02T14:45:56.533 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/Zoran 2024-04-02T14:45:56.535 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/ 2024-04-02T14:45:56.536 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/CONTRIBUTORS 2024-04-02T14:45:56.538 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/Cards 2024-04-02T14:45:56.540 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/ICs 2024-04-02T14:45:56.541 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/Insmod-options 2024-04-02T14:45:56.543 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/MAKEDEV 2024-04-02T14:45:56.544 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modprobe.conf 2024-04-02T14:45:56.546 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modules.conf 2024-04-02T14:45:56.547 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/PROBLEMS 2024-04-02T14:45:56.549 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/README 2024-04-02T14:45:56.550 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.WINVIEW 2024-04-02T14:45:56.552 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.freeze 2024-04-02T14:45:56.554 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.quirks 2024-04-02T14:45:56.555 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/Sound-FAQ 2024-04-02T14:45:56.557 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/Specs 2024-04-02T14:45:56.558 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/THANKS 2024-04-02T14:45:56.560 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/bttv/Tuners 2024-04-02T14:45:56.562 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cafe_ccic 2024-04-02T14:45:56.564 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cpia2_overview.txt 2024-04-02T14:45:56.565 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx18.txt 2024-04-02T14:45:56.567 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/ 2024-04-02T14:45:56.568 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.hm12 2024-04-02T14:45:56.570 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.vbi 2024-04-02T14:45:56.572 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-calling.txt 2024-04-02T14:45:56.573 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-api.txt 2024-04-02T14:45:56.575 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-regs.txt 2024-04-02T14:45:56.577 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-dma.txt 2024-04-02T14:45:56.578 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-encoder-api.txt 2024-04-02T14:45:56.581 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-memory.txt 2024-04-02T14:45:56.582 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-osd-api.txt 2024-04-02T14:45:56.584 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-upload.txt 2024-04-02T14:45:56.585 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx88/ 2024-04-02T14:45:56.587 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/cx88/hauppauge-wintv-cx88-ir.txt 2024-04-02T14:45:56.589 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/extract_xc3028.pl 2024-04-02T14:45:56.591 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/fimc.txt 2024-04-02T14:45:56.592 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/gspca.txt 2024-04-02T14:45:56.594 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/hauppauge-wintv-cx88-ir.txt 2024-04-02T14:45:56.596 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/lifeview.txt 2024-04-02T14:45:56.598 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/meye.txt 2024-04-02T14:45:56.600 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/not-in-cx2388x-datasheet.txt 2024-04-02T14:45:56.601 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/omap3isp.txt 2024-04-02T14:45:56.603 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/omap4_camera.txt 2024-04-02T14:45:56.604 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/pxa_camera.txt 2024-04-02T14:45:56.606 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/radiotrack.txt 2024-04-02T14:45:56.608 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/sh_mobile_ceu_camera.txt 2024-04-02T14:45:56.609 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/si470x.txt 2024-04-02T14:45:56.611 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/si4713.txt 2024-04-02T14:45:56.613 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/si476x.txt 2024-04-02T14:45:56.618 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/soc-camera.txt 2024-04-02T14:45:56.618 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/uvcvideo.txt 2024-04-02T14:45:56.618 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/v4l2-controls.txt 2024-04-02T14:45:56.619 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/v4l2-framework.txt 2024-04-02T14:45:56.622 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/v4l2-pci-skeleton.c 2024-04-02T14:45:56.624 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/videobuf 2024-04-02T14:45:56.626 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/vivid.txt 2024-04-02T14:45:56.628 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/video4linux/zr364xx.txt 2024-04-02T14:45:56.630 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/ 2024-04-02T14:45:56.632 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/00-INDEX 2024-04-02T14:45:56.634 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/ 2024-04-02T14:45:56.635 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/00-INDEX 2024-04-02T14:45:56.637 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/api.txt 2024-04-02T14:45:56.640 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/cpuid.txt 2024-04-02T14:45:56.642 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/ 2024-04-02T14:45:56.644 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/README 2024-04-02T14:45:56.645 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt 2024-04-02T14:45:56.647 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/mpic.txt 2024-04-02T14:45:56.649 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/s390_flic.txt 2024-04-02T14:45:56.651 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vfio.txt 2024-04-02T14:45:56.653 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vm.txt 2024-04-02T14:45:56.654 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/xics.txt 2024-04-02T14:45:56.656 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/hypercalls.txt 2024-04-02T14:45:56.658 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/locking.txt 2024-04-02T14:45:56.660 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/mmu.txt 2024-04-02T14:45:56.661 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/msr.txt 2024-04-02T14:45:56.664 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/nested-vmx.txt 2024-04-02T14:45:56.665 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/ppc-pv.txt 2024-04-02T14:45:56.667 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/review-checklist.txt 2024-04-02T14:45:56.669 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/s390-diag.txt 2024-04-02T14:45:56.671 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/kvm/timekeeping.txt 2024-04-02T14:45:56.672 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/paravirt_ops.txt 2024-04-02T14:45:56.674 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/uml/ 2024-04-02T14:45:56.677 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt 2024-04-02T14:45:56.680 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/ 2024-04-02T14:45:56.684 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/.gitignore 2024-04-02T14:45:56.685 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/00-INDEX 2024-04-02T14:45:56.686 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/active_mm.txt 2024-04-02T14:45:56.689 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/balance 2024-04-02T14:45:56.690 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/cleancache.txt 2024-04-02T14:45:56.691 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/frontswap.txt 2024-04-02T14:45:56.693 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/highmem.txt 2024-04-02T14:45:56.695 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/hugetlbpage.txt 2024-04-02T14:45:56.700 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/hwpoison.txt 2024-04-02T14:45:56.702 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/ksm.txt 2024-04-02T14:45:56.703 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/numa 2024-04-02T14:45:56.705 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/numa_memory_policy.txt 2024-04-02T14:45:56.707 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/overcommit-accounting 2024-04-02T14:45:56.709 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/page_migration 2024-04-02T14:45:56.710 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/page_owner.txt 2024-04-02T14:45:56.712 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/pagemap.txt 2024-04-02T14:45:56.714 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/remap_file_pages.txt 2024-04-02T14:45:56.715 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/slub.txt 2024-04-02T14:45:56.717 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/soft-dirty.txt 2024-04-02T14:45:56.719 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/split_page_table_lock 2024-04-02T14:45:56.720 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/transhuge.txt 2024-04-02T14:45:56.722 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/unevictable-lru.txt 2024-04-02T14:45:56.725 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vm/zswap.txt 2024-04-02T14:45:56.726 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/vme_api.txt 2024-04-02T14:45:56.729 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/volatile-considered-harmful.txt 2024-04-02T14:45:56.731 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/ 2024-04-02T14:45:56.732 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/00-INDEX 2024-04-02T14:45:56.734 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/masters/ 2024-04-02T14:45:56.735 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/masters/00-INDEX 2024-04-02T14:45:56.737 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/masters/ds2482 2024-04-02T14:45:56.739 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/masters/ds2490 2024-04-02T14:45:56.741 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/masters/mxc-w1 2024-04-02T14:45:56.743 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/masters/omap-hdq 2024-04-02T14:45:56.744 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/masters/w1-gpio 2024-04-02T14:45:56.746 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/slaves/ 2024-04-02T14:45:56.747 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/slaves/00-INDEX 2024-04-02T14:45:56.749 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2406 2024-04-02T14:45:56.751 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2423 2024-04-02T14:45:56.753 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds28e04 2024-04-02T14:45:56.754 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/slaves/w1_therm 2024-04-02T14:45:56.756 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/w1.generic 2024-04-02T14:45:56.758 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/w1/w1.netlink 2024-04-02T14:45:56.759 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/ 2024-04-02T14:45:56.761 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/Makefile 2024-04-02T14:45:56.763 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/convert_drivers_to_kernel_api.txt 2024-04-02T14:45:56.764 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/hpwdt.txt 2024-04-02T14:45:56.766 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/pcwd-watchdog.txt 2024-04-02T14:45:56.768 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/src/ 2024-04-02T14:45:56.771 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/src/.gitignore 2024-04-02T14:45:56.772 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/src/Makefile 2024-04-02T14:45:56.774 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-simple.c 2024-04-02T14:45:56.775 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-test.c 2024-04-02T14:45:56.776 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/watchdog-api.txt 2024-04-02T14:45:56.778 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/watchdog-kernel-api.txt 2024-04-02T14:45:56.780 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/watchdog-parameters.txt 2024-04-02T14:45:56.781 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/watchdog/wdt.txt 2024-04-02T14:45:56.783 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/wimax/ 2024-04-02T14:45:56.785 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/wimax/README.i2400m 2024-04-02T14:45:56.787 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/wimax/README.wimax 2024-04-02T14:45:56.788 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/workqueue.txt 2024-04-02T14:45:56.790 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/ 2024-04-02T14:45:56.792 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/00-INDEX 2024-04-02T14:45:56.794 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/boot.txt 2024-04-02T14:45:56.796 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/early-microcode.txt 2024-04-02T14:45:56.797 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/earlyprintk.txt 2024-04-02T14:45:56.799 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/entry_64.txt 2024-04-02T14:45:56.801 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/exception-tables.txt 2024-04-02T14:45:56.802 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/i386/ 2024-04-02T14:45:56.803 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/i386/IO-APIC.txt 2024-04-02T14:45:56.805 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/intel_mpx.txt 2024-04-02T14:45:56.807 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/mtrr.txt 2024-04-02T14:45:56.809 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/pat.txt 2024-04-02T14:45:56.811 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/tlb.txt 2024-04-02T14:45:56.812 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/usb-legacy-support.txt 2024-04-02T14:45:56.814 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/x86_64/ 2024-04-02T14:45:56.816 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/x86_64/00-INDEX 2024-04-02T14:45:56.817 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/x86_64/boot-options.txt 2024-04-02T14:45:56.819 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/x86_64/cpu-hotplug-spec 2024-04-02T14:45:56.821 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/x86_64/fake-numa-for-cpusets 2024-04-02T14:45:56.823 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/x86_64/kernel-stacks 2024-04-02T14:45:56.824 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/x86_64/machinecheck 2024-04-02T14:45:56.826 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/x86_64/mm.txt 2024-04-02T14:45:56.828 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/x86_64/uefi.txt 2024-04-02T14:45:56.830 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/x86/zero-page.txt 2024-04-02T14:45:56.831 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/xillybus.txt 2024-04-02T14:45:56.833 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/xtensa/ 2024-04-02T14:45:56.835 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/xtensa/atomctl.txt 2024-04-02T14:45:56.836 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/xtensa/mmu.txt 2024-04-02T14:45:56.838 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/xz.txt 2024-04-02T14:45:56.840 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/ 2024-04-02T14:45:56.841 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/CodingStyle 2024-04-02T14:45:56.843 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/HOWTO 2024-04-02T14:45:56.845 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/IRQ.txt 2024-04-02T14:45:56.847 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/SecurityBugs 2024-04-02T14:45:56.849 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingDrivers 2024-04-02T14:45:56.851 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingPatches 2024-04-02T14:45:56.853 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/arm/ 2024-04-02T14:45:56.855 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/arm/Booting 2024-04-02T14:45:56.857 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/arm/kernel_user_helpers.txt 2024-04-02T14:45:56.859 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/arm64/ 2024-04-02T14:45:56.861 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/arm64/booting.txt 2024-04-02T14:45:56.862 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/arm64/memory.txt 2024-04-02T14:45:56.864 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/arm64/tagged-pointers.txt 2024-04-02T14:45:56.866 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/basic_profiling.txt 2024-04-02T14:45:56.868 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/email-clients.txt 2024-04-02T14:45:56.869 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/ 2024-04-02T14:45:56.871 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/sysfs.txt 2024-04-02T14:45:56.873 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/gpio.txt 2024-04-02T14:45:57.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:56 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:45:57.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:56 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:45:57.097 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/io_ordering.txt 2024-04-02T14:45:57.099 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/magic-number.txt 2024-04-02T14:45:57.100 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/oops-tracing.txt 2024-04-02T14:45:57.102 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/sparse.txt 2024-04-02T14:45:57.104 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/stable_api_nonsense.txt 2024-04-02T14:45:57.106 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/stable_kernel_rules.txt 2024-04-02T14:45:57.108 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/ 2024-04-02T14:45:57.109 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/omap3isp.txt 2024-04-02T14:45:57.111 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/v4l2-framework.txt 2024-04-02T14:45:57.113 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zh_CN/volatile-considered-harmful.txt 2024-04-02T14:45:57.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:56 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:45:57.115 INFO:tasks.workunit.client.0.smithi062.stdout:linux-4.0.5/Documentation/zorro.txt 2024-04-02T14:45:58.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:57 smithi204 ceph-mon[35948]: pgmap v61: 97 pgs: 97 active+clean; 92 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 7.6 MiB/s wr, 81 op/s 2024-04-02T14:45:58.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:57 smithi062 ceph-mon[32116]: pgmap v61: 97 pgs: 97 active+clean; 92 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 7.6 MiB/s wr, 81 op/s 2024-04-02T14:45:58.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:57 smithi113 ceph-mon[36135]: pgmap v61: 97 pgs: 97 active+clean; 92 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 7.6 MiB/s wr, 81 op/s 2024-04-02T14:45:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.642+0000 7fbbd0a00640 1 -- 172.21.15.62:0/1389383895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbcc073f30 msgr2=0x7fbbcc074310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.642+0000 7fbbd0a00640 1 --2- 172.21.15.62:0/1389383895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbcc073f30 0x7fbbcc074310 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009900 tx=0x7fbbbc02bcd0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.642+0000 7fbbd0a00640 1 -- 172.21.15.62:0/1389383895 shutdown_connections 2024-04-02T14:45:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.642+0000 7fbbd0a00640 1 --2- 172.21.15.62:0/1389383895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbcc10e2f0 0x7fbbcc110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.642+0000 7fbbd0a00640 1 --2- 172.21.15.62:0/1389383895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbcc074850 0x7fbbcc10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.642+0000 7fbbd0a00640 1 --2- 172.21.15.62:0/1389383895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbcc073f30 0x7fbbcc074310 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.642+0000 7fbbd0a00640 1 -- 172.21.15.62:0/1389383895 >> 172.21.15.62:0/1389383895 conn(0x7fbbcc0fd580 msgr2=0x7fbbcc0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.642+0000 7fbbd0a00640 1 -- 172.21.15.62:0/1389383895 shutdown_connections 2024-04-02T14:45:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.642+0000 7fbbd0a00640 1 -- 172.21.15.62:0/1389383895 wait complete. 2024-04-02T14:45:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.643+0000 7fbbd0a00640 1 Processor -- start 2024-04-02T14:45:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.644+0000 7fbbd0a00640 1 -- start start 2024-04-02T14:45:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.644+0000 7fbbd0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbcc073f30 0x7fbbcc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.644+0000 7fbbd0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbcc074850 0x7fbbcc19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.644+0000 7fbbd0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbcc10e2f0 0x7fbbcc19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.644+0000 7fbbd0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1141e0 con 0x7fbbcc074850 2024-04-02T14:45:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.644+0000 7fbbd0a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc114060 con 0x7fbbcc073f30 2024-04-02T14:45:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.644+0000 7fbbd0a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc114360 con 0x7fbbcc10e2f0 2024-04-02T14:45:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.645+0000 7fbbc9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbcc073f30 0x7fbbcc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.645+0000 7fbbc9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbcc073f30 0x7fbbcc19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:56338/0 (socket says 172.21.15.62:56338) 2024-04-02T14:45:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.645+0000 7fbbc9e00640 1 -- 172.21.15.62:0/4238550845 learned_addr learned my addr 172.21.15.62:0/4238550845 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.645+0000 7fbbc9400640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbcc074850 0x7fbbcc19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.645+0000 7fbbca800640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbcc10e2f0 0x7fbbcc19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.645+0000 7fbbc9e00640 1 -- 172.21.15.62:0/4238550845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbcc10e2f0 msgr2=0x7fbbcc19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.645+0000 7fbbc9e00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbcc10e2f0 0x7fbbcc19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.645+0000 7fbbc9e00640 1 -- 172.21.15.62:0/4238550845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbcc074850 msgr2=0x7fbbcc19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.645+0000 7fbbc9e00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbcc074850 0x7fbbcc19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.645+0000 7fbbc9e00640 1 -- 172.21.15.62:0/4238550845 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc19f630 con 0x7fbbcc073f30 2024-04-02T14:45:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.645+0000 7fbbc9e00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbcc073f30 0x7fbbcc19a640 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc000c00 tx=0x7fbbbc004a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.646+0000 7fbbc2a00640 1 -- 172.21.15.62:0/4238550845 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc043020 con 0x7fbbcc073f30 2024-04-02T14:45:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.646+0000 7fbbc2a00640 1 -- 172.21.15.62:0/4238550845 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbbbc004310 con 0x7fbbcc073f30 2024-04-02T14:45:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.646+0000 7fbbd0a00640 1 -- 172.21.15.62:0/4238550845 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbcc19f8c0 con 0x7fbbcc073f30 2024-04-02T14:45:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.646+0000 7fbbc2a00640 1 -- 172.21.15.62:0/4238550845 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc003b50 con 0x7fbbcc073f30 2024-04-02T14:45:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.646+0000 7fbbd0a00640 1 -- 172.21.15.62:0/4238550845 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbcc1a71b0 con 0x7fbbcc073f30 2024-04-02T14:45:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.650+0000 7fbbd37cb740 1 Processor -- start 2024-04-02T14:45:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.650+0000 7fbbd37cb740 1 -- start start 2024-04-02T14:45:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.650+0000 7fbbc2a00640 1 -- 172.21.15.62:0/4238550845 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbbbc046020 con 0x7fbbcc073f30 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.651+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac52290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.651+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac527d0 0x55f20ac5ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.651+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac5d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.651+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f20abb6d70 con 0x55f20ac51eb0 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.651+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f20ab5f2a0 con 0x55f20ac5b3b0 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.651+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f20abb2d20 con 0x55f20ac527d0 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.651+0000 7fbbc9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac527d0 0x55f20ac5ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.651+0000 7fbbca800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac52290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.651+0000 7fbbca800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac52290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53898/0 (socket says 172.21.15.62:53898) 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.651+0000 7fbbca800640 1 -- 172.21.15.62:0/2895664122 learned_addr learned my addr 172.21.15.62:0/2895664122 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.651+0000 7fbbc9400640 1 --2- 172.21.15.62:0/2895664122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac5d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.652+0000 7fbbca800640 1 -- 172.21.15.62:0/2895664122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac527d0 msgr2=0x55f20ac5ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.652+0000 7fbbca800640 1 --2- 172.21.15.62:0/2895664122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac527d0 0x55f20ac5ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.652+0000 7fbbca800640 1 -- 172.21.15.62:0/2895664122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 msgr2=0x55f20ac5d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.652+0000 7fbbca800640 1 --2- 172.21.15.62:0/2895664122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac5d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.652+0000 7fbbca800640 1 -- 172.21.15.62:0/2895664122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f20ab73840 con 0x55f20ac51eb0 2024-04-02T14:45:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.652+0000 7fbbc9400640 1 --2- 172.21.15.62:0/2895664122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac5d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.652+0000 7fbbca800640 1 --2- 172.21.15.62:0/2895664122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac52290 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009f40 tx=0x7fbbc4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861f253c3510cc76 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.652+0000 7fbbb7000640 1 -- 172.21.15.62:0/2895664122 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4003b70 con 0x55f20ac51eb0 2024-04-02T14:45:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.652+0000 7fbbb7000640 1 -- 172.21.15.62:0/2895664122 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbbc4003d10 con 0x55f20ac51eb0 2024-04-02T14:45:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.652+0000 7fbbb7000640 1 -- 172.21.15.62:0/2895664122 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4011b60 con 0x55f20ac51eb0 2024-04-02T14:45:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.652+0000 7fbbc2a00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbbb00a3dc0 0x7fbbb00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.653+0000 7fbbc2a00640 1 -- 172.21.15.62:0/4238550845 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fbbbc0f27a0 con 0x7fbbcc073f30 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.653+0000 7fbbc9e00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbbb00a3dc0 0x7fbbb00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.653+0000 7fbbd37cb740 1 -- 172.21.15.62:0/2895664122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 msgr2=0x55f20ac52290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.653+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/2895664122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac52290 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009f40 tx=0x7fbbc4010860 comp rx=0 tx=0).stop 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.653+0000 7fbbd37cb740 1 -- 172.21.15.62:0/2895664122 shutdown_connections 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.653+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/2895664122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac5d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.653+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/2895664122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac527d0 0x55f20ac5ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.653+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/2895664122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac52290 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.653+0000 7fbbd37cb740 1 -- 172.21.15.62:0/2895664122 >> 172.21.15.62:0/2895664122 conn(0x55f20ac4c5e0 msgr2=0x55f20ac4c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.653+0000 7fbbd37cb740 1 -- 172.21.15.62:0/2895664122 shutdown_connections 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.653+0000 7fbbd37cb740 1 -- 172.21.15.62:0/2895664122 wait complete. 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.654+0000 7fbbc9e00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbbb00a3dc0 0x7fbbb00a6280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55f20ac51980 tx=0x7fbbbc0f8530 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.654+0000 7fbbd37cb740 1 -- start start 2024-04-02T14:45:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.654+0000 7fbbd37cb740 1 Processor -- start 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac51eb0 0x55f20ac584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac527d0 0x55f20ac589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac58f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f20ac5e1a0 con 0x55f20ac527d0 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f20ab72660 con 0x55f20ac5b3b0 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f20ac5e320 con 0x55f20ac51eb0 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbd37cb740 10 client.0 _ll_register_callbacks cb 0x55f20ab72050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbc9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac527d0 0x55f20ac589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbc9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac527d0 0x55f20ac589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53914/0 (socket says 172.21.15.62:53914) 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbc9400640 1 -- 172.21.15.62:0/1825332582 learned_addr learned my addr 172.21.15.62:0/1825332582 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbca800640 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac51eb0 0x55f20ac584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbca800640 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac58f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbc9400640 1 -- 172.21.15.62:0/1825332582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac51eb0 msgr2=0x55f20ac584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbc9400640 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac51eb0 0x55f20ac584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbc9400640 1 -- 172.21.15.62:0/1825332582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 msgr2=0x55f20ac58f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbc9400640 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac58f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbc9400640 1 -- 172.21.15.62:0/1825332582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f20ab81e80 con 0x55f20ac527d0 2024-04-02T14:45:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.655+0000 7fbbca800640 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac58f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:45:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.656+0000 7fbbc9400640 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac527d0 0x55f20ac589e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800de80 tx=0x7fbbb80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0ca5376834f215 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.656+0000 7fbbab400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:45:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.656+0000 7fbbab400640 1 -- 172.21.15.62:0/1825332582 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8005220 con 0x55f20ac527d0 2024-04-02T14:45:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.656+0000 7fbbab400640 1 -- 172.21.15.62:0/1825332582 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fbbb80053c0 con 0x55f20ac527d0 2024-04-02T14:45:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.656+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1825332582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc4003d10 con 0x55f20ac527d0 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.656+0000 7fbbab400640 1 -- 172.21.15.62:0/1825332582 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbbb8020020 con 0x55f20ac527d0 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.656+0000 7fbbd37cb740 10 client.15990 fetch_fsmap learned FSMap version 30 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.657+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1825332582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f20a8cb210 con 0x55f20ac527d0 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.657+0000 7fbbab400640 1 -- 172.21.15.62:0/1825332582 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbbc4003d10 con 0x55f20ac527d0 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.657+0000 7fbbab400640 1 -- 172.21.15.62:0/1825332582 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbbb8012b80 con 0x55f20ac527d0 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.657+0000 7fbbd37cb740 10 client.15990 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.657+0000 7fbbd37cb740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.657+0000 7fbbd37cb740 20 client.15990 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.657+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f20ac6b850 0x55f20ac6dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.657+0000 7fbbd37cb740 4 client.15990 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.657+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1825332582 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f20ab66cd0 con 0x55f20ac6b850 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.658+0000 7fbbca800640 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f20ac6b850 0x55f20ac6dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.658+0000 7fbbab400640 10 client.15990 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.658+0000 7fbbca800640 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f20ac6b850 0x55f20ac6dc90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed109bb2067cdcc2 server_cookie=f6c23f02ba688f1e in_seq=0 out_seq=0 2024-04-02T14:45:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbab400640 1 -- 172.21.15.62:0/1825332582 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f20ab66cd0 con 0x55f20ac6b850 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbab400640 10 client.15990 handle_command_reply: tid=0 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbd37cb740 1 client.15990 shutdown 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1825332582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac527d0 msgr2=0x55f20ac589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac527d0 0x55f20ac589e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800de80 tx=0x7fbbb80117b0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1825332582 shutdown_connections 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f20ac6b850 0x55f20ac6dc90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac58f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac527d0 0x55f20ac589e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbab400640 0 client.15990 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1825332582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac51eb0 0x55f20ac584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.659+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1825332582 >> 172.21.15.62:0/1825332582 conn(0x55f20ac4c5e0 msgr2=0x55f20ac4e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.660+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1825332582 shutdown_connections 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.660+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1825332582 wait complete. 2024-04-02T14:45:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.660+0000 7fbbd37cb740 20 client.15990 trim_cache size 0 max 16384 2024-04-02T14:45:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.676+0000 7fbbd37cb740 1 Processor -- start 2024-04-02T14:45:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.676+0000 7fbbd37cb740 1 -- start start 2024-04-02T14:45:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.676+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac67060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.676+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac527d0 0x55f20ac58d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.676+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac735a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.676+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1af970 con 0x55f20ac51eb0 2024-04-02T14:45:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.676+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1b1570 con 0x55f20ac5b3b0 2024-04-02T14:45:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.676+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1b0040 con 0x55f20ac527d0 2024-04-02T14:45:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.676+0000 7fbbca800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac67060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.677+0000 7fbbca800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac67060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53924/0 (socket says 172.21.15.62:53924) 2024-04-02T14:45:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.677+0000 7fbbca800640 1 -- 172.21.15.62:0/2484053181 learned_addr learned my addr 172.21.15.62:0/2484053181 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.677+0000 7fbbc9400640 1 --2- 172.21.15.62:0/2484053181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac527d0 0x55f20ac58d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.677+0000 7fbbca800640 1 --2- 172.21.15.62:0/2484053181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.677+0000 7fbbca800640 1 -- 172.21.15.62:0/2484053181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac527d0 msgr2=0x55f20ac58d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.677+0000 7fbbca800640 1 --2- 172.21.15.62:0/2484053181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac527d0 0x55f20ac58d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.677+0000 7fbbca800640 1 -- 172.21.15.62:0/2484053181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 msgr2=0x55f20ac735a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.677+0000 7fbbca800640 1 --2- 172.21.15.62:0/2484053181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac735a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.677+0000 7fbbca800640 1 -- 172.21.15.62:0/2484053181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f20a997620 con 0x55f20ac51eb0 2024-04-02T14:45:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.677+0000 7fbbca800640 1 --2- 172.21.15.62:0/2484053181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac67060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4011670 tx=0x7fbbc4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c93e66271a8312 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.678+0000 7fbbb6600640 1 -- 172.21.15.62:0/2484053181 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4024040 con 0x55f20ac51eb0 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.678+0000 7fbbb6600640 1 -- 172.21.15.62:0/2484053181 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbbc4011d70 con 0x55f20ac51eb0 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.678+0000 7fbbb6600640 1 -- 172.21.15.62:0/2484053181 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc402a940 con 0x55f20ac51eb0 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.678+0000 7fbbd37cb740 1 -- 172.21.15.62:0/2484053181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 msgr2=0x55f20ac67060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.678+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/2484053181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac67060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4011670 tx=0x7fbbc4005fd0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.678+0000 7fbbd37cb740 1 -- 172.21.15.62:0/2484053181 shutdown_connections 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.678+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/2484053181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac5b3b0 0x55f20ac735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.678+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/2484053181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac527d0 0x55f20ac58d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.678+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/2484053181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac67060 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.678+0000 7fbbd37cb740 1 -- 172.21.15.62:0/2484053181 >> 172.21.15.62:0/2484053181 conn(0x55f20ac4c5e0 msgr2=0x55f20ac4e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.678+0000 7fbbd37cb740 1 -- 172.21.15.62:0/2484053181 shutdown_connections 2024-04-02T14:45:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.679+0000 7fbbd37cb740 1 -- 172.21.15.62:0/2484053181 wait complete. 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.679+0000 7fbbd37cb740 1 -- start start 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.679+0000 7fbbd37cb740 1 Processor -- start 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.679+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac55280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.679+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac527d0 0x55f20ac557c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbd37cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac5b3b0 0x55f20ac55d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1b0de0 con 0x55f20ac51eb0 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1b0710 con 0x55f20ac527d0 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbd37cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1ad950 con 0x55f20ac5b3b0 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbd37cb740 10 client.0 _ll_register_callbacks cb 0x7fbbb800da10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbca800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac55280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbca800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac55280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53934/0 (socket says 172.21.15.62:53934) 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbca800640 1 -- 172.21.15.62:0/1862821055 learned_addr learned my addr 172.21.15.62:0/1862821055 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbc9400640 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac527d0 0x55f20ac557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbca800640 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac5b3b0 0x55f20ac55d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbca800640 1 -- 172.21.15.62:0/1862821055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac5b3b0 msgr2=0x55f20ac55d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbca800640 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac5b3b0 0x55f20ac55d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbca800640 1 -- 172.21.15.62:0/1862821055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac527d0 msgr2=0x55f20ac557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:45:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbca800640 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac527d0 0x55f20ac557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.680+0000 7fbbca800640 1 -- 172.21.15.62:0/1862821055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f20acd0f40 con 0x55f20ac51eb0 2024-04-02T14:45:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.681+0000 7fbbca800640 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac55280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4005c40 tx=0x7fbbc4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957b949872c56275 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:45:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.681+0000 7fbbab400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:45:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.681+0000 7fbbab400640 1 -- 172.21.15.62:0/1862821055 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40119f0 con 0x55f20ac51eb0 2024-04-02T14:45:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.681+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1862821055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc4011d70 con 0x55f20ac51eb0 2024-04-02T14:45:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.681+0000 7fbbab400640 1 -- 172.21.15.62:0/1862821055 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fbbc402c4a0 con 0x55f20ac51eb0 2024-04-02T14:45:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.681+0000 7fbbab400640 1 -- 172.21.15.62:0/1862821055 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbbc403f050 con 0x55f20ac51eb0 2024-04-02T14:45:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.681+0000 7fbbd37cb740 10 client.15996 fetch_fsmap learned FSMap version 30 2024-04-02T14:45:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.681+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1862821055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f20acd2230 con 0x55f20ac51eb0 2024-04-02T14:45:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.681+0000 7fbbab400640 1 -- 172.21.15.62:0/1862821055 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbbc4011d70 con 0x55f20ac51eb0 2024-04-02T14:45:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.682+0000 7fbbab400640 1 -- 172.21.15.62:0/1862821055 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbbc402a070 con 0x55f20ac51eb0 2024-04-02T14:45:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.682+0000 7fbbd37cb740 10 client.15996 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:45:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.682+0000 7fbbd37cb740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:45:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.682+0000 7fbbd37cb740 20 client.15996 populate_metadata read hostname 'smithi062' 2024-04-02T14:45:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.682+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f20acd26c0 0x55f20acd4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:45:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.682+0000 7fbbd37cb740 4 client.15996 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:45:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.682+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1862821055 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f20abc42b0 con 0x55f20acd26c0 2024-04-02T14:45:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.682+0000 7fbbc9400640 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f20acd26c0 0x55f20acd4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:45:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.683+0000 7fbbc9400640 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f20acd26c0 0x55f20acd4ae0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b752ddb8656c4a11 server_cookie=e6886a6271e2e694 in_seq=0 out_seq=0 2024-04-02T14:45:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:58.683+0000 7fbbab400640 10 client.15996 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbab400640 1 -- 172.21.15.62:0/1862821055 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f20abc42b0 con 0x55f20acd26c0 2024-04-02T14:45:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbab400640 10 client.15996 handle_command_reply: tid=0 2024-04-02T14:45:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbd37cb740 1 client.15996 shutdown 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1862821055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 msgr2=0x55f20ac55280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac55280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4005c40 tx=0x7fbbc4005bd0 comp rx=0 tx=0).stop 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1862821055 shutdown_connections 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f20acd26c0 0x55f20acd4ae0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f20ac5b3b0 0x55f20ac55d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f20ac527d0 0x55f20ac557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbd37cb740 1 --2- 172.21.15.62:0/1862821055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f20ac51eb0 0x55f20ac55280 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbab400640 0 client.15996 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1862821055 >> 172.21.15.62:0/1862821055 conn(0x55f20ac4c5e0 msgr2=0x55f20ac4d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.392+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1862821055 shutdown_connections 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd37cb740 1 -- 172.21.15.62:0/1862821055 wait complete. 2024-04-02T14:45:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd37cb740 20 client.15996 trim_cache size 0 max 16384 2024-04-02T14:45:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:45:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:45:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:45:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd0a00640 1 -- 172.21.15.62:0/4238550845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbbb00a3dc0 msgr2=0x7fbbb00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd0a00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbbb00a3dc0 0x7fbbb00a6280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55f20ac51980 tx=0x7fbbbc0f8530 comp rx=0 tx=0).stop 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd0a00640 1 -- 172.21.15.62:0/4238550845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbcc073f30 msgr2=0x7fbbcc19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd0a00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbcc073f30 0x7fbbcc19a640 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc000c00 tx=0x7fbbbc004a00 comp rx=0 tx=0).stop 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd0a00640 1 -- 172.21.15.62:0/4238550845 shutdown_connections 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd0a00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbbb00a3dc0 0x7fbbb00a6280 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd0a00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbcc10e2f0 0x7fbbcc19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd0a00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbcc074850 0x7fbbcc19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd0a00640 1 --2- 172.21.15.62:0/4238550845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbcc073f30 0x7fbbcc19a640 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.393+0000 7fbbd0a00640 1 -- 172.21.15.62:0/4238550845 >> 172.21.15.62:0/4238550845 conn(0x7fbbcc0fd580 msgr2=0x7fbbcc110570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.394+0000 7fbbd0a00640 1 -- 172.21.15.62:0/4238550845 shutdown_connections 2024-04-02T14:45:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:45:59.394+0000 7fbbd0a00640 1 -- 172.21.15.62:0/4238550845 wait complete. 2024-04-02T14:46:00.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:45:59 smithi204 ceph-mon[35948]: pgmap v62: 97 pgs: 97 active+clean; 92 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 5.2 MiB/s wr, 63 op/s 2024-04-02T14:46:00.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:45:59 smithi062 ceph-mon[32116]: pgmap v62: 97 pgs: 97 active+clean; 92 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 5.2 MiB/s wr, 63 op/s 2024-04-02T14:46:00.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:45:59 smithi113 ceph-mon[36135]: pgmap v62: 97 pgs: 97 active+clean; 92 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 5.2 MiB/s wr, 63 op/s 2024-04-02T14:46:01.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:00 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:46:01.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:00 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:46:01.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:00 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:46:02.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:01 smithi204 ceph-mon[35948]: pgmap v63: 97 pgs: 97 active+clean; 92 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 5.2 MiB/s wr, 63 op/s 2024-04-02T14:46:02.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:01 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:46:02.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:01 smithi062 ceph-mon[32116]: pgmap v63: 97 pgs: 97 active+clean; 92 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 5.2 MiB/s wr, 63 op/s 2024-04-02T14:46:02.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:01 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:46:02.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:01 smithi113 ceph-mon[36135]: pgmap v63: 97 pgs: 97 active+clean; 92 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 5.2 MiB/s wr, 63 op/s 2024-04-02T14:46:02.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:01 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:46:02.948 INFO:tasks.workunit.client.0.smithi062.stderr:+ tar cf doc.tar linux-4.0.5 2024-04-02T14:46:04.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:03 smithi204 ceph-mon[35948]: pgmap v64: 97 pgs: 97 active+clean; 86 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 5.6 MiB/s wr, 757 op/s 2024-04-02T14:46:04.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:03 smithi062 ceph-mon[32116]: pgmap v64: 97 pgs: 97 active+clean; 86 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 5.6 MiB/s wr, 757 op/s 2024-04-02T14:46:04.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:03 smithi113 ceph-mon[36135]: pgmap v64: 97 pgs: 97 active+clean; 86 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 5.6 MiB/s wr, 757 op/s 2024-04-02T14:46:04.721 INFO:tasks.workunit.client.0.smithi062.stderr:+ tar xf doc.tar 2024-04-02T14:46:05.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:04 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:05.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:04 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:05.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:04 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:06.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:05 smithi204 ceph-mon[35948]: pgmap v65: 97 pgs: 97 active+clean; 86 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.4 MiB/s wr, 748 op/s 2024-04-02T14:46:06.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:05 smithi062 ceph-mon[32116]: pgmap v65: 97 pgs: 97 active+clean; 86 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.4 MiB/s wr, 748 op/s 2024-04-02T14:46:06.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:05 smithi113 ceph-mon[36135]: pgmap v65: 97 pgs: 97 active+clean; 86 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.4 MiB/s wr, 748 op/s 2024-04-02T14:46:08.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:07 smithi204 ceph-mon[35948]: pgmap v66: 97 pgs: 97 active+clean; 88 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 3.7 MiB/s wr, 781 op/s 2024-04-02T14:46:08.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:07 smithi062 ceph-mon[32116]: pgmap v66: 97 pgs: 97 active+clean; 88 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 3.7 MiB/s wr, 781 op/s 2024-04-02T14:46:08.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:07 smithi113 ceph-mon[36135]: pgmap v66: 97 pgs: 97 active+clean; 88 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 3.7 MiB/s wr, 781 op/s 2024-04-02T14:46:10.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:09 smithi204 ceph-mon[35948]: pgmap v67: 97 pgs: 97 active+clean; 88 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 677 KiB/s wr, 727 op/s 2024-04-02T14:46:10.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:09 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:10.089 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:09 smithi062 ceph-mon[32116]: pgmap v67: 97 pgs: 97 active+clean; 88 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 677 KiB/s wr, 727 op/s 2024-04-02T14:46:10.089 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:09 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:10.118 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:09 smithi113 ceph-mon[36135]: pgmap v67: 97 pgs: 97 active+clean; 88 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 677 KiB/s wr, 727 op/s 2024-04-02T14:46:10.118 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:09 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:11.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:10 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:46:11.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:10 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:46:11.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:10 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:46:12.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:11 smithi204 ceph-mon[35948]: pgmap v68: 97 pgs: 97 active+clean; 88 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 677 KiB/s wr, 727 op/s 2024-04-02T14:46:12.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:11 smithi062 ceph-mon[32116]: pgmap v68: 97 pgs: 97 active+clean; 88 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 677 KiB/s wr, 727 op/s 2024-04-02T14:46:12.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:11 smithi113 ceph-mon[36135]: pgmap v68: 97 pgs: 97 active+clean; 88 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 677 KiB/s wr, 727 op/s 2024-04-02T14:46:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.540+0000 7f1a9aa00640 1 -- 172.21.15.62:0/1722916359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a94073eb0 msgr2=0x7f1a94074290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.540+0000 7f1a9aa00640 1 --2- 172.21.15.62:0/1722916359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a94073eb0 0x7f1a94074290 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009900 tx=0x7f1a8402bcd0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.540+0000 7f1a9aa00640 1 -- 172.21.15.62:0/1722916359 shutdown_connections 2024-04-02T14:46:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.540+0000 7f1a9aa00640 1 --2- 172.21.15.62:0/1722916359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a9410e370 0x7f1a94110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.540+0000 7f1a9aa00640 1 --2- 172.21.15.62:0/1722916359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a940747d0 0x7f1a9410de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.540+0000 7f1a9aa00640 1 --2- 172.21.15.62:0/1722916359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a94073eb0 0x7f1a94074290 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.540+0000 7f1a9aa00640 1 -- 172.21.15.62:0/1722916359 >> 172.21.15.62:0/1722916359 conn(0x7f1a940fd5c0 msgr2=0x7f1a940ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.541+0000 7f1a9aa00640 1 -- 172.21.15.62:0/1722916359 shutdown_connections 2024-04-02T14:46:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.541+0000 7f1a9aa00640 1 -- 172.21.15.62:0/1722916359 wait complete. 2024-04-02T14:46:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.541+0000 7f1a9aa00640 1 Processor -- start 2024-04-02T14:46:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a9aa00640 1 -- start start 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a9aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a94073eb0 0x7f1a9419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a9aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a940747d0 0x7f1a9419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a9aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a9410e370 0x7f1a9419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a9aa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94114200 con 0x7f1a94073eb0 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a9aa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94114080 con 0x7f1a9410e370 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a9aa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94114380 con 0x7f1a940747d0 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a9410e370 0x7f1a9419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a94073eb0 0x7f1a9419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a9410e370 0x7f1a9419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:53388/0 (socket says 172.21.15.62:53388) 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a98a00640 1 -- 172.21.15.62:0/3724637188 learned_addr learned my addr 172.21.15.62:0/3724637188 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.542+0000 7f1a93400640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a940747d0 0x7f1a9419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.543+0000 7f1a98a00640 1 -- 172.21.15.62:0/3724637188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a940747d0 msgr2=0x7f1a9419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.543+0000 7f1a98a00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a940747d0 0x7f1a9419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.543+0000 7f1a98a00640 1 -- 172.21.15.62:0/3724637188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a94073eb0 msgr2=0x7f1a9419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.543+0000 7f1a98a00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a94073eb0 0x7f1a9419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.543+0000 7f1a98a00640 1 -- 172.21.15.62:0/3724637188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9419f5e0 con 0x7f1a9410e370 2024-04-02T14:46:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.543+0000 7f1a93e00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a94073eb0 0x7f1a9419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.543+0000 7f1a98a00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a9410e370 0x7f1a9419eeb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009fd0 tx=0x7f1a8c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.543+0000 7f1a90c00640 1 -- 172.21.15.62:0/3724637188 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c017070 con 0x7f1a9410e370 2024-04-02T14:46:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.543+0000 7f1a9aa00640 1 -- 172.21.15.62:0/3724637188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9419f8d0 con 0x7f1a9410e370 2024-04-02T14:46:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.543+0000 7f1a90c00640 1 -- 172.21.15.62:0/3724637188 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1a8c004020 con 0x7f1a9410e370 2024-04-02T14:46:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.544+0000 7f1a90c00640 1 -- 172.21.15.62:0/3724637188 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0106c0 con 0x7f1a9410e370 2024-04-02T14:46:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.544+0000 7f1a9aa00640 1 -- 172.21.15.62:0/3724637188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a941a7160 con 0x7f1a9410e370 2024-04-02T14:46:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.547+0000 7f1a9d674740 1 Processor -- start 2024-04-02T14:46:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.549+0000 7f1a90c00640 1 -- 172.21.15.62:0/3724637188 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1a8c0041d0 con 0x7f1a9410e370 2024-04-02T14:46:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.549+0000 7f1a9d674740 1 -- start start 2024-04-02T14:46:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb32eb0 0x5578ddb33290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb337d0 0x5578ddb3be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb3e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a9d674740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5578dda97d70 con 0x5578ddb337d0 2024-04-02T14:46:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a9d674740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5578dda402a0 con 0x5578ddb32eb0 2024-04-02T14:46:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a9d674740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5578dda93d20 con 0x5578ddb3c3b0 2024-04-02T14:46:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a93400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb32eb0 0x5578ddb33290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb337d0 0x5578ddb3be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a93400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb32eb0 0x5578ddb33290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:53402/0 (socket says 172.21.15.62:53402) 2024-04-02T14:46:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a93400640 1 -- 172.21.15.62:0/509264628 learned_addr learned my addr 172.21.15.62:0/509264628 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a93400640 1 --2- 172.21.15.62:0/509264628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb3e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.550+0000 7f1a93400640 1 -- 172.21.15.62:0/509264628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 msgr2=0x5578ddb3e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.551+0000 7f1a93400640 1 --2- 172.21.15.62:0/509264628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb3e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.551+0000 7f1a93400640 1 -- 172.21.15.62:0/509264628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb337d0 msgr2=0x5578ddb3be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.551+0000 7f1a93400640 1 --2- 172.21.15.62:0/509264628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb337d0 0x5578ddb3be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.551+0000 7f1a93400640 1 -- 172.21.15.62:0/509264628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578dda54840 con 0x5578ddb32eb0 2024-04-02T14:46:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.551+0000 7f1a90c00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1a740a3e40 0x7f1a740a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.552+0000 7f1a93400640 1 --2- 172.21.15.62:0/509264628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb32eb0 0x5578ddb33290 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009f90 tx=0x7f1a7c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b135c245bc10621 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.552+0000 7f1a93e00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1a740a3e40 0x7f1a740a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.553+0000 7f1a90c00640 1 -- 172.21.15.62:0/3724637188 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f1a8c0d0380 con 0x7f1a9410e370 2024-04-02T14:46:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.553+0000 7f1a89200640 1 -- 172.21.15.62:0/509264628 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c004050 con 0x5578ddb32eb0 2024-04-02T14:46:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.554+0000 7f1a89200640 1 -- 172.21.15.62:0/509264628 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1a7c005520 con 0x5578ddb32eb0 2024-04-02T14:46:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.554+0000 7f1a89200640 1 -- 172.21.15.62:0/509264628 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0035f0 con 0x5578ddb32eb0 2024-04-02T14:46:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.555+0000 7f1a93e00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1a740a3e40 0x7f1a740a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1a84005ab0 tx=0x7f1a84005a20 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.557+0000 7f1a9d674740 1 -- 172.21.15.62:0/509264628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb32eb0 msgr2=0x5578ddb33290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.557+0000 7f1a9d674740 1 --2- 172.21.15.62:0/509264628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb32eb0 0x5578ddb33290 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009f90 tx=0x7f1a7c005850 comp rx=0 tx=0).stop 2024-04-02T14:46:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.560+0000 7f1a9d674740 1 -- 172.21.15.62:0/509264628 shutdown_connections 2024-04-02T14:46:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.560+0000 7f1a9d674740 1 --2- 172.21.15.62:0/509264628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb3e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.560+0000 7f1a9d674740 1 --2- 172.21.15.62:0/509264628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb337d0 0x5578ddb3be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.560+0000 7f1a9d674740 1 --2- 172.21.15.62:0/509264628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb32eb0 0x5578ddb33290 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.560+0000 7f1a9d674740 1 -- 172.21.15.62:0/509264628 >> 172.21.15.62:0/509264628 conn(0x5578ddb2d5e0 msgr2=0x5578ddb2d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.561+0000 7f1a9d674740 1 -- 172.21.15.62:0/509264628 shutdown_connections 2024-04-02T14:46:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.561+0000 7f1a9d674740 1 -- 172.21.15.62:0/509264628 wait complete. 2024-04-02T14:46:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.562+0000 7f1a9d674740 1 -- start start 2024-04-02T14:46:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.562+0000 7f1a9d674740 1 Processor -- start 2024-04-02T14:46:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.566+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb32eb0 0x5578ddb394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.566+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb337d0 0x5578ddb399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.566+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb3c3b0 0x5578ddb39f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.566+0000 7f1a9d674740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5578ddb3f1a0 con 0x5578ddb337d0 2024-04-02T14:46:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.566+0000 7f1a9d674740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5578dda53660 con 0x5578ddb3c3b0 2024-04-02T14:46:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.566+0000 7f1a9d674740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5578ddb3f320 con 0x5578ddb32eb0 2024-04-02T14:46:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.567+0000 7f1a9d674740 10 client.0 _ll_register_callbacks cb 0x5578dda53050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.567+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb3c3b0 0x5578ddb39f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.567+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb3c3b0 0x5578ddb39f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:53404/0 (socket says 172.21.15.62:53404) 2024-04-02T14:46:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.567+0000 7f1a98a00640 1 -- 172.21.15.62:0/1169191320 learned_addr learned my addr 172.21.15.62:0/1169191320 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.568+0000 7f1a93e00640 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb32eb0 0x5578ddb394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.568+0000 7f1a93400640 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb337d0 0x5578ddb399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.568+0000 7f1a98a00640 1 -- 172.21.15.62:0/1169191320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb32eb0 msgr2=0x5578ddb394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.568+0000 7f1a98a00640 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb32eb0 0x5578ddb394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.568+0000 7f1a98a00640 1 -- 172.21.15.62:0/1169191320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb337d0 msgr2=0x5578ddb399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.568+0000 7f1a98a00640 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb337d0 0x5578ddb399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.568+0000 7f1a98a00640 1 -- 172.21.15.62:0/1169191320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578dda62e80 con 0x5578ddb3c3b0 2024-04-02T14:46:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.568+0000 7f1a93400640 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb337d0 0x5578ddb399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:46:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.569+0000 7f1a98a00640 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb3c3b0 0x5578ddb39f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c098d90 tx=0x7f1a8c09c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432dd3b85e97a6b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.569+0000 7f1a81600640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.570+0000 7f1a81600640 1 -- 172.21.15.62:0/1169191320 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c020070 con 0x5578ddb3c3b0 2024-04-02T14:46:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.570+0000 7f1a81600640 1 -- 172.21.15.62:0/1169191320 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f1a8c098a30 con 0x5578ddb3c3b0 2024-04-02T14:46:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.571+0000 7f1a9d674740 1 -- 172.21.15.62:0/1169191320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a7c005520 con 0x5578ddb3c3b0 2024-04-02T14:46:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.571+0000 7f1a81600640 1 -- 172.21.15.62:0/1169191320 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a8c0a4f80 con 0x5578ddb3c3b0 2024-04-02T14:46:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.572+0000 7f1a9d674740 10 client.26132 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.572+0000 7f1a9d674740 1 -- 172.21.15.62:0/1169191320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5578dd7ac210 con 0x5578ddb3c3b0 2024-04-02T14:46:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.573+0000 7f1a81600640 1 -- 172.21.15.62:0/1169191320 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1a7c005520 con 0x5578ddb3c3b0 2024-04-02T14:46:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.573+0000 7f1a81600640 1 -- 172.21.15.62:0/1169191320 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f1a8c098110 con 0x5578ddb3c3b0 2024-04-02T14:46:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.574+0000 7f1a9d674740 10 client.26132 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.574+0000 7f1a9d674740 10 client.26132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.574+0000 7f1a9d674740 20 client.26132 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.574+0000 7f1a9d674740 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5578ddb5d8e0 0x5578ddb5fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.574+0000 7f1a9d674740 4 client.26132 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.574+0000 7f1a9d674740 1 -- 172.21.15.62:0/1169191320 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578dda47cd0 con 0x5578ddb5d8e0 2024-04-02T14:46:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.575+0000 7f1a93e00640 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5578ddb5d8e0 0x5578ddb5fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.578+0000 7f1a93e00640 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5578ddb5d8e0 0x5578ddb5fd30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd1f7ee0890b01b3 server_cookie=ce10640b73f231af in_seq=0 out_seq=0 2024-04-02T14:46:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.578+0000 7f1a81600640 10 client.26132 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.580+0000 7f1a81600640 1 -- 172.21.15.62:0/1169191320 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578dda47cd0 con 0x5578ddb5d8e0 2024-04-02T14:46:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.580+0000 7f1a81600640 10 client.26132 handle_command_reply: tid=0 2024-04-02T14:46:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.581+0000 7f1a9d674740 1 client.26132 shutdown 2024-04-02T14:46:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.581+0000 7f1a9d674740 1 -- 172.21.15.62:0/1169191320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb3c3b0 msgr2=0x5578ddb39f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.581+0000 7f1a9d674740 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb3c3b0 0x5578ddb39f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c098d90 tx=0x7f1a8c09c3c0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.582+0000 7f1a9d674740 1 -- 172.21.15.62:0/1169191320 shutdown_connections 2024-04-02T14:46:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.582+0000 7f1a9d674740 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5578ddb5d8e0 0x5578ddb5fd30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.582+0000 7f1a81600640 0 client.26132 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.582+0000 7f1a9d674740 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb3c3b0 0x5578ddb39f30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.582+0000 7f1a9d674740 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb337d0 0x5578ddb399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.582+0000 7f1a9d674740 1 --2- 172.21.15.62:0/1169191320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb32eb0 0x5578ddb394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.582+0000 7f1a9d674740 1 -- 172.21.15.62:0/1169191320 >> 172.21.15.62:0/1169191320 conn(0x5578ddb2d5e0 msgr2=0x5578ddb2f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.582+0000 7f1a9d674740 1 -- 172.21.15.62:0/1169191320 shutdown_connections 2024-04-02T14:46:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.582+0000 7f1a9d674740 1 -- 172.21.15.62:0/1169191320 wait complete. 2024-04-02T14:46:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.583+0000 7f1a9d674740 20 client.26132 trim_cache size 0 max 16384 2024-04-02T14:46:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.632+0000 7f1a9d674740 1 Processor -- start 2024-04-02T14:46:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.632+0000 7f1a9d674740 1 -- start start 2024-04-02T14:46:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.633+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb32eb0 0x5578ddb3bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.633+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 0x5578ddb48060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.633+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb38630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.634+0000 7f1a9d674740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941a7330 con 0x5578ddb32eb0 2024-04-02T14:46:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.634+0000 7f1a9d674740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941a8f10 con 0x5578ddb337d0 2024-04-02T14:46:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.634+0000 7f1a9d674740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941a7980 con 0x5578ddb3c3b0 2024-04-02T14:46:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.634+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb38630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.634+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb32eb0 0x5578ddb3bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.634+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb38630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:51286/0 (socket says 172.21.15.62:51286) 2024-04-02T14:46:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.634+0000 7f1a98a00640 1 -- 172.21.15.62:0/1025985409 learned_addr learned my addr 172.21.15.62:0/1025985409 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.635+0000 7f1a93400640 1 --2- 172.21.15.62:0/1025985409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 0x5578ddb48060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.635+0000 7f1a98a00640 1 -- 172.21.15.62:0/1025985409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 msgr2=0x5578ddb48060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.635+0000 7f1a98a00640 1 --2- 172.21.15.62:0/1025985409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 0x5578ddb48060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.635+0000 7f1a98a00640 1 -- 172.21.15.62:0/1025985409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb32eb0 msgr2=0x5578ddb3bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.635+0000 7f1a98a00640 1 --2- 172.21.15.62:0/1025985409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb32eb0 0x5578ddb3bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.635+0000 7f1a98a00640 1 -- 172.21.15.62:0/1025985409 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578dd878620 con 0x5578ddb3c3b0 2024-04-02T14:46:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.636+0000 7f1a98a00640 1 --2- 172.21.15.62:0/1025985409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb38630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c09c0c0 tx=0x7f1a8c004800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df2b6c92bb2010e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.637+0000 7f1a83e00640 1 -- 172.21.15.62:0/1025985409 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0a4f80 con 0x5578ddb3c3b0 2024-04-02T14:46:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.637+0000 7f1a83e00640 1 -- 172.21.15.62:0/1025985409 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1a8c098a30 con 0x5578ddb3c3b0 2024-04-02T14:46:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.637+0000 7f1a83e00640 1 -- 172.21.15.62:0/1025985409 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c09ba70 con 0x5578ddb3c3b0 2024-04-02T14:46:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.638+0000 7f1a9d674740 1 -- 172.21.15.62:0/1025985409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 msgr2=0x5578ddb38630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.638+0000 7f1a9d674740 1 --2- 172.21.15.62:0/1025985409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb38630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c09c0c0 tx=0x7f1a8c004800 comp rx=0 tx=0).stop 2024-04-02T14:46:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.640+0000 7f1a9d674740 1 -- 172.21.15.62:0/1025985409 shutdown_connections 2024-04-02T14:46:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.640+0000 7f1a9d674740 1 --2- 172.21.15.62:0/1025985409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb38630 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.640+0000 7f1a9d674740 1 --2- 172.21.15.62:0/1025985409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 0x5578ddb48060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.640+0000 7f1a9d674740 1 --2- 172.21.15.62:0/1025985409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb32eb0 0x5578ddb3bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.640+0000 7f1a9d674740 1 -- 172.21.15.62:0/1025985409 >> 172.21.15.62:0/1025985409 conn(0x5578ddb2d5e0 msgr2=0x5578ddb2bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.642+0000 7f1a9d674740 1 -- 172.21.15.62:0/1025985409 shutdown_connections 2024-04-02T14:46:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.642+0000 7f1a9d674740 1 -- 172.21.15.62:0/1025985409 wait complete. 2024-04-02T14:46:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.642+0000 7f1a9d674740 1 -- start start 2024-04-02T14:46:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.643+0000 7f1a9d674740 1 Processor -- start 2024-04-02T14:46:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.644+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb32eb0 0x5578ddb44760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.645+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 0x5578ddb35d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.645+0000 7f1a9d674740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb36240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.645+0000 7f1a9d674740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941a8780 con 0x5578ddb32eb0 2024-04-02T14:46:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.645+0000 7f1a9d674740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941a8050 con 0x5578ddb337d0 2024-04-02T14:46:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.645+0000 7f1a9d674740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9419b940 con 0x5578ddb3c3b0 2024-04-02T14:46:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.645+0000 7f1a9d674740 10 client.0 _ll_register_callbacks cb 0x7f1a8c09bd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.646+0000 7f1a93400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 0x5578ddb35d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.646+0000 7f1a93400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 0x5578ddb35d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:53430/0 (socket says 172.21.15.62:53430) 2024-04-02T14:46:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.646+0000 7f1a93400640 1 -- 172.21.15.62:0/50760719 learned_addr learned my addr 172.21.15.62:0/50760719 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.646+0000 7f1a93e00640 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb32eb0 0x5578ddb44760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.646+0000 7f1a98a00640 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb36240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.647+0000 7f1a93e00640 1 -- 172.21.15.62:0/50760719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 msgr2=0x5578ddb36240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.647+0000 7f1a93e00640 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb36240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.647+0000 7f1a93e00640 1 -- 172.21.15.62:0/50760719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 msgr2=0x5578ddb35d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.647+0000 7f1a93e00640 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 0x5578ddb35d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.647+0000 7f1a93e00640 1 -- 172.21.15.62:0/50760719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ddb36780 con 0x5578ddb32eb0 2024-04-02T14:46:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.647+0000 7f1a98a00640 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb36240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.647+0000 7f1a93400640 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 0x5578ddb35d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.648+0000 7f1a93e00640 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb32eb0 0x5578ddb44760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1a8402c210 tx=0x7f1a8402bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca981d3d66f24c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.648+0000 7f1a81600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:46:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.649+0000 7f1a81600640 1 -- 172.21.15.62:0/50760719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84044040 con 0x5578ddb32eb0 2024-04-02T14:46:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.649+0000 7f1a9d674740 1 -- 172.21.15.62:0/50760719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a8c098a30 con 0x5578ddb32eb0 2024-04-02T14:46:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.650+0000 7f1a81600640 1 -- 172.21.15.62:0/50760719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f1a8403a040 con 0x5578ddb32eb0 2024-04-02T14:46:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.650+0000 7f1a81600640 1 -- 172.21.15.62:0/50760719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a84063050 con 0x5578ddb32eb0 2024-04-02T14:46:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.650+0000 7f1a9d674740 10 client.16011 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.651+0000 7f1a9d674740 1 -- 172.21.15.62:0/50760719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5578ddbc3b40 con 0x5578ddb32eb0 2024-04-02T14:46:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.652+0000 7f1a81600640 1 -- 172.21.15.62:0/50760719 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1a8c098a30 con 0x5578ddb32eb0 2024-04-02T14:46:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.653+0000 7f1a81600640 1 -- 172.21.15.62:0/50760719 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f1a840037b0 con 0x5578ddb32eb0 2024-04-02T14:46:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.655+0000 7f1a9d674740 10 client.16011 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.655+0000 7f1a9d674740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.655+0000 7f1a9d674740 20 client.16011 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.655+0000 7f1a9d674740 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5578ddbc4300 0x5578ddbc6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.655+0000 7f1a9d674740 4 client.16011 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:46:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.655+0000 7f1a9d674740 1 -- 172.21.15.62:0/50760719 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578ddaa52b0 con 0x5578ddbc4300 2024-04-02T14:46:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.655+0000 7f1a93400640 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5578ddbc4300 0x5578ddbc6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.658+0000 7f1a81600640 10 client.16011 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:12.658+0000 7f1a93400640 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5578ddbc4300 0x5578ddbc6740 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da02f6688908dc3 server_cookie=d2cbdab6bfdc469d in_seq=0 out_seq=0 2024-04-02T14:46:13.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:12 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:46:13.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:12 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:46:13.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:12 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:46:13.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:12 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:46:13.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:12 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:46:13.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:12 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:46:13.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:12 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:46:13.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:12 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:46:13.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:12 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:46:14.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:13 smithi204 ceph-mon[35948]: pgmap v69: 97 pgs: 97 active+clean; 108 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 889 op/s 2024-04-02T14:46:14.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:13 smithi062 ceph-mon[32116]: pgmap v69: 97 pgs: 97 active+clean; 108 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 889 op/s 2024-04-02T14:46:14.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:13 smithi113 ceph-mon[36135]: pgmap v69: 97 pgs: 97 active+clean; 108 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 889 op/s 2024-04-02T14:46:16.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:15 smithi204 ceph-mon[35948]: pgmap v70: 97 pgs: 97 active+clean; 108 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-04-02T14:46:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.040+0000 7f1a81600640 1 -- 172.21.15.62:0/50760719 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5578ddaa52b0 con 0x5578ddbc4300 2024-04-02T14:46:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.040+0000 7f1a81600640 10 client.16011 handle_command_reply: tid=0 2024-04-02T14:46:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.040+0000 7f1a9d674740 1 client.16011 shutdown 2024-04-02T14:46:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.042+0000 7f1a9d674740 1 -- 172.21.15.62:0/50760719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb32eb0 msgr2=0x5578ddb44760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.042+0000 7f1a9d674740 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb32eb0 0x5578ddb44760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1a8402c210 tx=0x7f1a8402bcd0 comp rx=0 tx=0).stop 2024-04-02T14:46:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.042+0000 7f1a9d674740 1 -- 172.21.15.62:0/50760719 shutdown_connections 2024-04-02T14:46:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.042+0000 7f1a9d674740 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5578ddbc4300 0x5578ddbc6740 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.042+0000 7f1a9d674740 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578ddb3c3b0 0x5578ddb36240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.042+0000 7f1a9d674740 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578ddb337d0 0x5578ddb35d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.042+0000 7f1a9d674740 1 --2- 172.21.15.62:0/50760719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578ddb32eb0 0x5578ddb44760 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.042+0000 7f1a9d674740 1 -- 172.21.15.62:0/50760719 >> 172.21.15.62:0/50760719 conn(0x5578ddb2d5e0 msgr2=0x5578ddb2f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.043+0000 7f1a81600640 0 client.16011 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.043+0000 7f1a9d674740 1 -- 172.21.15.62:0/50760719 shutdown_connections 2024-04-02T14:46:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.043+0000 7f1a9d674740 1 -- 172.21.15.62:0/50760719 wait complete. 2024-04-02T14:46:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.043+0000 7f1a9d674740 20 client.16011 trim_cache size 0 max 16384 2024-04-02T14:46:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:46:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:46:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:46:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:46:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.044+0000 7f1a9aa00640 1 -- 172.21.15.62:0/3724637188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1a740a3e40 msgr2=0x7f1a740a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.044+0000 7f1a9aa00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1a740a3e40 0x7f1a740a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1a84005ab0 tx=0x7f1a84005a20 comp rx=0 tx=0).stop 2024-04-02T14:46:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.044+0000 7f1a9aa00640 1 -- 172.21.15.62:0/3724637188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a9410e370 msgr2=0x7f1a9419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.044+0000 7f1a9aa00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a9410e370 0x7f1a9419eeb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009fd0 tx=0x7f1a8c00b260 comp rx=0 tx=0).stop 2024-04-02T14:46:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.045+0000 7f1a9aa00640 1 -- 172.21.15.62:0/3724637188 shutdown_connections 2024-04-02T14:46:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.045+0000 7f1a9aa00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1a740a3e40 0x7f1a740a6300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.045+0000 7f1a9aa00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a9410e370 0x7f1a9419eeb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.045+0000 7f1a9aa00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a940747d0 0x7f1a9419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.045+0000 7f1a9aa00640 1 --2- 172.21.15.62:0/3724637188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a94073eb0 0x7f1a9419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.045+0000 7f1a9aa00640 1 -- 172.21.15.62:0/3724637188 >> 172.21.15.62:0/3724637188 conn(0x7f1a940fd5c0 msgr2=0x7f1a940fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.045+0000 7f1a9aa00640 1 -- 172.21.15.62:0/3724637188 shutdown_connections 2024-04-02T14:46:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:16.045+0000 7f1a9aa00640 1 -- 172.21.15.62:0/3724637188 wait complete. 2024-04-02T14:46:16.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:15 smithi062 ceph-mon[32116]: pgmap v70: 97 pgs: 97 active+clean; 108 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-04-02T14:46:16.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:15 smithi113 ceph-mon[36135]: pgmap v70: 97 pgs: 97 active+clean; 108 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-04-02T14:46:17.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:16 smithi062 ceph-mon[32116]: pgmap v71: 97 pgs: 97 active+clean; 137 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 769 op/s 2024-04-02T14:46:17.115 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:16 smithi113 ceph-mon[36135]: pgmap v71: 97 pgs: 97 active+clean; 137 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 769 op/s 2024-04-02T14:46:17.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:16 smithi204 ceph-mon[35948]: pgmap v71: 97 pgs: 97 active+clean; 137 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 769 op/s 2024-04-02T14:46:19.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:19 smithi204 ceph-mon[35948]: pgmap v72: 97 pgs: 97 active+clean; 137 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 735 op/s 2024-04-02T14:46:19.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:19 smithi062 ceph-mon[32116]: pgmap v72: 97 pgs: 97 active+clean; 137 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 735 op/s 2024-04-02T14:46:19.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:19 smithi113 ceph-mon[36135]: pgmap v72: 97 pgs: 97 active+clean; 137 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 735 op/s 2024-04-02T14:46:21.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:21 smithi204 ceph-mon[35948]: pgmap v73: 97 pgs: 97 active+clean; 137 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 735 op/s 2024-04-02T14:46:21.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:21 smithi062 ceph-mon[32116]: pgmap v73: 97 pgs: 97 active+clean; 137 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 735 op/s 2024-04-02T14:46:21.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:21 smithi113 ceph-mon[36135]: pgmap v73: 97 pgs: 97 active+clean; 137 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 735 op/s 2024-04-02T14:46:22.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:46:22.792 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:46:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.038+0000 7f8764c00640 1 -- 172.21.15.62:0/999281681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760107980 msgr2=0x7f8760107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.038+0000 7f8764c00640 1 --2- 172.21.15.62:0/999281681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760107980 0x7f8760107d60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8750009930 tx=0x7f875002bc20 comp rx=0 tx=0).stop 2024-04-02T14:46:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.038+0000 7f8764c00640 1 -- 172.21.15.62:0/999281681 shutdown_connections 2024-04-02T14:46:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.038+0000 7f8764c00640 1 --2- 172.21.15.62:0/999281681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8760102330 0x7f876010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.038+0000 7f8764c00640 1 --2- 172.21.15.62:0/999281681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8760101970 0x7f8760101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.038+0000 7f8764c00640 1 --2- 172.21.15.62:0/999281681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760107980 0x7f8760107d60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.038+0000 7f8764c00640 1 -- 172.21.15.62:0/999281681 >> 172.21.15.62:0/999281681 conn(0x7f87600fd4a0 msgr2=0x7f87600ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.038+0000 7f8764c00640 1 -- 172.21.15.62:0/999281681 shutdown_connections 2024-04-02T14:46:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.038+0000 7f8764c00640 1 -- 172.21.15.62:0/999281681 wait complete. 2024-04-02T14:46:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.039+0000 7f8764c00640 1 Processor -- start 2024-04-02T14:46:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.039+0000 7f8764c00640 1 -- start start 2024-04-02T14:46:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.039+0000 7f8764c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8760101970 0x7f8760198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.039+0000 7f8764c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760102330 0x7f8760198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.039+0000 7f8764c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8760107980 0x7f876019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.039+0000 7f8764c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8760111e80 con 0x7f8760107980 2024-04-02T14:46:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.039+0000 7f8764c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8760111d00 con 0x7f8760101970 2024-04-02T14:46:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.039+0000 7f8764c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8760112000 con 0x7f8760102330 2024-04-02T14:46:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.040+0000 7f875f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8760101970 0x7f8760198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.040+0000 7f875f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8760101970 0x7f8760198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37542/0 (socket says 172.21.15.62:37542) 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.040+0000 7f875f400640 1 -- 172.21.15.62:0/52374484 learned_addr learned my addr 172.21.15.62:0/52374484 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.040+0000 7f875ea00640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760102330 0x7f8760198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.040+0000 7f875fe00640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8760107980 0x7f876019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.040+0000 7f875f400640 1 -- 172.21.15.62:0/52374484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760102330 msgr2=0x7f8760198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.040+0000 7f875f400640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760102330 0x7f8760198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.040+0000 7f875f400640 1 -- 172.21.15.62:0/52374484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8760107980 msgr2=0x7f876019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.040+0000 7f875f400640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8760107980 0x7f876019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.040+0000 7f875f400640 1 -- 172.21.15.62:0/52374484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876019d330 con 0x7f8760101970 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.041+0000 7f875f400640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8760101970 0x7f8760198340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8750009900 tx=0x7f8750004c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.041+0000 7f8757e00640 1 -- 172.21.15.62:0/52374484 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750043020 con 0x7f8760101970 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.041+0000 7f8757e00640 1 -- 172.21.15.62:0/52374484 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8750004e50 con 0x7f8760101970 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.041+0000 7f8764c00640 1 -- 172.21.15.62:0/52374484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876019d5c0 con 0x7f8760101970 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.041+0000 7f8757e00640 1 -- 172.21.15.62:0/52374484 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750003950 con 0x7f8760101970 2024-04-02T14:46:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.041+0000 7f8764c00640 1 -- 172.21.15.62:0/52374484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87601a4eb0 con 0x7f8760101970 2024-04-02T14:46:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.044+0000 7f8757e00640 1 -- 172.21.15.62:0/52374484 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8750046020 con 0x7f8760101970 2024-04-02T14:46:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.044+0000 7f8768fe8740 1 Processor -- start 2024-04-02T14:46:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f8768fe8740 1 -- start start 2024-04-02T14:46:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92d95b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92d96120 0x559d92da2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92da5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f8768fe8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559d92cf9770 con 0x559d92d95770 2024-04-02T14:46:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f8768fe8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559d92ca1700 con 0x559d92da3270 2024-04-02T14:46:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f8768fe8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559d92cf5720 con 0x559d92d96120 2024-04-02T14:46:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f875fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92d95b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f875ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92da5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f875f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92d96120 0x559d92da2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f875fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92d95b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33830/0 (socket says 172.21.15.62:33830) 2024-04-02T14:46:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f875ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92da5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37558/0 (socket says 172.21.15.62:37558) 2024-04-02T14:46:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.045+0000 7f875fe00640 1 -- 172.21.15.62:0/1196532389 learned_addr learned my addr 172.21.15.62:0/1196532389 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f875ea00640 1 -- 172.21.15.62:0/1196532389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92d96120 msgr2=0x559d92da2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f875ea00640 1 --2- 172.21.15.62:0/1196532389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92d96120 0x559d92da2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f875ea00640 1 -- 172.21.15.62:0/1196532389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 msgr2=0x559d92d95b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f875ea00640 1 --2- 172.21.15.62:0/1196532389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92d95b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f875ea00640 1 -- 172.21.15.62:0/1196532389 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d92ad77d0 con 0x559d92da3270 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f875fe00640 1 --2- 172.21.15.62:0/1196532389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92d95b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f875ea00640 1 --2- 172.21.15.62:0/1196532389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92da5660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f874c009c20 tx=0x7f874c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da809826770b1f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f8757e00640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87440a3d20 0x7f87440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f874be00640 1 -- 172.21.15.62:0/1196532389 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c004050 con 0x559d92da3270 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f874be00640 1 -- 172.21.15.62:0/1196532389 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f874c005520 con 0x559d92da3270 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f874be00640 1 -- 172.21.15.62:0/1196532389 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c0035f0 con 0x559d92da3270 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.046+0000 7f8757e00640 1 -- 172.21.15.62:0/52374484 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f87500f2660 con 0x7f8760101970 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.047+0000 7f875fe00640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87440a3d20 0x7f87440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.047+0000 7f8768fe8740 1 -- 172.21.15.62:0/1196532389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 msgr2=0x559d92da5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.047+0000 7f8768fe8740 1 --2- 172.21.15.62:0/1196532389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92da5660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f874c009c20 tx=0x7f874c011810 comp rx=0 tx=0).stop 2024-04-02T14:46:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.047+0000 7f8768fe8740 1 -- 172.21.15.62:0/1196532389 shutdown_connections 2024-04-02T14:46:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.047+0000 7f8768fe8740 1 --2- 172.21.15.62:0/1196532389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92da5660 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.047+0000 7f8768fe8740 1 --2- 172.21.15.62:0/1196532389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92d96120 0x559d92da2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.047+0000 7f8768fe8740 1 --2- 172.21.15.62:0/1196532389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92d95b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.047+0000 7f875fe00640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87440a3d20 0x7f87440a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f87580117b0 tx=0x7f8758009500 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.047+0000 7f8768fe8740 1 -- 172.21.15.62:0/1196532389 >> 172.21.15.62:0/1196532389 conn(0x559d92d90090 msgr2=0x559d92d90470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.047+0000 7f8768fe8740 1 -- 172.21.15.62:0/1196532389 shutdown_connections 2024-04-02T14:46:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.048+0000 7f8768fe8740 1 -- 172.21.15.62:0/1196532389 wait complete. 2024-04-02T14:46:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.048+0000 7f8768fe8740 1 -- start start 2024-04-02T14:46:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.048+0000 7f8768fe8740 1 Processor -- start 2024-04-02T14:46:23.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92da0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 0x559d92da07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92da3270 0x559d92da0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f8768fe8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559d92da6030 con 0x559d92d95770 2024-04-02T14:46:23.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f8768fe8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559d92cb4ac0 con 0x559d92d96120 2024-04-02T14:46:23.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f8768fe8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559d92da61b0 con 0x559d92da3270 2024-04-02T14:46:23.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f8768fe8740 10 client.0 _ll_register_callbacks cb 0x559d92cb7180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f875f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92da0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f875fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92da3270 0x559d92da0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f875f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92da0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33834/0 (socket says 172.21.15.62:33834) 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f875f400640 1 -- 172.21.15.62:0/3856724279 learned_addr learned my addr 172.21.15.62:0/3856724279 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f875ea00640 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 0x559d92da07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f875fe00640 1 -- 172.21.15.62:0/3856724279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 msgr2=0x559d92da07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f875fe00640 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 0x559d92da07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.049+0000 7f875fe00640 1 -- 172.21.15.62:0/3856724279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 msgr2=0x559d92da0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.050+0000 7f875fe00640 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92da0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.050+0000 7f875fe00640 1 -- 172.21.15.62:0/3856724279 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d92a0d3f0 con 0x559d92da3270 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.050+0000 7f875ea00640 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 0x559d92da07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.050+0000 7f875fe00640 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92da3270 0x559d92da0d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f87580107a0 tx=0x7f8758004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb6a0b707085b40b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.050+0000 7f8748c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.050+0000 7f8748c00640 1 -- 172.21.15.62:0/3856724279 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758009ad0 con 0x559d92da3270 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.050+0000 7f8768fe8740 1 -- 172.21.15.62:0/3856724279 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f874c005520 con 0x559d92da3270 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.050+0000 7f8748c00640 1 -- 172.21.15.62:0/3856724279 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8758017030 con 0x559d92da3270 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.050+0000 7f8748c00640 1 -- 172.21.15.62:0/3856724279 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8758004b20 con 0x559d92da3270 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.050+0000 7f8768fe8740 10 client.26119 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.050+0000 7f8768fe8740 1 -- 172.21.15.62:0/3856724279 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559d929cd0f0 con 0x559d92da3270 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.051+0000 7f8748c00640 1 -- 172.21.15.62:0/3856724279 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f874c005520 con 0x559d92da3270 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.051+0000 7f8748c00640 1 -- 172.21.15.62:0/3856724279 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f8758027410 con 0x559d92da3270 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.051+0000 7f8768fe8740 10 client.26119 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.051+0000 7f8768fe8740 10 client.26119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.051+0000 7f8768fe8740 20 client.26119 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.051+0000 7f8768fe8740 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x559d92db33f0 0x559d92db5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.051+0000 7f8768fe8740 4 client.26119 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.051+0000 7f8768fe8740 1 -- 172.21.15.62:0/3856724279 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d92c9a630 con 0x559d92db33f0 2024-04-02T14:46:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.051+0000 7f875f400640 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x559d92db33f0 0x559d92db5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.052+0000 7f875f400640 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x559d92db33f0 0x559d92db5870 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220d7e923fabf26e server_cookie=616e198100e20e79 in_seq=0 out_seq=0 2024-04-02T14:46:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.052+0000 7f8748c00640 10 client.26119 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.054+0000 7f8748c00640 1 -- 172.21.15.62:0/3856724279 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d92c9a630 con 0x559d92db33f0 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.054+0000 7f8748c00640 10 client.26119 handle_command_reply: tid=0 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.054+0000 7f8768fe8740 1 client.26119 shutdown 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8768fe8740 1 -- 172.21.15.62:0/3856724279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92da3270 msgr2=0x559d92da0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8768fe8740 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92da3270 0x559d92da0d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f87580107a0 tx=0x7f8758004ab0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8768fe8740 1 -- 172.21.15.62:0/3856724279 shutdown_connections 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8768fe8740 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x559d92db33f0 0x559d92db5870 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8768fe8740 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92da3270 0x559d92da0d10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8768fe8740 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 0x559d92da07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8768fe8740 1 --2- 172.21.15.62:0/3856724279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92da0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8748c00640 0 client.26119 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8768fe8740 1 -- 172.21.15.62:0/3856724279 >> 172.21.15.62:0/3856724279 conn(0x559d92d90090 msgr2=0x559d92d91de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8768fe8740 1 -- 172.21.15.62:0/3856724279 shutdown_connections 2024-04-02T14:46:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8768fe8740 1 -- 172.21.15.62:0/3856724279 wait complete. 2024-04-02T14:46:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.055+0000 7f8768fe8740 20 client.26119 trim_cache size 0 max 16384 2024-04-02T14:46:23.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.077+0000 7f8768fe8740 1 Processor -- start 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.077+0000 7f8768fe8740 1 -- start start 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.077+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92d95770 0x559d92da29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.077+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d96120 0x559d92daef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.077+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92d90470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.077+0000 7f8768fe8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f87601ab320 con 0x559d92d96120 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.077+0000 7f8768fe8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f87601acf80 con 0x559d92da3270 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.077+0000 7f8768fe8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87601ab9f0 con 0x559d92d95770 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.078+0000 7f875fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92d90470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.078+0000 7f875ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d96120 0x559d92daef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.078+0000 7f875fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92d90470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37570/0 (socket says 172.21.15.62:37570) 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.078+0000 7f875fe00640 1 -- 172.21.15.62:0/1874168500 learned_addr learned my addr 172.21.15.62:0/1874168500 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.078+0000 7f875f400640 1 --2- 172.21.15.62:0/1874168500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92d95770 0x559d92da29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.078+0000 7f875fe00640 1 -- 172.21.15.62:0/1874168500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92d95770 msgr2=0x559d92da29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.078+0000 7f875fe00640 1 --2- 172.21.15.62:0/1874168500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92d95770 0x559d92da29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.078+0000 7f875fe00640 1 -- 172.21.15.62:0/1874168500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d96120 msgr2=0x559d92daef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.078+0000 7f875fe00640 1 --2- 172.21.15.62:0/1874168500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d96120 0x559d92daef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.078+0000 7f875fe00640 1 -- 172.21.15.62:0/1874168500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d92d9b040 con 0x559d92da3270 2024-04-02T14:46:23.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.078+0000 7f875fe00640 1 --2- 172.21.15.62:0/1874168500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92d90470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8758000f80 tx=0x7f87580108a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c6c1963883da011 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.079+0000 7f874b400640 1 -- 172.21.15.62:0/1874168500 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758010cf0 con 0x559d92da3270 2024-04-02T14:46:23.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.079+0000 7f874b400640 1 -- 172.21.15.62:0/1874168500 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8758005bd0 con 0x559d92da3270 2024-04-02T14:46:23.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.079+0000 7f874b400640 1 -- 172.21.15.62:0/1874168500 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758030d10 con 0x559d92da3270 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.079+0000 7f8768fe8740 1 -- 172.21.15.62:0/1874168500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 msgr2=0x559d92d90470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.079+0000 7f8768fe8740 1 --2- 172.21.15.62:0/1874168500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92d90470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8758000f80 tx=0x7f87580108a0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.079+0000 7f8768fe8740 1 -- 172.21.15.62:0/1874168500 shutdown_connections 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.079+0000 7f8768fe8740 1 --2- 172.21.15.62:0/1874168500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92da3270 0x559d92d90470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.079+0000 7f8768fe8740 1 --2- 172.21.15.62:0/1874168500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d96120 0x559d92daef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.080+0000 7f8768fe8740 1 --2- 172.21.15.62:0/1874168500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92d95770 0x559d92da29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.080+0000 7f8768fe8740 1 -- 172.21.15.62:0/1874168500 >> 172.21.15.62:0/1874168500 conn(0x559d92d90090 msgr2=0x559d92d8f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.080+0000 7f8768fe8740 1 -- 172.21.15.62:0/1874168500 shutdown_connections 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.080+0000 7f8768fe8740 1 -- 172.21.15.62:0/1874168500 wait complete. 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.080+0000 7f8768fe8740 1 -- start start 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.080+0000 7f8768fe8740 1 Processor -- start 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92da6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 0x559d92da6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f8768fe8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92da3270 0x559d92e119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f8768fe8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f87601ac7f0 con 0x559d92d95770 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f8768fe8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f87601ac0c0 con 0x559d92d96120 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f8768fe8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87601a9300 con 0x559d92da3270 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f8768fe8740 10 client.0 _ll_register_callbacks cb 0x7f8758011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f875f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92da6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f875ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 0x559d92da6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f875f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92da6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33852/0 (socket says 172.21.15.62:33852) 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f875f400640 1 -- 172.21.15.62:0/426061859 learned_addr learned my addr 172.21.15.62:0/426061859 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f875fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92da3270 0x559d92e119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f875ea00640 1 -- 172.21.15.62:0/426061859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92da3270 msgr2=0x559d92e119e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f875ea00640 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92da3270 0x559d92e119e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f875ea00640 1 -- 172.21.15.62:0/426061859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 msgr2=0x559d92da6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f875ea00640 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92da6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f875ea00640 1 -- 172.21.15.62:0/426061859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d92e11fd0 con 0x559d92d96120 2024-04-02T14:46:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.081+0000 7f875f400640 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92da6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.082+0000 7f875ea00640 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 0x559d92da6ca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f874c01b040 tx=0x7f874c009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ded1a76db15e1ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.082+0000 7f8748c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.082+0000 7f8748c00640 1 -- 172.21.15.62:0/426061859 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c003a00 con 0x559d92d96120 2024-04-02T14:46:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.082+0000 7f8748c00640 1 -- 172.21.15.62:0/426061859 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f874c003ba0 con 0x559d92d96120 2024-04-02T14:46:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.082+0000 7f8768fe8740 1 -- 172.21.15.62:0/426061859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8758005bd0 con 0x559d92d96120 2024-04-02T14:46:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.082+0000 7f8748c00640 1 -- 172.21.15.62:0/426061859 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f874c003da0 con 0x559d92d96120 2024-04-02T14:46:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.082+0000 7f8768fe8740 10 client.26156 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.082+0000 7f8768fe8740 1 -- 172.21.15.62:0/426061859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559d92e19380 con 0x559d92d96120 2024-04-02T14:46:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.083+0000 7f8748c00640 1 -- 172.21.15.62:0/426061859 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8758005bd0 con 0x559d92d96120 2024-04-02T14:46:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.083+0000 7f8748c00640 1 -- 172.21.15.62:0/426061859 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f874c0364c0 con 0x559d92d96120 2024-04-02T14:46:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.083+0000 7f8768fe8740 10 client.26156 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.083+0000 7f8768fe8740 10 client.26156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.083+0000 7f8768fe8740 20 client.26156 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:23.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.083+0000 7f8768fe8740 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x559d92e198e0 0x559d92e1bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.083+0000 7f8768fe8740 4 client.26156 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:46:23.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.083+0000 7f8768fe8740 1 -- 172.21.15.62:0/426061859 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d92d06cb0 con 0x559d92e198e0 2024-04-02T14:46:23.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.083+0000 7f875f400640 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x559d92e198e0 0x559d92e1bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.084+0000 7f875f400640 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x559d92e198e0 0x559d92e1bd00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d8849f5073f88d4 server_cookie=2616dc93357ba5dc in_seq=0 out_seq=0 2024-04-02T14:46:23.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.084+0000 7f8748c00640 10 client.26156 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8748c00640 1 -- 172.21.15.62:0/426061859 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d92d06cb0 con 0x559d92e198e0 2024-04-02T14:46:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8748c00640 10 client.26156 handle_command_reply: tid=0 2024-04-02T14:46:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8768fe8740 1 client.26156 shutdown 2024-04-02T14:46:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8768fe8740 1 -- 172.21.15.62:0/426061859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 msgr2=0x559d92da6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8768fe8740 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 0x559d92da6ca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f874c01b040 tx=0x7f874c009f40 comp rx=0 tx=0).stop 2024-04-02T14:46:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8768fe8740 1 -- 172.21.15.62:0/426061859 shutdown_connections 2024-04-02T14:46:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8768fe8740 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x559d92e198e0 0x559d92e1bd00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8768fe8740 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d92da3270 0x559d92e119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8768fe8740 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d92d96120 0x559d92da6ca0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8768fe8740 1 --2- 172.21.15.62:0/426061859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d92d95770 0x559d92da6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8748c00640 0 client.26156 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.086+0000 7f8768fe8740 1 -- 172.21.15.62:0/426061859 >> 172.21.15.62:0/426061859 conn(0x559d92d90090 msgr2=0x559d92d8f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.087+0000 7f8768fe8740 1 -- 172.21.15.62:0/426061859 shutdown_connections 2024-04-02T14:46:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.087+0000 7f8768fe8740 1 -- 172.21.15.62:0/426061859 wait complete. 2024-04-02T14:46:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.087+0000 7f8768fe8740 20 client.26156 trim_cache size 0 max 16384 2024-04-02T14:46:23.084 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:46:23.084 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-02T14:46:23.084 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-02T14:46:23.085 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:46:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.087+0000 7f8764c00640 1 -- 172.21.15.62:0/52374484 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87440a3d20 msgr2=0x7f87440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.087+0000 7f8764c00640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87440a3d20 0x7f87440a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f87580117b0 tx=0x7f8758009500 comp rx=0 tx=0).stop 2024-04-02T14:46:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.087+0000 7f8764c00640 1 -- 172.21.15.62:0/52374484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8760101970 msgr2=0x7f8760198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.087+0000 7f8764c00640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8760101970 0x7f8760198340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8750009900 tx=0x7f8750004c30 comp rx=0 tx=0).stop 2024-04-02T14:46:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.088+0000 7f8764c00640 1 -- 172.21.15.62:0/52374484 shutdown_connections 2024-04-02T14:46:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.088+0000 7f8764c00640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87440a3d20 0x7f87440a61e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.088+0000 7f8764c00640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8760107980 0x7f876019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.088+0000 7f8764c00640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760102330 0x7f8760198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.088+0000 7f8764c00640 1 --2- 172.21.15.62:0/52374484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8760101970 0x7f8760198340 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.088+0000 7f8764c00640 1 -- 172.21.15.62:0/52374484 >> 172.21.15.62:0/52374484 conn(0x7f87600fd4a0 msgr2=0x7f87600fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.088+0000 7f8764c00640 1 -- 172.21.15.62:0/52374484 shutdown_connections 2024-04-02T14:46:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.088+0000 7f8764c00640 1 -- 172.21.15.62:0/52374484 wait complete. 2024-04-02T14:46:23.102 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:46:23.102 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:46:23.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.354+0000 7f1375600640 1 -- 172.21.15.62:0/3781920833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13701012a0 msgr2=0x7f137010df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.354+0000 7f1375600640 1 --2- 172.21.15.62:0/3781920833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13701012a0 0x7f137010df60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1358009900 tx=0x7f135802bc40 comp rx=0 tx=0).stop 2024-04-02T14:46:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.354+0000 7f1375600640 1 -- 172.21.15.62:0/3781920833 shutdown_connections 2024-04-02T14:46:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.354+0000 7f1375600640 1 --2- 172.21.15.62:0/3781920833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f137010e4a0 0x7f1370110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.354+0000 7f1375600640 1 --2- 172.21.15.62:0/3781920833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13701012a0 0x7f137010df60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.354+0000 7f1375600640 1 --2- 172.21.15.62:0/3781920833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13701008f0 0x7f1370100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.354+0000 7f1375600640 1 -- 172.21.15.62:0/3781920833 >> 172.21.15.62:0/3781920833 conn(0x7f1370077b00 msgr2=0x7f1370077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.354+0000 7f1375600640 1 -- 172.21.15.62:0/3781920833 shutdown_connections 2024-04-02T14:46:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.355+0000 7f1375600640 1 -- 172.21.15.62:0/3781920833 wait complete. 2024-04-02T14:46:23.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.356+0000 7f1375600640 1 Processor -- start 2024-04-02T14:46:23.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.356+0000 7f1375600640 1 -- start start 2024-04-02T14:46:23.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.357+0000 7f1375600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13701008f0 0x7f137010de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.357+0000 7f1375600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13701012a0 0x7f137010aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.357+0000 7f1375600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137010e4a0 0x7f137010b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.357+0000 7f1375600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1370114200 con 0x7f137010e4a0 2024-04-02T14:46:23.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.357+0000 7f1375600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1370114080 con 0x7f13701012a0 2024-04-02T14:46:23.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.357+0000 7f1375600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1370114380 con 0x7f13701008f0 2024-04-02T14:46:23.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.357+0000 7f136ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13701008f0 0x7f137010de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.357+0000 7f136ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13701008f0 0x7f137010de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50300/0 (socket says 172.21.15.62:50300) 2024-04-02T14:46:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.357+0000 7f136ea00640 1 -- 172.21.15.62:0/3832536737 learned_addr learned my addr 172.21.15.62:0/3832536737 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.358+0000 7f136ea00640 1 -- 172.21.15.62:0/3832536737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13701012a0 msgr2=0x7f137010aea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T14:46:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.358+0000 7f136ea00640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13701012a0 0x7f137010aea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.358+0000 7f136ea00640 1 -- 172.21.15.62:0/3832536737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137010e4a0 msgr2=0x7f137010b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.358+0000 7f136ea00640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137010e4a0 0x7f137010b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.358+0000 7f136ea00640 1 -- 172.21.15.62:0/3832536737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137010bca0 con 0x7f13701008f0 2024-04-02T14:46:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.359+0000 7f136ea00640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13701008f0 0x7f137010de00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f136000e9f0 tx=0x7f136000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.359+0000 7f1367e00640 1 -- 172.21.15.62:0/3832536737 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136000cd80 con 0x7f13701008f0 2024-04-02T14:46:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.359+0000 7f1367e00640 1 -- 172.21.15.62:0/3832536737 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1360010930 con 0x7f13701008f0 2024-04-02T14:46:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.359+0000 7f1375600640 1 -- 172.21.15.62:0/3832536737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13701ab060 con 0x7f13701008f0 2024-04-02T14:46:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.359+0000 7f1367e00640 1 -- 172.21.15.62:0/3832536737 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360018b00 con 0x7f13701008f0 2024-04-02T14:46:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.360+0000 7f1375600640 1 -- 172.21.15.62:0/3832536737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13701ab5d0 con 0x7f13701008f0 2024-04-02T14:46:23.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.361+0000 7f1375600640 1 -- 172.21.15.62:0/3832536737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13700694a0 con 0x7f13701008f0 2024-04-02T14:46:23.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.364+0000 7f1367e00640 1 -- 172.21.15.62:0/3832536737 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1360018ca0 con 0x7f13701008f0 2024-04-02T14:46:23.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.365+0000 7f1367e00640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f13540a3de0 0x7f13540a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.365+0000 7f1367e00640 1 -- 172.21.15.62:0/3832536737 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f13600d0cf0 con 0x7f13701008f0 2024-04-02T14:46:23.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.366+0000 7f1366000640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f13540a3de0 0x7f13540a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.366+0000 7f1366000640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f13540a3de0 0x7f13540a62a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1358005c20 tx=0x7f1358005fd0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.367+0000 7f1367e00640 1 -- 172.21.15.62:0/3832536737 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f136009d3e0 con 0x7f13701008f0 2024-04-02T14:46:23.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:23 smithi204 ceph-mon[35948]: pgmap v74: 97 pgs: 97 active+clean; 144 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 858 op/s 2024-04-02T14:46:23.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:23 smithi062 ceph-mon[32116]: pgmap v74: 97 pgs: 97 active+clean; 144 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 858 op/s 2024-04-02T14:46:23.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:23 smithi113 ceph-mon[36135]: pgmap v74: 97 pgs: 97 active+clean; 144 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 858 op/s 2024-04-02T14:46:23.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.647+0000 7f1375600640 1 -- 172.21.15.62:0/3832536737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13700688f0 con 0x7f13701008f0 2024-04-02T14:46:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.648+0000 7f1367e00640 1 -- 172.21.15.62:0/3832536737 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11945 (secure 0 0 0) 0x7f136009cce0 con 0x7f13701008f0 2024-04-02T14:46:23.644 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:46:23.645 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:46:07.778359+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584,24535,24533,24553,24565]},"id":1}]} 2024-04-02T14:46:23.645 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 32 2024-04-02T14:46:23.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.651+0000 7f1375600640 1 -- 172.21.15.62:0/3832536737 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f13540a3de0 msgr2=0x7f13540a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.651+0000 7f1375600640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f13540a3de0 0x7f13540a62a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1358005c20 tx=0x7f1358005fd0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.651+0000 7f1375600640 1 -- 172.21.15.62:0/3832536737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13701008f0 msgr2=0x7f137010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.651+0000 7f1375600640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13701008f0 0x7f137010de00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f136000e9f0 tx=0x7f136000eeb0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.651+0000 7f1375600640 1 -- 172.21.15.62:0/3832536737 shutdown_connections 2024-04-02T14:46:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.651+0000 7f1375600640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f13540a3de0 0x7f13540a62a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.651+0000 7f1375600640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137010e4a0 0x7f137010b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.651+0000 7f1375600640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13701012a0 0x7f137010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.651+0000 7f1375600640 1 --2- 172.21.15.62:0/3832536737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13701008f0 0x7f137010de00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.651+0000 7f1375600640 1 -- 172.21.15.62:0/3832536737 >> 172.21.15.62:0/3832536737 conn(0x7f1370077b00 msgr2=0x7f137010fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.652+0000 7f1375600640 1 -- 172.21.15.62:0/3832536737 shutdown_connections 2024-04-02T14:46:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.652+0000 7f1375600640 1 -- 172.21.15.62:0/3832536737 wait complete. 2024-04-02T14:46:23.669 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:46:23.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.907+0000 7f8606e00640 1 -- 172.21.15.62:0/2924492401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8600103690 msgr2=0x7f860010bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.907+0000 7f8606e00640 1 --2- 172.21.15.62:0/2924492401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8600103690 0x7f860010bb70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009900 tx=0x7f85ec02bcd0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.907+0000 7f8606e00640 1 -- 172.21.15.62:0/2924492401 shutdown_connections 2024-04-02T14:46:23.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.907+0000 7f8606e00640 1 --2- 172.21.15.62:0/2924492401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f860010c140 0x7f860010e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.907+0000 7f8606e00640 1 --2- 172.21.15.62:0/2924492401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8600103690 0x7f860010bb70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.907+0000 7f8606e00640 1 --2- 172.21.15.62:0/2924492401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8600102d70 0x7f8600103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.907+0000 7f8606e00640 1 -- 172.21.15.62:0/2924492401 >> 172.21.15.62:0/2924492401 conn(0x7f8600077b90 msgr2=0x7f8600077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.907+0000 7f8606e00640 1 -- 172.21.15.62:0/2924492401 shutdown_connections 2024-04-02T14:46:23.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.907+0000 7f8606e00640 1 -- 172.21.15.62:0/2924492401 wait complete. 2024-04-02T14:46:23.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.908+0000 7f8606e00640 1 Processor -- start 2024-04-02T14:46:23.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.908+0000 7f8606e00640 1 -- start start 2024-04-02T14:46:23.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.908+0000 7f8606e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8600102d70 0x7f86001a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f8606e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8600103690 0x7f86001a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f8606e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860010c140 0x7f860019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f8606e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f86001120e0 con 0x7f8600103690 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f8606e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8600111f60 con 0x7f860010c140 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f8606e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8600112260 con 0x7f8600102d70 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f85ffe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8600102d70 0x7f86001a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f8604e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860010c140 0x7f860019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f85ff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8600103690 0x7f86001a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f85ffe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8600102d70 0x7f86001a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50320/0 (socket says 172.21.15.62:50320) 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f85ffe00640 1 -- 172.21.15.62:0/1246510067 learned_addr learned my addr 172.21.15.62:0/1246510067 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f8604e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860010c140 0x7f860019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37620/0 (socket says 172.21.15.62:37620) 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f85ff400640 1 -- 172.21.15.62:0/1246510067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8600102d70 msgr2=0x7f86001a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f85ff400640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8600102d70 0x7f86001a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f85ff400640 1 -- 172.21.15.62:0/1246510067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860010c140 msgr2=0x7f860019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f85ff400640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860010c140 0x7f860019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f85ff400640 1 -- 172.21.15.62:0/1246510067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860019ad40 con 0x7f8600103690 2024-04-02T14:46:23.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.909+0000 7f85ffe00640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8600102d70 0x7f86001a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:23.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.910+0000 7f85ff400640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8600103690 0x7f86001a0950 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f85ec02c210 tx=0x7f85ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.910+0000 7f85fcc00640 1 -- 172.21.15.62:0/1246510067 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec043020 con 0x7f8600103690 2024-04-02T14:46:23.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.910+0000 7f8606e00640 1 -- 172.21.15.62:0/1246510067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860019afd0 con 0x7f8600103690 2024-04-02T14:46:23.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.910+0000 7f85fcc00640 1 -- 172.21.15.62:0/1246510067 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85ec004250 con 0x7f8600103690 2024-04-02T14:46:23.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.910+0000 7f85fcc00640 1 -- 172.21.15.62:0/1246510067 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec003ac0 con 0x7f8600103690 2024-04-02T14:46:23.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.910+0000 7f8606e00640 1 -- 172.21.15.62:0/1246510067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86001a72d0 con 0x7f8600103690 2024-04-02T14:46:23.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.914+0000 7f85fcc00640 1 -- 172.21.15.62:0/1246510067 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f85ec046020 con 0x7f8600103690 2024-04-02T14:46:23.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.914+0000 7f8609a6a740 1 Processor -- start 2024-04-02T14:46:23.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.914+0000 7f8609a6a740 1 -- start start 2024-04-02T14:46:23.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6ad10 0x5577cfc6b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc74080 0x5577cfc76470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8609a6a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5577cfbab400 con 0x5577cfc74080 2024-04-02T14:46:23.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8609a6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5577cfb8ff10 con 0x5577cfc6b9f0 2024-04-02T14:46:23.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8609a6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5577cfbcf6f0 con 0x5577cfc6ad10 2024-04-02T14:46:23.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8604e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8604e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37636/0 (socket says 172.21.15.62:37636) 2024-04-02T14:46:23.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8604e00640 1 -- 172.21.15.62:0/4013185277 learned_addr learned my addr 172.21.15.62:0/4013185277 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:23.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f85ff400640 1 --2- 172.21.15.62:0/4013185277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc74080 0x5577cfc76470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8604e00640 1 -- 172.21.15.62:0/4013185277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6ad10 msgr2=0x5577cfc6b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8604e00640 1 --2- 172.21.15.62:0/4013185277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6ad10 0x5577cfc6b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8604e00640 1 -- 172.21.15.62:0/4013185277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc74080 msgr2=0x5577cfc76470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8604e00640 1 --2- 172.21.15.62:0/4013185277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc74080 0x5577cfc76470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.915+0000 7f8604e00640 1 -- 172.21.15.62:0/4013185277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577cf8e33f0 con 0x5577cfc6b9f0 2024-04-02T14:46:23.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.916+0000 7f85fcc00640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f85e40a3da0 0x7f85e40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.916+0000 7f8604e00640 1 --2- 172.21.15.62:0/4013185277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6a740 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f85f800a430 tx=0x7f85f8009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54b1a0ce60a384aa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.916+0000 7f85fcc00640 1 -- 172.21.15.62:0/1246510067 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f85ec0f2580 con 0x7f8600103690 2024-04-02T14:46:23.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.916+0000 7f85f5600640 1 -- 172.21.15.62:0/4013185277 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8013790 con 0x5577cfc6b9f0 2024-04-02T14:46:23.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.916+0000 7f85ffe00640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f85e40a3da0 0x7f85e40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.916+0000 7f85f5600640 1 -- 172.21.15.62:0/4013185277 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85f8019070 con 0x5577cfc6b9f0 2024-04-02T14:46:23.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.916+0000 7f85f5600640 1 -- 172.21.15.62:0/4013185277 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f80122e0 con 0x5577cfc6b9f0 2024-04-02T14:46:23.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.917+0000 7f85ffe00640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f85e40a3da0 0x7f85e40a6260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5577cfc6bf40 tx=0x7f85f0009570 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.917+0000 7f8609a6a740 1 -- 172.21.15.62:0/4013185277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6b9f0 msgr2=0x5577cfc6a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.917+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4013185277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6a740 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f85f800a430 tx=0x7f85f8009650 comp rx=0 tx=0).stop 2024-04-02T14:46:23.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.917+0000 7f8609a6a740 1 -- 172.21.15.62:0/4013185277 shutdown_connections 2024-04-02T14:46:23.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.917+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4013185277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc74080 0x5577cfc76470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.917+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4013185277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6ad10 0x5577cfc6b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.917+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4013185277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6a740 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.917+0000 7f8609a6a740 1 -- 172.21.15.62:0/4013185277 >> 172.21.15.62:0/4013185277 conn(0x5577cfc31a50 msgr2=0x5577cfc65760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.917+0000 7f8609a6a740 1 -- 172.21.15.62:0/4013185277 shutdown_connections 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.917+0000 7f8609a6a740 1 -- 172.21.15.62:0/4013185277 wait complete. 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.918+0000 7f8609a6a740 1 -- start start 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.918+0000 7f8609a6a740 1 Processor -- start 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.918+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc6ad10 0x5577cfc71080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.918+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.918+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 0x5577cfc71b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.918+0000 7f8609a6a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5577cfbcb6a0 con 0x5577cfc6ad10 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.918+0000 7f8609a6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5577cfb77980 con 0x5577cfc74080 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.918+0000 7f8609a6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5577cfb8acb0 con 0x5577cfc6b9f0 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.919+0000 7f8609a6a740 10 client.0 _ll_register_callbacks cb 0x5577cfb8a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:23.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.919+0000 7f85ffe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc6ad10 0x5577cfc71080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.919+0000 7f8604e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 0x5577cfc71b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.919+0000 7f8604e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 0x5577cfc71b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37638/0 (socket says 172.21.15.62:37638) 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.919+0000 7f8604e00640 1 -- 172.21.15.62:0/2648979886 learned_addr learned my addr 172.21.15.62:0/2648979886 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.919+0000 7f85ff400640 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.919+0000 7f8604e00640 1 -- 172.21.15.62:0/2648979886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 msgr2=0x5577cfc715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.919+0000 7f8604e00640 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.919+0000 7f8604e00640 1 -- 172.21.15.62:0/2648979886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc6ad10 msgr2=0x5577cfc71080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.919+0000 7f8604e00640 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc6ad10 0x5577cfc71080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.919+0000 7f8604e00640 1 -- 172.21.15.62:0/2648979886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577cf9ad7d0 con 0x5577cfc74080 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.920+0000 7f8604e00640 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 0x5577cfc71b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f85f8004a80 tx=0x7f85f8014a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e647d0cb511df1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.920+0000 7f85ea000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.920+0000 7f85ea000640 1 -- 172.21.15.62:0/2648979886 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f80139a0 con 0x5577cfc74080 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.920+0000 7f85ea000640 1 -- 172.21.15.62:0/2648979886 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f85f8013b40 con 0x5577cfc74080 2024-04-02T14:46:23.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.920+0000 7f8609a6a740 1 -- 172.21.15.62:0/2648979886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f8019070 con 0x5577cfc74080 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.920+0000 7f85ea000640 1 -- 172.21.15.62:0/2648979886 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f85f8013d40 con 0x5577cfc74080 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.920+0000 7f8609a6a740 10 client.26171 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.920+0000 7f8609a6a740 1 -- 172.21.15.62:0/2648979886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5577cfc82f60 con 0x5577cfc74080 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.921+0000 7f85ea000640 1 -- 172.21.15.62:0/2648979886 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85f8019070 con 0x5577cfc74080 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.921+0000 7f85ea000640 1 -- 172.21.15.62:0/2648979886 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f85f80284a0 con 0x5577cfc74080 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.921+0000 7f8609a6a740 10 client.26171 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.921+0000 7f8609a6a740 10 client.26171 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.921+0000 7f8609a6a740 20 client.26171 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.921+0000 7f8609a6a740 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5577cfc842b0 0x5577cfc866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.921+0000 7f8609a6a740 4 client.26171 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.921+0000 7f8609a6a740 1 -- 172.21.15.62:0/2648979886 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577cfb8f040 con 0x5577cfc842b0 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.921+0000 7f85ffe00640 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5577cfc842b0 0x5577cfc866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.922+0000 7f85ea000640 10 client.26171 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:46:23.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.922+0000 7f85ffe00640 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5577cfc842b0 0x5577cfc866a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a1d17faefaebbfe server_cookie=3bdcdab451c25661 in_seq=0 out_seq=0 2024-04-02T14:46:23.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f85ea000640 1 -- 172.21.15.62:0/2648979886 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577cfb8f040 con 0x5577cfc842b0 2024-04-02T14:46:23.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f85ea000640 10 client.26171 handle_command_reply: tid=0 2024-04-02T14:46:23.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f8609a6a740 1 client.26171 shutdown 2024-04-02T14:46:23.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f8609a6a740 1 -- 172.21.15.62:0/2648979886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 msgr2=0x5577cfc71b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f8609a6a740 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 0x5577cfc71b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f85f8004a80 tx=0x7f85f8014a50 comp rx=0 tx=0).stop 2024-04-02T14:46:23.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f8609a6a740 1 -- 172.21.15.62:0/2648979886 shutdown_connections 2024-04-02T14:46:23.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f8609a6a740 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5577cfc842b0 0x5577cfc866a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f8609a6a740 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 0x5577cfc71b00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f8609a6a740 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f85ea000640 0 client.26171 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:46:23.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f8609a6a740 1 --2- 172.21.15.62:0/2648979886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc6ad10 0x5577cfc71080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.923+0000 7f8609a6a740 1 -- 172.21.15.62:0/2648979886 >> 172.21.15.62:0/2648979886 conn(0x5577cfc31a50 msgr2=0x5577cfc674b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.924+0000 7f8609a6a740 1 -- 172.21.15.62:0/2648979886 shutdown_connections 2024-04-02T14:46:23.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.924+0000 7f8609a6a740 1 -- 172.21.15.62:0/2648979886 wait complete. 2024-04-02T14:46:23.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.924+0000 7f8609a6a740 20 client.26171 trim_cache size 0 max 16384 2024-04-02T14:46:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.939+0000 7f8609a6a740 1 Processor -- start 2024-04-02T14:46:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.939+0000 7f8609a6a740 1 -- start start 2024-04-02T14:46:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.939+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6ad10 0x5577cfc737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.939+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc75470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc74080 0x5577cfc759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f8609a6a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f86001af9c0 con 0x5577cfc74080 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f8609a6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f86001b1620 con 0x5577cfc6ad10 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f8609a6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86001b0090 con 0x5577cfc6b9f0 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f85ffe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6ad10 0x5577cfc737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f85ffe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6ad10 0x5577cfc737b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37654/0 (socket says 172.21.15.62:37654) 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f85ffe00640 1 -- 172.21.15.62:0/4208971599 learned_addr learned my addr 172.21.15.62:0/4208971599 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f8604e00640 1 --2- 172.21.15.62:0/4208971599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc74080 0x5577cfc759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f85ff400640 1 --2- 172.21.15.62:0/4208971599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc75470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f85ffe00640 1 -- 172.21.15.62:0/4208971599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 msgr2=0x5577cfc75470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f85ffe00640 1 --2- 172.21.15.62:0/4208971599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc75470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f85ffe00640 1 -- 172.21.15.62:0/4208971599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc74080 msgr2=0x5577cfc759b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f85ffe00640 1 --2- 172.21.15.62:0/4208971599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc74080 0x5577cfc759b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.940+0000 7f85ffe00640 1 -- 172.21.15.62:0/4208971599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577cfc66950 con 0x5577cfc6ad10 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.941+0000 7f85ffe00640 1 --2- 172.21.15.62:0/4208971599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6ad10 0x5577cfc737b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5577cfb77aa0 tx=0x7f85f0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3169693bb95293ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.941+0000 7f85f4c00640 1 -- 172.21.15.62:0/4208971599 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f002a670 con 0x5577cfc6ad10 2024-04-02T14:46:23.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.941+0000 7f85f4c00640 1 -- 172.21.15.62:0/4208971599 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85f001cde0 con 0x5577cfc6ad10 2024-04-02T14:46:23.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.941+0000 7f85f4c00640 1 -- 172.21.15.62:0/4208971599 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f00047a0 con 0x5577cfc6ad10 2024-04-02T14:46:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.941+0000 7f8609a6a740 1 -- 172.21.15.62:0/4208971599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6ad10 msgr2=0x5577cfc737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.941+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4208971599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6ad10 0x5577cfc737b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5577cfb77aa0 tx=0x7f85f0014000 comp rx=0 tx=0).stop 2024-04-02T14:46:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.942+0000 7f8609a6a740 1 -- 172.21.15.62:0/4208971599 shutdown_connections 2024-04-02T14:46:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.942+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4208971599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc74080 0x5577cfc759b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.942+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4208971599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc75470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.942+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4208971599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc6ad10 0x5577cfc737b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.942+0000 7f8609a6a740 1 -- 172.21.15.62:0/4208971599 >> 172.21.15.62:0/4208971599 conn(0x5577cfc31a50 msgr2=0x5577cfc67440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.942+0000 7f8609a6a740 1 -- 172.21.15.62:0/4208971599 shutdown_connections 2024-04-02T14:46:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.942+0000 7f8609a6a740 1 -- 172.21.15.62:0/4208971599 wait complete. 2024-04-02T14:46:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.942+0000 7f8609a6a740 1 -- start start 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.942+0000 7f8609a6a740 1 Processor -- start 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc6ad10 0x5577cfc89bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f8609a6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 0x5577cfc6e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f8609a6a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f86001b0e90 con 0x5577cfc6ad10 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f8609a6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f86001b0760 con 0x5577cfc74080 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f8609a6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86001ad9c0 con 0x5577cfc6b9f0 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f8609a6a740 10 client.0 _ll_register_callbacks cb 0x7f85f8014730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f85ff400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f8604e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 0x5577cfc6e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f85ff400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50374/0 (socket says 172.21.15.62:50374) 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f85ff400640 1 -- 172.21.15.62:0/4091650477 learned_addr learned my addr 172.21.15.62:0/4091650477 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.943+0000 7f85ffe00640 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc6ad10 0x5577cfc89bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f85ff400640 1 -- 172.21.15.62:0/4091650477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 msgr2=0x5577cfc6e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f85ff400640 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 0x5577cfc6e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f85ff400640 1 -- 172.21.15.62:0/4091650477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc6ad10 msgr2=0x5577cfc89bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f85ff400640 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc6ad10 0x5577cfc89bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f85ff400640 1 -- 172.21.15.62:0/4091650477 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577cfc69d40 con 0x5577cfc6b9f0 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f85ffe00640 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc6ad10 0x5577cfc89bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f85ff400640 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6daf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0baa80 tx=0x7f85ec000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2028b804222df06 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f85ea000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f85ea000640 1 -- 172.21.15.62:0/4091650477 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0bdf80 con 0x5577cfc6b9f0 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f85ea000640 1 -- 172.21.15.62:0/4091650477 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f85ec0053e0 con 0x5577cfc6b9f0 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f8609a6a740 1 -- 172.21.15.62:0/4091650477 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f001cde0 con 0x5577cfc6b9f0 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f85ea000640 1 -- 172.21.15.62:0/4091650477 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f85ec0bed20 con 0x5577cfc6b9f0 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f8609a6a740 10 client.26143 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.944+0000 7f8609a6a740 1 -- 172.21.15.62:0/4091650477 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5577cfcea3c0 con 0x5577cfc6b9f0 2024-04-02T14:46:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.945+0000 7f85ea000640 1 -- 172.21.15.62:0/4091650477 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85f001cde0 con 0x5577cfc6b9f0 2024-04-02T14:46:23.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.945+0000 7f85ea000640 1 -- 172.21.15.62:0/4091650477 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f85ec0eff40 con 0x5577cfc6b9f0 2024-04-02T14:46:23.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.945+0000 7f8609a6a740 10 client.26143 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:23.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.945+0000 7f8609a6a740 10 client.26143 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:46:23.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.945+0000 7f8609a6a740 20 client.26143 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:23.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.945+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5577cfceab90 0x5577cfcecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:23.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.945+0000 7f8609a6a740 4 client.26143 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:46:23.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.945+0000 7f8609a6a740 1 -- 172.21.15.62:0/4091650477 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577cfbdcc30 con 0x5577cfceab90 2024-04-02T14:46:23.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.945+0000 7f85ffe00640 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5577cfceab90 0x5577cfcecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:23.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f85ffe00640 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5577cfceab90 0x5577cfcecfb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c4a4509decc34038 server_cookie=762160753a03c3e2 in_seq=0 out_seq=0 2024-04-02T14:46:23.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f85ea000640 10 client.26143 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:46:23.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f85ea000640 1 -- 172.21.15.62:0/4091650477 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577cfbdcc30 con 0x5577cfceab90 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f85ea000640 10 client.26143 handle_command_reply: tid=0 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 1 client.26143 shutdown 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 1 -- 172.21.15.62:0/4091650477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 msgr2=0x5577cfc6daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6daf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0baa80 tx=0x7f85ec000bc0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 1 -- 172.21.15.62:0/4091650477 shutdown_connections 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x5577cfceab90 0x5577cfcecfb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577cfc74080 0x5577cfc6e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f85ea000640 0 client.26143 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577cfc6b9f0 0x5577cfc6daf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 1 --2- 172.21.15.62:0/4091650477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577cfc6ad10 0x5577cfc89bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 1 -- 172.21.15.62:0/4091650477 >> 172.21.15.62:0/4091650477 conn(0x5577cfc31a50 msgr2=0x5577cfc66b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 1 -- 172.21.15.62:0/4091650477 shutdown_connections 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 1 -- 172.21.15.62:0/4091650477 wait complete. 2024-04-02T14:46:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.946+0000 7f8609a6a740 20 client.26143 trim_cache size 0 max 16384 2024-04-02T14:46:23.947 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:46:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.947+0000 7f8606e00640 1 -- 172.21.15.62:0/1246510067 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f85e40a3da0 msgr2=0x7f85e40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.947+0000 7f8606e00640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f85e40a3da0 0x7f85e40a6260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5577cfc6bf40 tx=0x7f85f0009570 comp rx=0 tx=0).stop 2024-04-02T14:46:23.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.947+0000 7f8606e00640 1 -- 172.21.15.62:0/1246510067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8600103690 msgr2=0x7f86001a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:23.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.947+0000 7f8606e00640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8600103690 0x7f86001a0950 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f85ec02c210 tx=0x7f85ec004ab0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.947+0000 7f8606e00640 1 -- 172.21.15.62:0/1246510067 shutdown_connections 2024-04-02T14:46:23.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.947+0000 7f8606e00640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f85e40a3da0 0x7f85e40a6260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.947+0000 7f8606e00640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860010c140 0x7f860019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.947+0000 7f8606e00640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8600103690 0x7f86001a0950 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.947+0000 7f8606e00640 1 --2- 172.21.15.62:0/1246510067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8600102d70 0x7f86001a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:23.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.947+0000 7f8606e00640 1 -- 172.21.15.62:0/1246510067 >> 172.21.15.62:0/1246510067 conn(0x7f8600077b90 msgr2=0x7f86000fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:23.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.947+0000 7f8606e00640 1 -- 172.21.15.62:0/1246510067 shutdown_connections 2024-04-02T14:46:23.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:23.948+0000 7f8606e00640 1 -- 172.21.15.62:0/1246510067 wait complete. 2024-04-02T14:46:23.961 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:46:24.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.188+0000 7fe119600640 1 -- 172.21.15.62:0/3382030168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe114103f10 msgr2=0x7fe1141042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.188+0000 7fe119600640 1 --2- 172.21.15.62:0/3382030168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe114103f10 0x7fe1141042f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe104009860 tx=0x7fe10402bb90 comp rx=0 tx=0).stop 2024-04-02T14:46:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.188+0000 7fe119600640 1 -- 172.21.15.62:0/3382030168 shutdown_connections 2024-04-02T14:46:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.188+0000 7fe119600640 1 --2- 172.21.15.62:0/3382030168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe114108fe0 0x7fe114110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.188+0000 7fe119600640 1 --2- 172.21.15.62:0/3382030168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1141048c0 0x7fe114108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.188+0000 7fe119600640 1 --2- 172.21.15.62:0/3382030168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe114103f10 0x7fe1141042f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.188+0000 7fe119600640 1 -- 172.21.15.62:0/3382030168 >> 172.21.15.62:0/3382030168 conn(0x7fe1140ff750 msgr2=0x7fe114101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.188+0000 7fe119600640 1 -- 172.21.15.62:0/3382030168 shutdown_connections 2024-04-02T14:46:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.188+0000 7fe119600640 1 -- 172.21.15.62:0/3382030168 wait complete. 2024-04-02T14:46:24.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.189+0000 7fe119600640 1 Processor -- start 2024-04-02T14:46:24.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe119600640 1 -- start start 2024-04-02T14:46:24.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe119600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe114103f10 0x7fe11419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe119600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe114108fe0 0x7fe11419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe119600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11419ef80 0x7fe11419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe119600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141141d0 con 0x7fe11419ef80 2024-04-02T14:46:24.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe119600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe114114050 con 0x7fe114108fe0 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe119600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe114114350 con 0x7fe114103f10 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe10be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe114108fe0 0x7fe11419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe112a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe114103f10 0x7fe11419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe113400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11419ef80 0x7fe11419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe10be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe114108fe0 0x7fe11419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37686/0 (socket says 172.21.15.62:37686) 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.190+0000 7fe10be00640 1 -- 172.21.15.62:0/3742078458 learned_addr learned my addr 172.21.15.62:0/3742078458 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe10be00640 1 -- 172.21.15.62:0/3742078458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe114103f10 msgr2=0x7fe11419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe10be00640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe114103f10 0x7fe11419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe10be00640 1 -- 172.21.15.62:0/3742078458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11419ef80 msgr2=0x7fe11419f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe10be00640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11419ef80 0x7fe11419f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe10be00640 1 -- 172.21.15.62:0/3742078458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11419fa70 con 0x7fe114108fe0 2024-04-02T14:46:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe113400640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11419ef80 0x7fe11419f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:24.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe10be00640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe114108fe0 0x7fe11419ac50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00e9c0 tx=0x7fe0fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe10b400640 1 -- 172.21.15.62:0/3742078458 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc00cd80 con 0x7fe114108fe0 2024-04-02T14:46:24.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe10b400640 1 -- 172.21.15.62:0/3742078458 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe0fc005520 con 0x7fe114108fe0 2024-04-02T14:46:24.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe119600640 1 -- 172.21.15.62:0/3742078458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11419fd60 con 0x7fe114108fe0 2024-04-02T14:46:24.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe10b400640 1 -- 172.21.15.62:0/3742078458 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc010690 con 0x7fe114108fe0 2024-04-02T14:46:24.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.191+0000 7fe119600640 1 -- 172.21.15.62:0/3742078458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1141a7550 con 0x7fe114108fe0 2024-04-02T14:46:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.194+0000 7fe11c25a740 1 Processor -- start 2024-04-02T14:46:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.194+0000 7fe10b400640 1 -- 172.21.15.62:0/3742078458 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe0fc0046c0 con 0x7fe114108fe0 2024-04-02T14:46:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.195+0000 7fe11c25a740 1 -- start start 2024-04-02T14:46:24.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.195+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c09740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.195+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 0x55b9b4c0a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.195+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c13080 0x55b9b4c15470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.195+0000 7fe11c25a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b9b4b4a400 con 0x55b9b4c09d10 2024-04-02T14:46:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.195+0000 7fe11c25a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b9b4b2ef10 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.195+0000 7fe11c25a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b9b4b6e6f0 con 0x55b9b4c13080 2024-04-02T14:46:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.195+0000 7fe113400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c09740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.195+0000 7fe113400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c09740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37702/0 (socket says 172.21.15.62:37702) 2024-04-02T14:46:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.195+0000 7fe113400640 1 -- 172.21.15.62:0/2953027986 learned_addr learned my addr 172.21.15.62:0/2953027986 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.195+0000 7fe112a00640 1 --2- 172.21.15.62:0/2953027986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 0x55b9b4c0a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.196+0000 7fe10be00640 1 --2- 172.21.15.62:0/2953027986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c13080 0x55b9b4c15470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.196+0000 7fe113400640 1 -- 172.21.15.62:0/2953027986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c13080 msgr2=0x55b9b4c15470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.196+0000 7fe113400640 1 --2- 172.21.15.62:0/2953027986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c13080 0x55b9b4c15470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.196+0000 7fe113400640 1 -- 172.21.15.62:0/2953027986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 msgr2=0x55b9b4c0a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.196+0000 7fe113400640 1 --2- 172.21.15.62:0/2953027986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 0x55b9b4c0a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.196+0000 7fe113400640 1 -- 172.21.15.62:0/2953027986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b48823f0 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.196+0000 7fe113400640 1 --2- 172.21.15.62:0/2953027986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c09740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00e870 tx=0x7fe10c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=794a9c903db0f7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.196+0000 7fe10b400640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0f80a3d20 0x7fe0f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.196+0000 7fe103400640 1 -- 172.21.15.62:0/2953027986 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00ddd0 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.196+0000 7fe103400640 1 -- 172.21.15.62:0/2953027986 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe10c01c070 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.196+0000 7fe103400640 1 -- 172.21.15.62:0/2953027986 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c016740 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.197+0000 7fe10b400640 1 -- 172.21.15.62:0/3742078458 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fe0fc0d0b90 con 0x7fe114108fe0 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.197+0000 7fe112a00640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0f80a3d20 0x7fe0f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.197+0000 7fe11c25a740 1 -- 172.21.15.62:0/2953027986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 msgr2=0x55b9b4c09740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.197+0000 7fe11c25a740 1 --2- 172.21.15.62:0/2953027986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c09740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00e870 tx=0x7fe10c0095c0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.197+0000 7fe11c25a740 1 -- 172.21.15.62:0/2953027986 shutdown_connections 2024-04-02T14:46:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.197+0000 7fe11c25a740 1 --2- 172.21.15.62:0/2953027986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c13080 0x55b9b4c15470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.197+0000 7fe11c25a740 1 --2- 172.21.15.62:0/2953027986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 0x55b9b4c0a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.197+0000 7fe11c25a740 1 --2- 172.21.15.62:0/2953027986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c09740 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.197+0000 7fe11c25a740 1 -- 172.21.15.62:0/2953027986 >> 172.21.15.62:0/2953027986 conn(0x55b9b4bd0a50 msgr2=0x55b9b4c04760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.197+0000 7fe11c25a740 1 -- 172.21.15.62:0/2953027986 shutdown_connections 2024-04-02T14:46:24.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.198+0000 7fe11c25a740 1 -- 172.21.15.62:0/2953027986 wait complete. 2024-04-02T14:46:24.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.198+0000 7fe112a00640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0f80a3d20 0x7fe0f80a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe10402c1a0 tx=0x7fe1040056d0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.198+0000 7fe11c25a740 1 -- start start 2024-04-02T14:46:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.198+0000 7fe11c25a740 1 Processor -- start 2024-04-02T14:46:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 0x55b9b4c100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c105e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c13080 0x55b9b4c10b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe11c25a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b9b4b6a6a0 con 0x55b9b4c09d10 2024-04-02T14:46:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe11c25a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b9b4b16980 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe11c25a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b9b4b29cb0 con 0x55b9b4c13080 2024-04-02T14:46:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe11c25a740 10 client.0 _ll_register_callbacks cb 0x55b9b4b296a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe112a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 0x55b9b4c100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe10be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe112a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 0x55b9b4c100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33960/0 (socket says 172.21.15.62:33960) 2024-04-02T14:46:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe112a00640 1 -- 172.21.15.62:0/598821819 learned_addr learned my addr 172.21.15.62:0/598821819 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.199+0000 7fe113400640 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c13080 0x55b9b4c10b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe10be00640 1 -- 172.21.15.62:0/598821819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c13080 msgr2=0x55b9b4c10b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe10be00640 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c13080 0x55b9b4c10b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe10be00640 1 -- 172.21.15.62:0/598821819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 msgr2=0x55b9b4c100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe10be00640 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 0x55b9b4c100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe10be00640 1 -- 172.21.15.62:0/598821819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b494c7d0 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe112a00640 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 0x55b9b4c100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe10be00640 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c105e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0ce2d0 tx=0x7fe0fc098e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=639f2da1405e41f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe0f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe0f7e00640 1 -- 172.21.15.62:0/598821819 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0d5040 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe0f7e00640 1 -- 172.21.15.62:0/598821819 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fe0fc0a5300 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe11c25a740 1 -- 172.21.15.62:0/598821819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10c01c070 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe0f7e00640 1 -- 172.21.15.62:0/598821819 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe0fc0b5030 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe11c25a740 10 client.26189 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.200+0000 7fe11c25a740 1 -- 172.21.15.62:0/598821819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b9b4c21fc0 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.201+0000 7fe0f7e00640 1 -- 172.21.15.62:0/598821819 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe10c01c070 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.201+0000 7fe0f7e00640 1 -- 172.21.15.62:0/598821819 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fe0fc0bbce0 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.201+0000 7fe11c25a740 10 client.26189 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.201+0000 7fe11c25a740 10 client.26189 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.201+0000 7fe11c25a740 20 client.26189 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.201+0000 7fe11c25a740 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55b9b4c23480 0x55b9b4c258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.201+0000 7fe11c25a740 4 client.26189 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.201+0000 7fe11c25a740 1 -- 172.21.15.62:0/598821819 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9b4b2e040 con 0x55b9b4c23480 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.201+0000 7fe112a00640 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55b9b4c23480 0x55b9b4c258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.202+0000 7fe112a00640 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55b9b4c23480 0x55b9b4c258b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5556172938be4ca1 server_cookie=d48f2496b11d432c in_seq=0 out_seq=0 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.202+0000 7fe0f7e00640 10 client.26189 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.202+0000 7fe0f7e00640 1 -- 172.21.15.62:0/598821819 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9b4b2e040 con 0x55b9b4c23480 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe0f7e00640 10 client.26189 handle_command_reply: tid=0 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 1 client.26189 shutdown 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 1 -- 172.21.15.62:0/598821819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 msgr2=0x55b9b4c105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c105e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0ce2d0 tx=0x7fe0fc098e80 comp rx=0 tx=0).stop 2024-04-02T14:46:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 1 -- 172.21.15.62:0/598821819 shutdown_connections 2024-04-02T14:46:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55b9b4c23480 0x55b9b4c258b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c13080 0x55b9b4c10b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c105e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 1 --2- 172.21.15.62:0/598821819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c09d10 0x55b9b4c100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe0f7e00640 0 client.26189 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:46:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 1 -- 172.21.15.62:0/598821819 >> 172.21.15.62:0/598821819 conn(0x55b9b4bd0a50 msgr2=0x55b9b4c064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 1 -- 172.21.15.62:0/598821819 shutdown_connections 2024-04-02T14:46:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 1 -- 172.21.15.62:0/598821819 wait complete. 2024-04-02T14:46:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.203+0000 7fe11c25a740 20 client.26189 trim_cache size 0 max 16384 2024-04-02T14:46:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.221+0000 7fe11c25a740 1 Processor -- start 2024-04-02T14:46:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.221+0000 7fe11c25a740 1 -- start start 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.221+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c09d10 0x55b9b4c127d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.221+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c1eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.221+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c13080 0x55b9b4c06250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.221+0000 7fe11c25a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141af9e0 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.221+0000 7fe11c25a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141b1640 con 0x55b9b4c13080 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.221+0000 7fe11c25a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141b00b0 con 0x55b9b4c09d10 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.222+0000 7fe10be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c1eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.222+0000 7fe113400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c13080 0x55b9b4c06250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.222+0000 7fe112a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c09d10 0x55b9b4c127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.222+0000 7fe10be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c1eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33972/0 (socket says 172.21.15.62:33972) 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.222+0000 7fe113400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c13080 0x55b9b4c06250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37714/0 (socket says 172.21.15.62:37714) 2024-04-02T14:46:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.222+0000 7fe10be00640 1 -- 172.21.15.62:0/3661211076 learned_addr learned my addr 172.21.15.62:0/3661211076 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.222+0000 7fe112a00640 1 -- 172.21.15.62:0/3661211076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c13080 msgr2=0x55b9b4c06250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.222+0000 7fe112a00640 1 --2- 172.21.15.62:0/3661211076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c13080 0x55b9b4c06250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.222+0000 7fe112a00640 1 -- 172.21.15.62:0/3661211076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c0a9f0 msgr2=0x55b9b4c1eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.222+0000 7fe112a00640 1 --2- 172.21.15.62:0/3661211076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c1eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.222+0000 7fe112a00640 1 -- 172.21.15.62:0/3661211076 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b4c08d40 con 0x55b9b4c09d10 2024-04-02T14:46:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.223+0000 7fe112a00640 1 --2- 172.21.15.62:0/3661211076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c09d10 0x55b9b4c127d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55b9b4c260f0 tx=0x7fe10403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f31feed49e759272 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.223+0000 7fe102a00640 1 -- 172.21.15.62:0/3661211076 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104043040 con 0x55b9b4c09d10 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.223+0000 7fe102a00640 1 -- 172.21.15.62:0/3661211076 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe104044040 con 0x55b9b4c09d10 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.223+0000 7fe102a00640 1 -- 172.21.15.62:0/3661211076 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104039040 con 0x55b9b4c09d10 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.224+0000 7fe11c25a740 1 -- 172.21.15.62:0/3661211076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c09d10 msgr2=0x55b9b4c127d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.224+0000 7fe11c25a740 1 --2- 172.21.15.62:0/3661211076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c09d10 0x55b9b4c127d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55b9b4c260f0 tx=0x7fe10403f000 comp rx=0 tx=0).stop 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.224+0000 7fe11c25a740 1 -- 172.21.15.62:0/3661211076 shutdown_connections 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.224+0000 7fe11c25a740 1 --2- 172.21.15.62:0/3661211076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c13080 0x55b9b4c06250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.224+0000 7fe11c25a740 1 --2- 172.21.15.62:0/3661211076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c1eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.224+0000 7fe11c25a740 1 --2- 172.21.15.62:0/3661211076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c09d10 0x55b9b4c127d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.224+0000 7fe11c25a740 1 -- 172.21.15.62:0/3661211076 >> 172.21.15.62:0/3661211076 conn(0x55b9b4bd0a50 msgr2=0x55b9b4c05b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.224+0000 7fe11c25a740 1 -- 172.21.15.62:0/3661211076 shutdown_connections 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.224+0000 7fe11c25a740 1 -- 172.21.15.62:0/3661211076 wait complete. 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.224+0000 7fe11c25a740 1 -- start start 2024-04-02T14:46:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.224+0000 7fe11c25a740 1 Processor -- start 2024-04-02T14:46:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c09d10 0x55b9b4c1b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c1b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe11c25a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c13080 0x55b9b4c14160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe11c25a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141b0eb0 con 0x55b9b4c13080 2024-04-02T14:46:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe11c25a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141b0780 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe11c25a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141ad9e0 con 0x55b9b4c09d10 2024-04-02T14:46:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe11c25a740 10 client.0 _ll_register_callbacks cb 0x7fe0fc004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe10be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c1b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe112a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c09d10 0x55b9b4c1b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe10be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c1b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37722/0 (socket says 172.21.15.62:37722) 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe10be00640 1 -- 172.21.15.62:0/319869983 learned_addr learned my addr 172.21.15.62:0/319869983 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.225+0000 7fe113400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c13080 0x55b9b4c14160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.226+0000 7fe10be00640 1 -- 172.21.15.62:0/319869983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c09d10 msgr2=0x55b9b4c1b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.226+0000 7fe10be00640 1 --2- 172.21.15.62:0/319869983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c09d10 0x55b9b4c1b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.226+0000 7fe10be00640 1 -- 172.21.15.62:0/319869983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c13080 msgr2=0x55b9b4c14160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.226+0000 7fe10be00640 1 --2- 172.21.15.62:0/319869983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c13080 0x55b9b4c14160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.226+0000 7fe10be00640 1 -- 172.21.15.62:0/319869983 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b4c146a0 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.226+0000 7fe10be00640 1 --2- 172.21.15.62:0/319869983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c1b540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc098f80 tx=0x7fe0fc09c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=475db2a99383b14c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.226+0000 7fe0f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.226+0000 7fe0f7e00640 1 -- 172.21.15.62:0/319869983 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc07f020 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.226+0000 7fe0f7e00640 1 -- 172.21.15.62:0/319869983 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fe0fc0a5300 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.226+0000 7fe11c25a740 1 -- 172.21.15.62:0/319869983 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe104044040 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.226+0000 7fe0f7e00640 1 -- 172.21.15.62:0/319869983 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe0fc09c2a0 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.227+0000 7fe11c25a740 10 client.26195 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.227+0000 7fe11c25a740 1 -- 172.21.15.62:0/319869983 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b9b4c8a1e0 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.227+0000 7fe0f7e00640 1 -- 172.21.15.62:0/319869983 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe104044040 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.227+0000 7fe0f7e00640 1 -- 172.21.15.62:0/319869983 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fe0fc0ce550 con 0x55b9b4c0a9f0 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.227+0000 7fe11c25a740 10 client.26195 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.227+0000 7fe11c25a740 10 client.26195 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.227+0000 7fe11c25a740 20 client.26195 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.227+0000 7fe11c25a740 1 --2- 172.21.15.62:0/319869983 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55b9b4c8a990 0x55b9b4c8cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.227+0000 7fe11c25a740 4 client.26195 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.227+0000 7fe11c25a740 1 -- 172.21.15.62:0/319869983 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9b4b7bc30 con 0x55b9b4c8a990 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.227+0000 7fe112a00640 1 --2- 172.21.15.62:0/319869983 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55b9b4c8a990 0x55b9b4c8cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.228+0000 7fe112a00640 1 --2- 172.21.15.62:0/319869983 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55b9b4c8a990 0x55b9b4c8cdb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c8b03f97338004fe server_cookie=e6f6b62d028d6f0e in_seq=0 out_seq=0 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.228+0000 7fe0f7e00640 10 client.26195 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.228+0000 7fe0f7e00640 1 -- 172.21.15.62:0/319869983 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9b4b7bc30 con 0x55b9b4c8a990 2024-04-02T14:46:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.228+0000 7fe0f7e00640 10 client.26195 handle_command_reply: tid=0 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.228+0000 7fe11c25a740 1 client.26195 shutdown 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.228+0000 7fe11c25a740 1 -- 172.21.15.62:0/319869983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 msgr2=0x55b9b4c1b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.228+0000 7fe11c25a740 1 --2- 172.21.15.62:0/319869983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c1b540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc098f80 tx=0x7fe0fc09c9b0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe11c25a740 1 -- 172.21.15.62:0/319869983 shutdown_connections 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe11c25a740 1 --2- 172.21.15.62:0/319869983 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55b9b4c8a990 0x55b9b4c8cdb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe11c25a740 1 --2- 172.21.15.62:0/319869983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9b4c13080 0x55b9b4c14160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe11c25a740 1 --2- 172.21.15.62:0/319869983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9b4c0a9f0 0x55b9b4c1b540 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe11c25a740 1 --2- 172.21.15.62:0/319869983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b4c09d10 0x55b9b4c1b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe0f7e00640 0 client.26195 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe11c25a740 1 -- 172.21.15.62:0/319869983 >> 172.21.15.62:0/319869983 conn(0x55b9b4bd0a50 msgr2=0x55b9b4c063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe11c25a740 1 -- 172.21.15.62:0/319869983 shutdown_connections 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe11c25a740 1 -- 172.21.15.62:0/319869983 wait complete. 2024-04-02T14:46:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe11c25a740 20 client.26195 trim_cache size 0 max 16384 2024-04-02T14:46:24.229 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:46:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe119600640 1 -- 172.21.15.62:0/3742078458 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0f80a3d20 msgr2=0x7fe0f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe119600640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0f80a3d20 0x7fe0f80a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe10402c1a0 tx=0x7fe1040056d0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe119600640 1 -- 172.21.15.62:0/3742078458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe114108fe0 msgr2=0x7fe11419ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.229+0000 7fe119600640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe114108fe0 0x7fe11419ac50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00e9c0 tx=0x7fe0fc00ee80 comp rx=0 tx=0).stop 2024-04-02T14:46:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.230+0000 7fe119600640 1 -- 172.21.15.62:0/3742078458 shutdown_connections 2024-04-02T14:46:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.230+0000 7fe119600640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0f80a3d20 0x7fe0f80a61e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.230+0000 7fe119600640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11419ef80 0x7fe11419f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.230+0000 7fe119600640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe114108fe0 0x7fe11419ac50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.230+0000 7fe119600640 1 --2- 172.21.15.62:0/3742078458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe114103f10 0x7fe11419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.230+0000 7fe119600640 1 -- 172.21.15.62:0/3742078458 >> 172.21.15.62:0/3742078458 conn(0x7fe1140ff750 msgr2=0x7fe114101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.230+0000 7fe119600640 1 -- 172.21.15.62:0/3742078458 shutdown_connections 2024-04-02T14:46:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.230+0000 7fe119600640 1 -- 172.21.15.62:0/3742078458 wait complete. 2024-04-02T14:46:24.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:46:24.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:24 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3832536737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:46:24.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.512+0000 7f8f49800640 1 -- 172.21.15.62:0/1370980921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f44102d90 msgr2=0x7f8f44103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.512+0000 7f8f49800640 1 --2- 172.21.15.62:0/1370980921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f44102d90 0x7f8f44103170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009900 tx=0x7f8f3402bcd0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.512+0000 7f8f49800640 1 -- 172.21.15.62:0/1370980921 shutdown_connections 2024-04-02T14:46:24.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.512+0000 7f8f49800640 1 --2- 172.21.15.62:0/1370980921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4410c0d0 0x7f8f4410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.512+0000 7f8f49800640 1 --2- 172.21.15.62:0/1370980921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f441036b0 0x7f8f4410bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.512+0000 7f8f49800640 1 --2- 172.21.15.62:0/1370980921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f44102d90 0x7f8f44103170 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.512+0000 7f8f49800640 1 -- 172.21.15.62:0/1370980921 >> 172.21.15.62:0/1370980921 conn(0x7f8f44077b60 msgr2=0x7f8f44077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.512+0000 7f8f49800640 1 -- 172.21.15.62:0/1370980921 shutdown_connections 2024-04-02T14:46:24.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.513+0000 7f8f49800640 1 -- 172.21.15.62:0/1370980921 wait complete. 2024-04-02T14:46:24.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.514+0000 7f8f49800640 1 Processor -- start 2024-04-02T14:46:24.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.514+0000 7f8f49800640 1 -- start start 2024-04-02T14:46:24.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.514+0000 7f8f49800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f44102d90 0x7f8f4419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.514+0000 7f8f49800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f441036b0 0x7f8f4419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.514+0000 7f8f49800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f4410c0d0 0x7f8f4419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.514+0000 7f8f49800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8f44112010 con 0x7f8f441036b0 2024-04-02T14:46:24.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.514+0000 7f8f49800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8f44111e90 con 0x7f8f4410c0d0 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.514+0000 7f8f49800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8f44112190 con 0x7f8f44102d90 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.514+0000 7f8f42000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f441036b0 0x7f8f4419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f43400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f4410c0d0 0x7f8f4419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f42000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f441036b0 0x7f8f4419ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34000/0 (socket says 172.21.15.62:34000) 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f43400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f4410c0d0 0x7f8f4419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37738/0 (socket says 172.21.15.62:37738) 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f42000640 1 -- 172.21.15.62:0/3822183909 learned_addr learned my addr 172.21.15.62:0/3822183909 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f42a00640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f44102d90 0x7f8f4419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f43400640 1 -- 172.21.15.62:0/3822183909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f44102d90 msgr2=0x7f8f4419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f43400640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f44102d90 0x7f8f4419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f43400640 1 -- 172.21.15.62:0/3822183909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f441036b0 msgr2=0x7f8f4419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f43400640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f441036b0 0x7f8f4419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f43400640 1 -- 172.21.15.62:0/3822183909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4419f710 con 0x7f8f4410c0d0 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f42000640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f441036b0 0x7f8f4419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.515+0000 7f8f43400640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f4410c0d0 0x7f8f4419efe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c005fd0 tx=0x7f8f3c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.516+0000 7f8f3b400640 1 -- 172.21.15.62:0/3822183909 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00cd80 con 0x7f8f4410c0d0 2024-04-02T14:46:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.516+0000 7f8f3b400640 1 -- 172.21.15.62:0/3822183909 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8f3c004b20 con 0x7f8f4410c0d0 2024-04-02T14:46:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.516+0000 7f8f3b400640 1 -- 172.21.15.62:0/3822183909 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c010790 con 0x7f8f4410c0d0 2024-04-02T14:46:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.516+0000 7f8f49800640 1 -- 172.21.15.62:0/3822183909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4419fa00 con 0x7f8f4410c0d0 2024-04-02T14:46:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.516+0000 7f8f49800640 1 -- 172.21.15.62:0/3822183909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f441a7290 con 0x7f8f4410c0d0 2024-04-02T14:46:24.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.519+0000 7f8f3b400640 1 -- 172.21.15.62:0/3822183909 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8f3c005020 con 0x7f8f4410c0d0 2024-04-02T14:46:24.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.519+0000 7f8f4c5a5740 1 Processor -- start 2024-04-02T14:46:24.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.519+0000 7f8f4c5a5740 1 -- start start 2024-04-02T14:46:24.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36c0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55eec35f5400 con 0x55eec36b59f0 2024-04-02T14:46:24.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55eec35d9f10 con 0x55eec36b4d10 2024-04-02T14:46:24.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eec36196f0 con 0x55eec36be080 2024-04-02T14:46:24.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f42a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f42000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f42a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37750/0 (socket says 172.21.15.62:37750) 2024-04-02T14:46:24.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f42a00640 1 -- 172.21.15.62:0/2630716264 learned_addr learned my addr 172.21.15.62:0/2630716264 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f42000640 1 --2- 172.21.15.62:0/2630716264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36c0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f42a00640 1 -- 172.21.15.62:0/2630716264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 msgr2=0x55eec36c0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f42a00640 1 --2- 172.21.15.62:0/2630716264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36c0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f42a00640 1 -- 172.21.15.62:0/2630716264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 msgr2=0x55eec36b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.520+0000 7f8f42a00640 1 --2- 172.21.15.62:0/2630716264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.521+0000 7f8f42a00640 1 -- 172.21.15.62:0/2630716264 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec332d3f0 con 0x55eec36b4d10 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.521+0000 7f8f3b400640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8f280a3da0 0x7f8f280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.521+0000 7f8f42a00640 1 --2- 172.21.15.62:0/2630716264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36b5190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f34000c00 tx=0x7f8f3402c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a07fc9afc92a1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.521+0000 7f8f3b400640 1 -- 172.21.15.62:0/3822183909 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8f3c0d0e00 con 0x7f8f4410c0d0 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.521+0000 7f8f42000640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8f280a3da0 0x7f8f280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.521+0000 7f8f2fe00640 1 -- 172.21.15.62:0/2630716264 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3402ca70 con 0x55eec36b4d10 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.521+0000 7f8f2fe00640 1 -- 172.21.15.62:0/2630716264 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8f3402cc10 con 0x55eec36b4d10 2024-04-02T14:46:24.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.521+0000 7f8f2fe00640 1 -- 172.21.15.62:0/2630716264 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34004920 con 0x55eec36b4d10 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.522+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/2630716264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 msgr2=0x55eec36b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.522+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/2630716264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36b5190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f34000c00 tx=0x7f8f3402c690 comp rx=0 tx=0).stop 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.522+0000 7f8f42000640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8f280a3da0 0x7f8f280a6260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8f30005e00 tx=0x7f8f3000bc70 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.522+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/2630716264 shutdown_connections 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.522+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/2630716264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36c0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.522+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/2630716264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36b5190 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.522+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/2630716264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.522+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/2630716264 >> 172.21.15.62:0/2630716264 conn(0x55eec367ba50 msgr2=0x55eec36af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.522+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/2630716264 shutdown_connections 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.522+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/2630716264 wait complete. 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.523+0000 7f8f4c5a5740 1 -- start start 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.523+0000 7f8f4c5a5740 1 Processor -- start 2024-04-02T14:46:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.523+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36b4d10 0x55eec36bb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.523+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b59f0 0x55eec36bb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.523+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36be080 0x55eec36bbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.523+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55eec36156a0 con 0x55eec36be080 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55eec35c1980 con 0x55eec36b59f0 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eec35d4cb0 con 0x55eec36b4d10 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f4c5a5740 10 client.0 _ll_register_callbacks cb 0x55eec35d46a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f42000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b59f0 0x55eec36bb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f42000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b59f0 0x55eec36bb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37754/0 (socket says 172.21.15.62:37754) 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f42000640 1 -- 172.21.15.62:0/1615387565 learned_addr learned my addr 172.21.15.62:0/1615387565 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f42a00640 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36b4d10 0x55eec36bb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f43400640 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36be080 0x55eec36bbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f42a00640 1 -- 172.21.15.62:0/1615387565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b59f0 msgr2=0x55eec36bb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f42a00640 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b59f0 0x55eec36bb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f42a00640 1 -- 172.21.15.62:0/1615387565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36be080 msgr2=0x55eec36bbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f42a00640 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36be080 0x55eec36bbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.524+0000 7f8f42a00640 1 -- 172.21.15.62:0/1615387565 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec33f77d0 con 0x55eec36b4d10 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.525+0000 7f8f42a00640 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36b4d10 0x55eec36bb020 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f34005ec0 tx=0x7f8f340045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d7b02bb7818802 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.525+0000 7f8f2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.525+0000 7f8f2cc00640 1 -- 172.21.15.62:0/1615387565 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3402ca70 con 0x55eec36b4d10 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.525+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/1615387565 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f3402cc10 con 0x55eec36b4d10 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.525+0000 7f8f2cc00640 1 -- 172.21.15.62:0/1615387565 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8f340037f0 con 0x55eec36b4d10 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.525+0000 7f8f2cc00640 1 -- 172.21.15.62:0/1615387565 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f340039f0 con 0x55eec36b4d10 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.525+0000 7f8f4c5a5740 10 client.26173 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.525+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/1615387565 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eec36cce70 con 0x55eec36b4d10 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.525+0000 7f8f2cc00640 1 -- 172.21.15.62:0/1615387565 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8f3402cc10 con 0x55eec36b4d10 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.526+0000 7f8f2cc00640 1 -- 172.21.15.62:0/1615387565 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f8f34034cb0 con 0x55eec36b4d10 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.526+0000 7f8f4c5a5740 10 client.26173 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.526+0000 7f8f4c5a5740 10 client.26173 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:46:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.526+0000 7f8f4c5a5740 20 client.26173 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.526+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55eec36ce1c0 0x55eec36d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.526+0000 7f8f4c5a5740 4 client.26173 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.526+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/1615387565 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eec35d9040 con 0x55eec36ce1c0 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.526+0000 7f8f42000640 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55eec36ce1c0 0x55eec36d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.527+0000 7f8f42000640 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55eec36ce1c0 0x55eec36d05b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=970ef69b528741f1 server_cookie=bcff5046c7e45894 in_seq=0 out_seq=0 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.527+0000 7f8f2cc00640 10 client.26173 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.527+0000 7f8f2cc00640 1 -- 172.21.15.62:0/1615387565 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eec35d9040 con 0x55eec36ce1c0 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.527+0000 7f8f2cc00640 10 client.26173 handle_command_reply: tid=0 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.527+0000 7f8f4c5a5740 1 client.26173 shutdown 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/1615387565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36b4d10 msgr2=0x55eec36bb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36b4d10 0x55eec36bb020 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f34005ec0 tx=0x7f8f340045a0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/1615387565 shutdown_connections 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55eec36ce1c0 0x55eec36d05b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36be080 0x55eec36bbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b59f0 0x55eec36bb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/1615387565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36b4d10 0x55eec36bb020 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/1615387565 >> 172.21.15.62:0/1615387565 conn(0x55eec367ba50 msgr2=0x55eec36b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f2cc00640 0 client.26173 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:46:24.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/1615387565 shutdown_connections 2024-04-02T14:46:24.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/1615387565 wait complete. 2024-04-02T14:46:24.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.528+0000 7f8f4c5a5740 20 client.26173 trim_cache size 0 max 16384 2024-04-02T14:46:24.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.544+0000 7f8f4c5a5740 1 Processor -- start 2024-04-02T14:46:24.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.544+0000 7f8f4c5a5740 1 -- start start 2024-04-02T14:46:24.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36bd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441af950 con 0x55eec36b59f0 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441b15b0 con 0x55eec36b4d10 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441b0020 con 0x55eec36be080 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f42a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36bd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f42a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36bd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37760/0 (socket says 172.21.15.62:37760) 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f42a00640 1 -- 172.21.15.62:0/780636887 learned_addr learned my addr 172.21.15.62:0/780636887 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f42000640 1 --2- 172.21.15.62:0/780636887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f43400640 1 --2- 172.21.15.62:0/780636887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f42a00640 1 -- 172.21.15.62:0/780636887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 msgr2=0x55eec36ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f42a00640 1 --2- 172.21.15.62:0/780636887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f42a00640 1 -- 172.21.15.62:0/780636887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 msgr2=0x55eec36c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f42a00640 1 --2- 172.21.15.62:0/780636887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f42a00640 1 -- 172.21.15.62:0/780636887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec36b3d40 con 0x55eec36b4d10 2024-04-02T14:46:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.545+0000 7f8f43400640 1 --2- 172.21.15.62:0/780636887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.546+0000 7f8f42a00640 1 --2- 172.21.15.62:0/780636887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36bd750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f34000c00 tx=0x7f8f34004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffd23af186f2b880 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.546+0000 7f8f2f400640 1 -- 172.21.15.62:0/780636887 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34005410 con 0x55eec36b4d10 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.546+0000 7f8f2f400640 1 -- 172.21.15.62:0/780636887 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8f3404b070 con 0x55eec36b4d10 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.546+0000 7f8f2f400640 1 -- 172.21.15.62:0/780636887 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34003b90 con 0x55eec36b4d10 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.546+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/780636887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 msgr2=0x55eec36bd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.546+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/780636887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36bd750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f34000c00 tx=0x7f8f34004610 comp rx=0 tx=0).stop 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.546+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/780636887 shutdown_connections 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.546+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/780636887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.546+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/780636887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.546+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/780636887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36bd750 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.547+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/780636887 >> 172.21.15.62:0/780636887 conn(0x55eec367ba50 msgr2=0x55eec36b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.547+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/780636887 shutdown_connections 2024-04-02T14:46:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.547+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/780636887 wait complete. 2024-04-02T14:46:24.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.547+0000 7f8f4c5a5740 1 -- start start 2024-04-02T14:46:24.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.547+0000 7f8f4c5a5740 1 Processor -- start 2024-04-02T14:46:24.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f4c5a5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441b0e20 con 0x55eec36b59f0 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441b06f0 con 0x55eec36b4d10 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f4c5a5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441ad930 con 0x55eec36be080 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f4c5a5740 10 client.0 _ll_register_callbacks cb 0x7f8f34004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f42a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f42a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37776/0 (socket says 172.21.15.62:37776) 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f42000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f42a00640 1 -- 172.21.15.62:0/3786781233 learned_addr learned my addr 172.21.15.62:0/3786781233 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f43400640 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f42a00640 1 -- 172.21.15.62:0/3786781233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 msgr2=0x55eec36b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f42a00640 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f42a00640 1 -- 172.21.15.62:0/3786781233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 msgr2=0x55eec36b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f42a00640 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.548+0000 7f8f42a00640 1 -- 172.21.15.62:0/3786781233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec36b8360 con 0x55eec36b4d10 2024-04-02T14:46:24.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.549+0000 7f8f42a00640 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36c6190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8f3402c2e0 tx=0x7f8f34058830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45845eb2d4ef1b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.549+0000 7f8f2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.549+0000 7f8f2cc00640 1 -- 172.21.15.62:0/3786781233 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3405d040 con 0x55eec36b4d10 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.549+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/3786781233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f3404b070 con 0x55eec36b4d10 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.549+0000 7f8f2cc00640 1 -- 172.21.15.62:0/3786781233 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8f340037f0 con 0x55eec36b4d10 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.549+0000 7f8f2cc00640 1 -- 172.21.15.62:0/3786781233 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f34069050 con 0x55eec36b4d10 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.549+0000 7f8f4c5a5740 10 client.26213 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.549+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/3786781233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eec3734180 con 0x55eec36b4d10 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.550+0000 7f8f2cc00640 1 -- 172.21.15.62:0/3786781233 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8f3404b070 con 0x55eec36b4d10 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.550+0000 7f8f2cc00640 1 -- 172.21.15.62:0/3786781233 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f8f34050630 con 0x55eec36b4d10 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.550+0000 7f8f4c5a5740 10 client.26213 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.550+0000 7f8f4c5a5740 10 client.26213 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.550+0000 7f8f4c5a5740 20 client.26213 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.550+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55eec3734780 0x55eec3736b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.550+0000 7f8f4c5a5740 4 client.26213 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.550+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/3786781233 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eec3626c30 con 0x55eec3734780 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.550+0000 7f8f42000640 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55eec3734780 0x55eec3736b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f42000640 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55eec3734780 0x55eec3736b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71bbc87bf7e124c0 server_cookie=d2c92401db94d748 in_seq=0 out_seq=0 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f2cc00640 10 client.26213 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f2cc00640 1 -- 172.21.15.62:0/3786781233 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eec3626c30 con 0x55eec3734780 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f2cc00640 10 client.26213 handle_command_reply: tid=0 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f4c5a5740 1 client.26213 shutdown 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/3786781233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 msgr2=0x55eec36c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36c6190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8f3402c2e0 tx=0x7f8f34058830 comp rx=0 tx=0).stop 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/3786781233 shutdown_connections 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55eec3734780 0x55eec3736b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eec36be080 0x55eec36b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eec36b59f0 0x55eec36b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f2cc00640 0 client.26213 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:46:24.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f4c5a5740 1 --2- 172.21.15.62:0/3786781233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eec36b4d10 0x55eec36c6190 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/3786781233 >> 172.21.15.62:0/3786781233 conn(0x55eec367ba50 msgr2=0x55eec36b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.551+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/3786781233 shutdown_connections 2024-04-02T14:46:24.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f4c5a5740 1 -- 172.21.15.62:0/3786781233 wait complete. 2024-04-02T14:46:24.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f4c5a5740 20 client.26213 trim_cache size 0 max 16384 2024-04-02T14:46:24.552 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:46:24.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f49800640 1 -- 172.21.15.62:0/3822183909 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8f280a3da0 msgr2=0x7f8f280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f49800640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8f280a3da0 0x7f8f280a6260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8f30005e00 tx=0x7f8f3000bc70 comp rx=0 tx=0).stop 2024-04-02T14:46:24.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f49800640 1 -- 172.21.15.62:0/3822183909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f4410c0d0 msgr2=0x7f8f4419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f49800640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f4410c0d0 0x7f8f4419efe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c005fd0 tx=0x7f8f3c00ee60 comp rx=0 tx=0).stop 2024-04-02T14:46:24.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f49800640 1 -- 172.21.15.62:0/3822183909 shutdown_connections 2024-04-02T14:46:24.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f49800640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8f280a3da0 0x7f8f280a6260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f49800640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f4410c0d0 0x7f8f4419efe0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f49800640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f441036b0 0x7f8f4419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f49800640 1 --2- 172.21.15.62:0/3822183909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f44102d90 0x7f8f4419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.552+0000 7f8f49800640 1 -- 172.21.15.62:0/3822183909 >> 172.21.15.62:0/3822183909 conn(0x7f8f44077b60 msgr2=0x7f8f44107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.553+0000 7f8f49800640 1 -- 172.21.15.62:0/3822183909 shutdown_connections 2024-04-02T14:46:24.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.553+0000 7f8f49800640 1 -- 172.21.15.62:0/3822183909 wait complete. 2024-04-02T14:46:24.563 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:24 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3832536737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:46:24.566 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:46:24.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:24 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3832536737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:46:24.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.772+0000 7fe20f200640 1 -- 172.21.15.62:0/3622122739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe21010d300 msgr2=0x7fe210114ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.772+0000 7fe20f200640 1 --2- 172.21.15.62:0/3622122739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe21010d300 0x7fe210114ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe208009ca0 tx=0x7fe20802c100 comp rx=0 tx=0).stop 2024-04-02T14:46:24.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.772+0000 7fe20f200640 1 -- 172.21.15.62:0/3622122739 shutdown_connections 2024-04-02T14:46:24.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.772+0000 7fe20f200640 1 --2- 172.21.15.62:0/3622122739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe21010d300 0x7fe210114ed0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.772+0000 7fe20f200640 1 --2- 172.21.15.62:0/3622122739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe210108c60 0x7fe21010cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.772+0000 7fe20f200640 1 --2- 172.21.15.62:0/3622122739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2101082b0 0x7fe210108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.772+0000 7fe20f200640 1 -- 172.21.15.62:0/3622122739 >> 172.21.15.62:0/3622122739 conn(0x7fe210077920 msgr2=0x7fe210106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.772+0000 7fe20f200640 1 -- 172.21.15.62:0/3622122739 shutdown_connections 2024-04-02T14:46:24.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.772+0000 7fe20f200640 1 -- 172.21.15.62:0/3622122739 wait complete. 2024-04-02T14:46:24.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.773+0000 7fe20f200640 1 Processor -- start 2024-04-02T14:46:24.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.773+0000 7fe20f200640 1 -- start start 2024-04-02T14:46:24.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2101082b0 0x7fe2101a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe21010d300 0x7fe2101a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe21019e6f0 0x7fe21019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe210118510 con 0x7fe21010d300 2024-04-02T14:46:24.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe210118390 con 0x7fe2101082b0 2024-04-02T14:46:24.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe210118690 con 0x7fe21019e6f0 2024-04-02T14:46:24.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2101082b0 0x7fe2101a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2101082b0 0x7fe2101a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37794/0 (socket says 172.21.15.62:37794) 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20de00640 1 -- 172.21.15.62:0/1928378926 learned_addr learned my addr 172.21.15.62:0/1928378926 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20d400640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe21010d300 0x7fe2101a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20e800640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe21019e6f0 0x7fe21019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20de00640 1 -- 172.21.15.62:0/1928378926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe21019e6f0 msgr2=0x7fe21019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20de00640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe21019e6f0 0x7fe21019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20de00640 1 -- 172.21.15.62:0/1928378926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe21010d300 msgr2=0x7fe2101a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20de00640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe21010d300 0x7fe2101a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.774+0000 7fe20de00640 1 -- 172.21.15.62:0/1928378926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21019f2f0 con 0x7fe2101082b0 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.775+0000 7fe20de00640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2101082b0 0x7fe2101a4630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe20000e9f0 tx=0x7fe20000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.775+0000 7fe206a00640 1 -- 172.21.15.62:0/1928378926 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20000cd80 con 0x7fe2101082b0 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.775+0000 7fe206a00640 1 -- 172.21.15.62:0/1928378926 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe200005520 con 0x7fe2101082b0 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.775+0000 7fe20f200640 1 -- 172.21.15.62:0/1928378926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2101ab3d0 con 0x7fe2101082b0 2024-04-02T14:46:24.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.775+0000 7fe206a00640 1 -- 172.21.15.62:0/1928378926 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200010690 con 0x7fe2101082b0 2024-04-02T14:46:24.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.775+0000 7fe20f200640 1 -- 172.21.15.62:0/1928378926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2101ab780 con 0x7fe2101082b0 2024-04-02T14:46:24.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe218007740 1 Processor -- start 2024-04-02T14:46:24.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe206a00640 1 -- 172.21.15.62:0/1928378926 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe2000046c0 con 0x7fe2101082b0 2024-04-02T14:46:24.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe218007740 1 -- start start 2024-04-02T14:46:24.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe218007740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d63ba0 0x559011d628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe218007740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d63340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe218007740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d6c230 0x559011d6e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe218007740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559011ca3400 con 0x559011d63ba0 2024-04-02T14:46:24.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe218007740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559011c87f10 con 0x559011d62ec0 2024-04-02T14:46:24.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe218007740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559011cc76f0 con 0x559011d6c230 2024-04-02T14:46:24.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe20de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d63340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe20de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d63340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37800/0 (socket says 172.21.15.62:37800) 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe20de00640 1 -- 172.21.15.62:0/2597214964 learned_addr learned my addr 172.21.15.62:0/2597214964 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.779+0000 7fe20d400640 1 --2- 172.21.15.62:0/2597214964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d6c230 0x559011d6e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.780+0000 7fe20e800640 1 --2- 172.21.15.62:0/2597214964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d63ba0 0x559011d628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.780+0000 7fe20de00640 1 -- 172.21.15.62:0/2597214964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d6c230 msgr2=0x559011d6e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.780+0000 7fe20de00640 1 --2- 172.21.15.62:0/2597214964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d6c230 0x559011d6e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.780+0000 7fe20de00640 1 -- 172.21.15.62:0/2597214964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d63ba0 msgr2=0x559011d628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.780+0000 7fe20de00640 1 --2- 172.21.15.62:0/2597214964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d63ba0 0x559011d628f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.780+0000 7fe20de00640 1 -- 172.21.15.62:0/2597214964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590119db3f0 con 0x559011d62ec0 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.780+0000 7fe20de00640 1 --2- 172.21.15.62:0/2597214964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d63340 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe2000ce2a0 tx=0x7fe2000d4cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ad5e47e55dca61 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.780+0000 7fe206a00640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1f40a3d20 0x7fe1f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.780+0000 7fe1fb400640 1 -- 172.21.15.62:0/2597214964 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200098e50 con 0x559011d62ec0 2024-04-02T14:46:24.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.781+0000 7fe1fb400640 1 -- 172.21.15.62:0/2597214964 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe20009d2b0 con 0x559011d62ec0 2024-04-02T14:46:24.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.781+0000 7fe20e800640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1f40a3d20 0x7fe1f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.781+0000 7fe1fb400640 1 -- 172.21.15.62:0/2597214964 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20009c000 con 0x559011d62ec0 2024-04-02T14:46:24.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.781+0000 7fe206a00640 1 -- 172.21.15.62:0/1928378926 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fe2000d0c20 con 0x7fe2101082b0 2024-04-02T14:46:24.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.781+0000 7fe218007740 1 -- 172.21.15.62:0/2597214964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 msgr2=0x559011d63340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.781+0000 7fe218007740 1 --2- 172.21.15.62:0/2597214964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d63340 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe2000ce2a0 tx=0x7fe2000d4cc0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.781+0000 7fe20e800640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1f40a3d20 0x7fe1f40a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe208004af0 tx=0x7fe20802e040 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.782+0000 7fe218007740 1 -- 172.21.15.62:0/2597214964 shutdown_connections 2024-04-02T14:46:24.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.782+0000 7fe218007740 1 --2- 172.21.15.62:0/2597214964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d6c230 0x559011d6e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.782+0000 7fe218007740 1 --2- 172.21.15.62:0/2597214964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d63340 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.782+0000 7fe218007740 1 --2- 172.21.15.62:0/2597214964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d63ba0 0x559011d628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.782+0000 7fe218007740 1 -- 172.21.15.62:0/2597214964 >> 172.21.15.62:0/2597214964 conn(0x559011d29a50 msgr2=0x559011d5d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.782+0000 7fe218007740 1 -- 172.21.15.62:0/2597214964 shutdown_connections 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.782+0000 7fe218007740 1 -- 172.21.15.62:0/2597214964 wait complete. 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.782+0000 7fe218007740 1 -- start start 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.782+0000 7fe218007740 1 Processor -- start 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe218007740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe218007740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 0x559011d69730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe218007740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d6c230 0x559011d69c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe218007740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559011cc36a0 con 0x559011d6c230 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe218007740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559011c6f980 con 0x559011d62ec0 2024-04-02T14:46:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe218007740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559011c82cb0 con 0x559011d63ba0 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe218007740 10 client.0 _ll_register_callbacks cb 0x559011c826a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe20de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe20de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d691f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37802/0 (socket says 172.21.15.62:37802) 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe20de00640 1 -- 172.21.15.62:0/2944893895 learned_addr learned my addr 172.21.15.62:0/2944893895 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe20e800640 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d6c230 0x559011d69c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe20d400640 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 0x559011d69730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe20de00640 1 -- 172.21.15.62:0/2944893895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 msgr2=0x559011d69730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe20de00640 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 0x559011d69730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe20de00640 1 -- 172.21.15.62:0/2944893895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d6c230 msgr2=0x559011d69c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.783+0000 7fe20de00640 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d6c230 0x559011d69c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.784+0000 7fe20de00640 1 -- 172.21.15.62:0/2944893895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559011aa57d0 con 0x559011d62ec0 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.784+0000 7fe20e800640 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d6c230 0x559011d69c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.784+0000 7fe20de00640 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d691f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe20009cb40 tx=0x7fe200004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d145a653225679 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.784+0000 7fe1efe00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.784+0000 7fe1efe00640 1 -- 172.21.15.62:0/2944893895 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20009c330 con 0x559011d62ec0 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.784+0000 7fe1efe00640 1 -- 172.21.15.62:0/2944893895 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fe2000bd0d0 con 0x559011d62ec0 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.784+0000 7fe218007740 1 -- 172.21.15.62:0/2944893895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe20009d2b0 con 0x559011d62ec0 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.784+0000 7fe1efe00640 1 -- 172.21.15.62:0/2944893895 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe20009c510 con 0x559011d62ec0 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.784+0000 7fe218007740 10 client.26225 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.784+0000 7fe218007740 1 -- 172.21.15.62:0/2944893895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559011d7b0a0 con 0x559011d62ec0 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.784+0000 7fe1efe00640 1 -- 172.21.15.62:0/2944893895 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe20009d2b0 con 0x559011d62ec0 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.785+0000 7fe1efe00640 1 -- 172.21.15.62:0/2944893895 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fe2000d7070 con 0x559011d62ec0 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.785+0000 7fe218007740 10 client.26225 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.785+0000 7fe218007740 10 client.26225 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.785+0000 7fe218007740 20 client.26225 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:24.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.785+0000 7fe218007740 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559011d7c3f0 0x559011d7e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.785+0000 7fe218007740 4 client.26225 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:24.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.785+0000 7fe218007740 1 -- 172.21.15.62:0/2944893895 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559011c87040 con 0x559011d7c3f0 2024-04-02T14:46:24.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.785+0000 7fe20d400640 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559011d7c3f0 0x559011d7e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.786+0000 7fe20d400640 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559011d7c3f0 0x559011d7e7e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=915728981cf9e490 server_cookie=d1860371a4837ab7 in_seq=0 out_seq=0 2024-04-02T14:46:24.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.786+0000 7fe1efe00640 10 client.26225 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:46:24.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.786+0000 7fe1efe00640 1 -- 172.21.15.62:0/2944893895 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559011c87040 con 0x559011d7c3f0 2024-04-02T14:46:24.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.786+0000 7fe1efe00640 10 client.26225 handle_command_reply: tid=0 2024-04-02T14:46:24.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 1 client.26225 shutdown 2024-04-02T14:46:24.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 1 -- 172.21.15.62:0/2944893895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 msgr2=0x559011d691f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d691f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe20009cb40 tx=0x7fe200004b10 comp rx=0 tx=0).stop 2024-04-02T14:46:24.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 1 -- 172.21.15.62:0/2944893895 shutdown_connections 2024-04-02T14:46:24.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559011d7c3f0 0x559011d7e7e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d6c230 0x559011d69c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 0x559011d69730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 1 --2- 172.21.15.62:0/2944893895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d62ec0 0x559011d691f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 1 -- 172.21.15.62:0/2944893895 >> 172.21.15.62:0/2944893895 conn(0x559011d29a50 msgr2=0x559011d5f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe1efe00640 0 client.26225 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:46:24.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 1 -- 172.21.15.62:0/2944893895 shutdown_connections 2024-04-02T14:46:24.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 1 -- 172.21.15.62:0/2944893895 wait complete. 2024-04-02T14:46:24.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.787+0000 7fe218007740 20 client.26225 trim_cache size 0 max 16384 2024-04-02T14:46:24.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.802+0000 7fe218007740 1 Processor -- start 2024-04-02T14:46:24.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.802+0000 7fe218007740 1 -- start start 2024-04-02T14:46:24.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe218007740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d62ec0 0x559011d6b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe218007740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d63ba0 0x559011d5f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe218007740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d6c230 0x559011d81800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe218007740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101b3c00 con 0x559011d6c230 2024-04-02T14:46:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe218007740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101b5860 con 0x559011d63ba0 2024-04-02T14:46:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe218007740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101b42d0 con 0x559011d62ec0 2024-04-02T14:46:24.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe20d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d63ba0 0x559011d5f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe20e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d6c230 0x559011d81800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe20d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d63ba0 0x559011d5f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37816/0 (socket says 172.21.15.62:37816) 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe20d400640 1 -- 172.21.15.62:0/3925663295 learned_addr learned my addr 172.21.15.62:0/3925663295 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe20de00640 1 --2- 172.21.15.62:0/3925663295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d62ec0 0x559011d6b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe20d400640 1 -- 172.21.15.62:0/3925663295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d62ec0 msgr2=0x559011d6b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe20d400640 1 --2- 172.21.15.62:0/3925663295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d62ec0 0x559011d6b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe20d400640 1 -- 172.21.15.62:0/3925663295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d6c230 msgr2=0x559011d81800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe20d400640 1 --2- 172.21.15.62:0/3925663295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d6c230 0x559011d81800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.803+0000 7fe20d400640 1 -- 172.21.15.62:0/3925663295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559011d63ff0 con 0x559011d63ba0 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.804+0000 7fe20d400640 1 --2- 172.21.15.62:0/3925663295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d63ba0 0x559011d5f470 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009dd0 tx=0x7fe1fc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7abe2520c8331e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.804+0000 7fe1faa00640 1 -- 172.21.15.62:0/3925663295 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc0200f0 con 0x559011d63ba0 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.804+0000 7fe1faa00640 1 -- 172.21.15.62:0/3925663295 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe1fc025070 con 0x559011d63ba0 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.804+0000 7fe1faa00640 1 -- 172.21.15.62:0/3925663295 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc010c70 con 0x559011d63ba0 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.804+0000 7fe218007740 1 -- 172.21.15.62:0/3925663295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d63ba0 msgr2=0x559011d5f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.804+0000 7fe218007740 1 --2- 172.21.15.62:0/3925663295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d63ba0 0x559011d5f470 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009dd0 tx=0x7fe1fc005f90 comp rx=0 tx=0).stop 2024-04-02T14:46:24.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.805+0000 7fe218007740 1 -- 172.21.15.62:0/3925663295 shutdown_connections 2024-04-02T14:46:24.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.805+0000 7fe218007740 1 --2- 172.21.15.62:0/3925663295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d6c230 0x559011d81800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.805+0000 7fe218007740 1 --2- 172.21.15.62:0/3925663295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d63ba0 0x559011d5f470 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.805+0000 7fe218007740 1 --2- 172.21.15.62:0/3925663295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d62ec0 0x559011d6b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.805+0000 7fe218007740 1 -- 172.21.15.62:0/3925663295 >> 172.21.15.62:0/3925663295 conn(0x559011d29a50 msgr2=0x559011d5ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.805+0000 7fe218007740 1 -- 172.21.15.62:0/3925663295 shutdown_connections 2024-04-02T14:46:24.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.805+0000 7fe218007740 1 -- 172.21.15.62:0/3925663295 wait complete. 2024-04-02T14:46:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.805+0000 7fe218007740 1 -- start start 2024-04-02T14:46:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.805+0000 7fe218007740 1 Processor -- start 2024-04-02T14:46:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe218007740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d62ec0 0x559011d7f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe218007740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 0x559011d7f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe218007740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d6c230 0x559011d7fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe218007740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101b50d0 con 0x559011d62ec0 2024-04-02T14:46:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe218007740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101b49a0 con 0x559011d6c230 2024-04-02T14:46:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe218007740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101b1be0 con 0x559011d63ba0 2024-04-02T14:46:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe218007740 10 client.0 _ll_register_callbacks cb 0x7fe2000d4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe20e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d6c230 0x559011d7fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe20d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 0x559011d7f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe20de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d62ec0 0x559011d7f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe20d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 0x559011d7f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50524/0 (socket says 172.21.15.62:50524) 2024-04-02T14:46:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe20d400640 1 -- 172.21.15.62:0/1627804671 learned_addr learned my addr 172.21.15.62:0/1627804671 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe20d400640 1 -- 172.21.15.62:0/1627804671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d6c230 msgr2=0x559011d7fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe20d400640 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d6c230 0x559011d7fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe20d400640 1 -- 172.21.15.62:0/1627804671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d62ec0 msgr2=0x559011d7f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe20d400640 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d62ec0 0x559011d7f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe20d400640 1 -- 172.21.15.62:0/1627804671 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559011d61ef0 con 0x559011d63ba0 2024-04-02T14:46:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.806+0000 7fe20e800640 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d6c230 0x559011d7fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.807+0000 7fe20d400640 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 0x559011d7f800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009b90 tx=0x7fe1fc009bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ce7cf3ada0e6d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.807+0000 7fe1efe00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.807+0000 7fe1efe00640 1 -- 172.21.15.62:0/1627804671 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc003e80 con 0x559011d63ba0 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.807+0000 7fe1efe00640 1 -- 172.21.15.62:0/1627804671 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fe1fc0200f0 con 0x559011d63ba0 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.807+0000 7fe218007740 1 -- 172.21.15.62:0/1627804671 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1fc025070 con 0x559011d63ba0 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.807+0000 7fe1efe00640 1 -- 172.21.15.62:0/1627804671 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe1fc035020 con 0x559011d63ba0 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.807+0000 7fe218007740 10 client.26197 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.807+0000 7fe218007740 1 -- 172.21.15.62:0/1627804671 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559011de2790 con 0x559011d63ba0 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.808+0000 7fe1efe00640 1 -- 172.21.15.62:0/1627804671 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe1fc025070 con 0x559011d63ba0 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.808+0000 7fe1efe00640 1 -- 172.21.15.62:0/1627804671 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fe1fc03a8f0 con 0x559011d63ba0 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.808+0000 7fe218007740 10 client.26197 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.808+0000 7fe218007740 10 client.26197 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.808+0000 7fe218007740 20 client.26197 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.808+0000 7fe218007740 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559011de2fa0 0x559011de53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.808+0000 7fe218007740 4 client.26197 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.808+0000 7fe218007740 1 -- 172.21.15.62:0/1627804671 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559011cd4c30 con 0x559011de2fa0 2024-04-02T14:46:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.808+0000 7fe20de00640 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559011de2fa0 0x559011de53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe20de00640 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559011de2fa0 0x559011de53c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef28746f554a7e2f server_cookie=2cf338937442868b in_seq=0 out_seq=0 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe1efe00640 10 client.26197 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe1efe00640 1 -- 172.21.15.62:0/1627804671 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559011cd4c30 con 0x559011de2fa0 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe1efe00640 10 client.26197 handle_command_reply: tid=0 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe218007740 1 client.26197 shutdown 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe218007740 1 -- 172.21.15.62:0/1627804671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 msgr2=0x559011d7f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe218007740 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 0x559011d7f800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009b90 tx=0x7fe1fc009bc0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe218007740 1 -- 172.21.15.62:0/1627804671 shutdown_connections 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe218007740 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x559011de2fa0 0x559011de53c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe218007740 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559011d6c230 0x559011d7fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe218007740 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559011d63ba0 0x559011d7f800 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe218007740 1 --2- 172.21.15.62:0/1627804671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559011d62ec0 0x559011d7f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe1efe00640 0 client.26197 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:46:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.809+0000 7fe218007740 1 -- 172.21.15.62:0/1627804671 >> 172.21.15.62:0/1627804671 conn(0x559011d29a50 msgr2=0x559011d771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe218007740 1 -- 172.21.15.62:0/1627804671 shutdown_connections 2024-04-02T14:46:24.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe218007740 1 -- 172.21.15.62:0/1627804671 wait complete. 2024-04-02T14:46:24.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe218007740 20 client.26197 trim_cache size 0 max 16384 2024-04-02T14:46:24.810 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:46:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe20f200640 1 -- 172.21.15.62:0/1928378926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1f40a3d20 msgr2=0x7fe1f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe20f200640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1f40a3d20 0x7fe1f40a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe208004af0 tx=0x7fe20802e040 comp rx=0 tx=0).stop 2024-04-02T14:46:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe20f200640 1 -- 172.21.15.62:0/1928378926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2101082b0 msgr2=0x7fe2101a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe20f200640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2101082b0 0x7fe2101a4630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe20000e9f0 tx=0x7fe20000edf0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe20f200640 1 -- 172.21.15.62:0/1928378926 shutdown_connections 2024-04-02T14:46:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe20f200640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe1f40a3d20 0x7fe1f40a61e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe20f200640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe21019e6f0 0x7fe21019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe20f200640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe21010d300 0x7fe2101a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe20f200640 1 --2- 172.21.15.62:0/1928378926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2101082b0 0x7fe2101a4630 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.810+0000 7fe20f200640 1 -- 172.21.15.62:0/1928378926 >> 172.21.15.62:0/1928378926 conn(0x7fe210077920 msgr2=0x7fe210077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.811+0000 7fe20f200640 1 -- 172.21.15.62:0/1928378926 shutdown_connections 2024-04-02T14:46:24.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:24.811+0000 7fe20f200640 1 -- 172.21.15.62:0/1928378926 wait complete. 2024-04-02T14:46:24.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:46:25.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.038+0000 7f6658600640 1 -- 172.21.15.62:0/642470738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6650103660 msgr2=0x7f665010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:25.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.038+0000 7f6658600640 1 --2- 172.21.15.62:0/642470738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6650103660 0x7f665010bb40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f663c009900 tx=0x7f663c02bcd0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.038+0000 7f6658600640 1 -- 172.21.15.62:0/642470738 shutdown_connections 2024-04-02T14:46:25.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.038+0000 7f6658600640 1 --2- 172.21.15.62:0/642470738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f665010c110 0x7f665010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.038+0000 7f6658600640 1 --2- 172.21.15.62:0/642470738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6650103660 0x7f665010bb40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.038+0000 7f6658600640 1 --2- 172.21.15.62:0/642470738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650102d40 0x7f6650103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.038+0000 7f6658600640 1 -- 172.21.15.62:0/642470738 >> 172.21.15.62:0/642470738 conn(0x7f6650076d80 msgr2=0x7f6650077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:25.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.038+0000 7f6658600640 1 -- 172.21.15.62:0/642470738 shutdown_connections 2024-04-02T14:46:25.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.038+0000 7f6658600640 1 -- 172.21.15.62:0/642470738 wait complete. 2024-04-02T14:46:25.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.039+0000 7f6658600640 1 Processor -- start 2024-04-02T14:46:25.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.039+0000 7f6658600640 1 -- start start 2024-04-02T14:46:25.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6658600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650102d40 0x7f6650100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6658600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6650103660 0x7f6650100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6658600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665010c110 0x7f6650101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6658600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6650112010 con 0x7f6650102d40 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6658600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6650111e90 con 0x7f665010c110 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6658600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6650112190 con 0x7f6650103660 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6655c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650102d40 0x7f6650100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6656600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665010c110 0x7f6650101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6655200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6650103660 0x7f6650100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6655c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650102d40 0x7f6650100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34084/0 (socket says 172.21.15.62:34084) 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6655c00640 1 -- 172.21.15.62:0/3073731059 learned_addr learned my addr 172.21.15.62:0/3073731059 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6656600640 1 -- 172.21.15.62:0/3073731059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6650103660 msgr2=0x7f6650100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6656600640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6650103660 0x7f6650100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6656600640 1 -- 172.21.15.62:0/3073731059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650102d40 msgr2=0x7f6650100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6656600640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650102d40 0x7f6650100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.040+0000 7f6656600640 1 -- 172.21.15.62:0/3073731059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66501a6950 con 0x7f665010c110 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.041+0000 7f6656600640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665010c110 0x7f6650101410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6648005fd0 tx=0x7f664800edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.041+0000 7f664e000640 1 -- 172.21.15.62:0/3073731059 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648004d20 con 0x7f665010c110 2024-04-02T14:46:25.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.041+0000 7f664e000640 1 -- 172.21.15.62:0/3073731059 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f664800cb20 con 0x7f665010c110 2024-04-02T14:46:25.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.041+0000 7f6658600640 1 -- 172.21.15.62:0/3073731059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66501a6c40 con 0x7f665010c110 2024-04-02T14:46:25.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.041+0000 7f664e000640 1 -- 172.21.15.62:0/3073731059 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66480116d0 con 0x7f665010c110 2024-04-02T14:46:25.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.041+0000 7f6658600640 1 -- 172.21.15.62:0/3073731059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66501a7160 con 0x7f665010c110 2024-04-02T14:46:25.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.045+0000 7f664e000640 1 -- 172.21.15.62:0/3073731059 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f6648011870 con 0x7f665010c110 2024-04-02T14:46:25.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.045+0000 7f665b216740 1 Processor -- start 2024-04-02T14:46:25.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.045+0000 7f665b216740 1 -- start start 2024-04-02T14:46:25.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.045+0000 7f665b216740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17aefc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.045+0000 7f665b216740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefcd10 0x55e17aefd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.045+0000 7f665b216740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17af06080 0x55e17af08470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.045+0000 7f665b216740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e17ae3d400 con 0x55e17af06080 2024-04-02T14:46:25.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.045+0000 7f665b216740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e17ae21f10 con 0x55e17aefcd10 2024-04-02T14:46:25.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.045+0000 7f665b216740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e17ae616f0 con 0x55e17aefd9f0 2024-04-02T14:46:25.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefcd10 0x55e17aefd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefcd10 0x55e17aefd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37840/0 (socket says 172.21.15.62:37840) 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655c00640 1 -- 172.21.15.62:0/2596763437 learned_addr learned my addr 172.21.15.62:0/2596763437 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655200640 1 --2- 172.21.15.62:0/2596763437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17aefc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655200640 1 --2- 172.21.15.62:0/2596763437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17af06080 0x55e17af08470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655c00640 1 -- 172.21.15.62:0/2596763437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 msgr2=0x55e17aefc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655c00640 1 --2- 172.21.15.62:0/2596763437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17aefc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655c00640 1 -- 172.21.15.62:0/2596763437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17af06080 msgr2=0x55e17af08470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655c00640 1 --2- 172.21.15.62:0/2596763437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17af06080 0x55e17af08470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655c00640 1 -- 172.21.15.62:0/2596763437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e17ab753f0 con 0x55e17aefcd10 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655200640 1 --2- 172.21.15.62:0/2596763437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17aefc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f6655c00640 1 --2- 172.21.15.62:0/2596763437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefcd10 0x55e17aefd190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f664400bda0 tx=0x7f66440107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92c2096db6ec65e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.046+0000 7f664e000640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66380a3dd0 0x7f66380a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.047+0000 7f6642a00640 1 -- 172.21.15.62:0/2596763437 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664401c680 con 0x55e17aefcd10 2024-04-02T14:46:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.047+0000 7f6642a00640 1 -- 172.21.15.62:0/2596763437 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6644011940 con 0x55e17aefcd10 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.047+0000 7f664e000640 1 -- 172.21.15.62:0/3073731059 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f66480d0a90 con 0x7f665010c110 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.047+0000 7f6642a00640 1 -- 172.21.15.62:0/2596763437 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664401a920 con 0x55e17aefcd10 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.047+0000 7f6655200640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66380a3dd0 0x7f66380a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.047+0000 7f665b216740 1 -- 172.21.15.62:0/2596763437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefcd10 msgr2=0x55e17aefd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.047+0000 7f665b216740 1 --2- 172.21.15.62:0/2596763437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefcd10 0x55e17aefd190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f664400bda0 tx=0x7f66440107a0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.047+0000 7f665b216740 1 -- 172.21.15.62:0/2596763437 shutdown_connections 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.048+0000 7f665b216740 1 --2- 172.21.15.62:0/2596763437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17af06080 0x55e17af08470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.048+0000 7f665b216740 1 --2- 172.21.15.62:0/2596763437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefcd10 0x55e17aefd190 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.048+0000 7f665b216740 1 --2- 172.21.15.62:0/2596763437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17aefc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.048+0000 7f665b216740 1 -- 172.21.15.62:0/2596763437 >> 172.21.15.62:0/2596763437 conn(0x55e17aec3a50 msgr2=0x55e17aef7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.048+0000 7f6655200640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66380a3dd0 0x7f66380a6290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f663c005ab0 tx=0x7f663c005a20 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.048+0000 7f665b216740 1 -- 172.21.15.62:0/2596763437 shutdown_connections 2024-04-02T14:46:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.048+0000 7f665b216740 1 -- 172.21.15.62:0/2596763437 wait complete. 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.048+0000 7f665b216740 1 -- start start 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.048+0000 7f665b216740 1 Processor -- start 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f665b216740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af03060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f665b216740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17af035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f665b216740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af03ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f665b216740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e17ae5d6a0 con 0x55e17aefcd10 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f665b216740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e17ae09980 con 0x55e17af06080 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f665b216740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e17ae1ccb0 con 0x55e17aefd9f0 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f665b216740 10 client.0 _ll_register_callbacks cb 0x55e17ae1c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f6655c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af03060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f6656600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af03ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f6655200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17af035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f6655200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17af035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50556/0 (socket says 172.21.15.62:50556) 2024-04-02T14:46:25.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f6656600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af03ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37846/0 (socket says 172.21.15.62:37846) 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.049+0000 7f6655200640 1 -- 172.21.15.62:0/406797000 learned_addr learned my addr 172.21.15.62:0/406797000 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f6656600640 1 -- 172.21.15.62:0/406797000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 msgr2=0x55e17af035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f6656600640 1 --2- 172.21.15.62:0/406797000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17af035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f6656600640 1 -- 172.21.15.62:0/406797000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 msgr2=0x55e17af03060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f6656600640 1 --2- 172.21.15.62:0/406797000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af03060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f6656600640 1 -- 172.21.15.62:0/406797000 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e17ac3f7d0 con 0x55e17af06080 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f6656600640 1 --2- 172.21.15.62:0/406797000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af03ae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f66480ce1a0 tx=0x7f664809cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5838959b74eb328a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f6637400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f6637400640 1 -- 172.21.15.62:0/406797000 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664801f030 con 0x55e17af06080 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f6637400640 1 -- 172.21.15.62:0/406797000 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f66480bbbe0 con 0x55e17af06080 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f665b216740 1 -- 172.21.15.62:0/406797000 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6644011940 con 0x55e17af06080 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f6637400640 1 -- 172.21.15.62:0/406797000 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f66480a5690 con 0x55e17af06080 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f665b216740 10 client.26243 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.050+0000 7f665b216740 1 -- 172.21.15.62:0/406797000 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e17af14e90 con 0x55e17af06080 2024-04-02T14:46:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.051+0000 7f6637400640 1 -- 172.21.15.62:0/406797000 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6644011940 con 0x55e17af06080 2024-04-02T14:46:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.051+0000 7f6637400640 1 -- 172.21.15.62:0/406797000 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f664800f040 con 0x55e17af06080 2024-04-02T14:46:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.051+0000 7f665b216740 10 client.26243 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.051+0000 7f665b216740 10 client.26243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.051+0000 7f665b216740 20 client.26243 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.051+0000 7f665b216740 1 --2- 172.21.15.62:0/406797000 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55e17af16470 0x55e17af18910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.051+0000 7f665b216740 4 client.26243 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.051+0000 7f665b216740 1 -- 172.21.15.62:0/406797000 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e17ae21040 con 0x55e17af16470 2024-04-02T14:46:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.051+0000 7f6655c00640 1 --2- 172.21.15.62:0/406797000 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55e17af16470 0x55e17af18910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.052+0000 7f6655c00640 1 --2- 172.21.15.62:0/406797000 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55e17af16470 0x55e17af18910 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6602d50c6fd8abff server_cookie=f02836fa7fc5dc63 in_seq=0 out_seq=0 2024-04-02T14:46:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.052+0000 7f6637400640 10 client.26243 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:25.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f6637400640 1 -- 172.21.15.62:0/406797000 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e17ae21040 con 0x55e17af16470 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f6637400640 10 client.26243 handle_command_reply: tid=0 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f665b216740 1 client.26243 shutdown 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f665b216740 1 -- 172.21.15.62:0/406797000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 msgr2=0x55e17af03ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f665b216740 1 --2- 172.21.15.62:0/406797000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af03ae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f66480ce1a0 tx=0x7f664809cad0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f665b216740 1 -- 172.21.15.62:0/406797000 shutdown_connections 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f665b216740 1 --2- 172.21.15.62:0/406797000 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55e17af16470 0x55e17af18910 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f665b216740 1 --2- 172.21.15.62:0/406797000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af03ae0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f665b216740 1 --2- 172.21.15.62:0/406797000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17af035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f6637400640 0 client.26243 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f665b216740 1 --2- 172.21.15.62:0/406797000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af03060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f665b216740 1 -- 172.21.15.62:0/406797000 >> 172.21.15.62:0/406797000 conn(0x55e17aec3a50 msgr2=0x55e17aef94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f665b216740 1 -- 172.21.15.62:0/406797000 shutdown_connections 2024-04-02T14:46:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.054+0000 7f665b216740 1 -- 172.21.15.62:0/406797000 wait complete. 2024-04-02T14:46:25.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.055+0000 7f665b216740 20 client.26243 trim_cache size 0 max 16384 2024-04-02T14:46:25.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.069+0000 7f665b216740 1 Processor -- start 2024-04-02T14:46:25.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.069+0000 7f665b216740 1 -- start start 2024-04-02T14:46:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f665b216740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af05790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f665b216740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefd9f0 0x55e17af11af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f665b216740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17af06080 0x55e17aef2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f665b216740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66501af9e0 con 0x55e17aefcd10 2024-04-02T14:46:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f665b216740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f66501b1640 con 0x55e17aefd9f0 2024-04-02T14:46:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f665b216740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66501b00b0 con 0x55e17af06080 2024-04-02T14:46:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f6655200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefd9f0 0x55e17af11af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f6655c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af05790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f6655200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefd9f0 0x55e17af11af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37858/0 (socket says 172.21.15.62:37858) 2024-04-02T14:46:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f6655200640 1 -- 172.21.15.62:0/3609822900 learned_addr learned my addr 172.21.15.62:0/3609822900 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f6656600640 1 --2- 172.21.15.62:0/3609822900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17af06080 0x55e17aef2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f6655200640 1 -- 172.21.15.62:0/3609822900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17af06080 msgr2=0x55e17aef2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f6655200640 1 --2- 172.21.15.62:0/3609822900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17af06080 0x55e17aef2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f6655200640 1 -- 172.21.15.62:0/3609822900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 msgr2=0x55e17af05790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f6655200640 1 --2- 172.21.15.62:0/3609822900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af05790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f6655200640 1 -- 172.21.15.62:0/3609822900 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e17aef7970 con 0x55e17aefd9f0 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.070+0000 7f6655c00640 1 --2- 172.21.15.62:0/3609822900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af05790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.071+0000 7f6655200640 1 --2- 172.21.15.62:0/3609822900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefd9f0 0x55e17af11af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f663c02c2e0 tx=0x7f663c040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7421e3bd69578ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.071+0000 7f6642000640 1 -- 172.21.15.62:0/3609822900 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c005050 con 0x55e17aefd9f0 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.071+0000 7f6642000640 1 -- 172.21.15.62:0/3609822900 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f663c052070 con 0x55e17aefd9f0 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.071+0000 7f6642000640 1 -- 172.21.15.62:0/3609822900 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c003850 con 0x55e17aefd9f0 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.072+0000 7f665b216740 1 -- 172.21.15.62:0/3609822900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefd9f0 msgr2=0x55e17af11af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.072+0000 7f665b216740 1 --2- 172.21.15.62:0/3609822900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefd9f0 0x55e17af11af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f663c02c2e0 tx=0x7f663c040000 comp rx=0 tx=0).stop 2024-04-02T14:46:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.072+0000 7f665b216740 1 -- 172.21.15.62:0/3609822900 shutdown_connections 2024-04-02T14:46:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.072+0000 7f665b216740 1 --2- 172.21.15.62:0/3609822900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17af06080 0x55e17aef2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.072+0000 7f665b216740 1 --2- 172.21.15.62:0/3609822900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17aefd9f0 0x55e17af11af0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.072+0000 7f665b216740 1 --2- 172.21.15.62:0/3609822900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af05790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.072+0000 7f665b216740 1 -- 172.21.15.62:0/3609822900 >> 172.21.15.62:0/3609822900 conn(0x55e17aec3a50 msgr2=0x55e17af03020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.072+0000 7f665b216740 1 -- 172.21.15.62:0/3609822900 shutdown_connections 2024-04-02T14:46:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.072+0000 7f665b216740 1 -- 172.21.15.62:0/3609822900 wait complete. 2024-04-02T14:46:25.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.072+0000 7f665b216740 1 -- start start 2024-04-02T14:46:25.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f665b216740 1 Processor -- start 2024-04-02T14:46:25.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f665b216740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af09250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f665b216740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17af09790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f665b216740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af74a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f665b216740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66501b0eb0 con 0x55e17aefcd10 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f665b216740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f66501b0780 con 0x55e17af06080 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f665b216740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66501ad9c0 con 0x55e17aefd9f0 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f665b216740 10 client.0 _ll_register_callbacks cb 0x7f664809c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f6656600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af74a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f6655c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af09250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f6656600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af74a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37864/0 (socket says 172.21.15.62:37864) 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.073+0000 7f6656600640 1 -- 172.21.15.62:0/332201215 learned_addr learned my addr 172.21.15.62:0/332201215 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f6655200640 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17af09790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f6656600640 1 -- 172.21.15.62:0/332201215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 msgr2=0x55e17af09790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f6656600640 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17af09790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f6656600640 1 -- 172.21.15.62:0/332201215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 msgr2=0x55e17af09250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f6656600640 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af09250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f6656600640 1 -- 172.21.15.62:0/332201215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e17af75050 con 0x55e17af06080 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f6655c00640 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af09250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f6656600640 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af74a10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f66480ce230 tx=0x7f6648005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23b5333c8b996cea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f6637400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f6637400640 1 -- 172.21.15.62:0/332201215 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664807f070 con 0x55e17af06080 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f6637400640 1 -- 172.21.15.62:0/332201215 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f66480a5200 con 0x55e17af06080 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.074+0000 7f665b216740 1 -- 172.21.15.62:0/332201215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663c052070 con 0x55e17af06080 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f6637400640 1 -- 172.21.15.62:0/332201215 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f664801f040 con 0x55e17af06080 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f665b216740 10 client.26249 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f665b216740 1 -- 172.21.15.62:0/332201215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e17af84c10 con 0x55e17af06080 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f6637400640 1 -- 172.21.15.62:0/332201215 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f663c052070 con 0x55e17af06080 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f6637400640 1 -- 172.21.15.62:0/332201215 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f664809ba70 con 0x55e17af06080 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f665b216740 10 client.26249 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f665b216740 10 client.26249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f665b216740 20 client.26249 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f665b216740 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55e17af16470 0x55e17af87290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f665b216740 4 client.26249 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:46:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f665b216740 1 -- 172.21.15.62:0/332201215 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e17ae6ec30 con 0x55e17af16470 2024-04-02T14:46:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.075+0000 7f6655c00640 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55e17af16470 0x55e17af87290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f6655c00640 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55e17af16470 0x55e17af87290 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5598808e47df7317 server_cookie=1d81660c3ea04508 in_seq=0 out_seq=0 2024-04-02T14:46:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f6637400640 10 client.26249 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f6637400640 1 -- 172.21.15.62:0/332201215 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e17ae6ec30 con 0x55e17af16470 2024-04-02T14:46:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f6637400640 10 client.26249 handle_command_reply: tid=0 2024-04-02T14:46:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f665b216740 1 client.26249 shutdown 2024-04-02T14:46:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f665b216740 1 -- 172.21.15.62:0/332201215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 msgr2=0x55e17af74a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f665b216740 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af74a10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f66480ce230 tx=0x7f6648005840 comp rx=0 tx=0).stop 2024-04-02T14:46:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f665b216740 1 -- 172.21.15.62:0/332201215 shutdown_connections 2024-04-02T14:46:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f665b216740 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55e17af16470 0x55e17af87290 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f665b216740 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e17af06080 0x55e17af74a10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f665b216740 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e17aefd9f0 0x55e17af09790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f665b216740 1 --2- 172.21.15.62:0/332201215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e17aefcd10 0x55e17af09250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f6637400640 0 client.26249 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.076+0000 7f665b216740 1 -- 172.21.15.62:0/332201215 >> 172.21.15.62:0/332201215 conn(0x55e17aec3a50 msgr2=0x55e17aef6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f665b216740 1 -- 172.21.15.62:0/332201215 shutdown_connections 2024-04-02T14:46:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f665b216740 1 -- 172.21.15.62:0/332201215 wait complete. 2024-04-02T14:46:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f665b216740 20 client.26249 trim_cache size 0 max 16384 2024-04-02T14:46:25.077 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:46:25.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f6658600640 1 -- 172.21.15.62:0/3073731059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66380a3dd0 msgr2=0x7f66380a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:25.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f6658600640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66380a3dd0 0x7f66380a6290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f663c005ab0 tx=0x7f663c005a20 comp rx=0 tx=0).stop 2024-04-02T14:46:25.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f6658600640 1 -- 172.21.15.62:0/3073731059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665010c110 msgr2=0x7f6650101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:25.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f6658600640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665010c110 0x7f6650101410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6648005fd0 tx=0x7f664800edd0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f6658600640 1 -- 172.21.15.62:0/3073731059 shutdown_connections 2024-04-02T14:46:25.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f6658600640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f66380a3dd0 0x7f66380a6290 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f6658600640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665010c110 0x7f6650101410 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f6658600640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6650103660 0x7f6650100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f6658600640 1 --2- 172.21.15.62:0/3073731059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650102d40 0x7f6650100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:25.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.077+0000 7f6658600640 1 -- 172.21.15.62:0/3073731059 >> 172.21.15.62:0/3073731059 conn(0x7f6650076d80 msgr2=0x7f66500fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:25.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.078+0000 7f6658600640 1 -- 172.21.15.62:0/3073731059 shutdown_connections 2024-04-02T14:46:25.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:25.078+0000 7f6658600640 1 -- 172.21.15.62:0/3073731059 wait complete. 2024-04-02T14:46:25.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:25 smithi204 ceph-mon[35948]: pgmap v75: 97 pgs: 97 active+clean; 144 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 696 op/s 2024-04-02T14:46:25.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:25 smithi062 ceph-mon[32116]: pgmap v75: 97 pgs: 97 active+clean; 144 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 696 op/s 2024-04-02T14:46:25.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:25 smithi113 ceph-mon[36135]: pgmap v75: 97 pgs: 97 active+clean; 144 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 696 op/s 2024-04-02T14:46:26.093 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:46:26.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:46:26.094 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:46:26.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.362+0000 7fda24e00640 1 -- 172.21.15.62:0/4063848972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda20112940 msgr2=0x7fda20114d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.362+0000 7fda24e00640 1 --2- 172.21.15.62:0/4063848972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda20112940 0x7fda20114d30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fda18009cd0 tx=0x7fda1802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.363+0000 7fda24e00640 1 -- 172.21.15.62:0/4063848972 shutdown_connections 2024-04-02T14:46:26.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.363+0000 7fda24e00640 1 --2- 172.21.15.62:0/4063848972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda20112940 0x7fda20114d30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.363+0000 7fda24e00640 1 --2- 172.21.15.62:0/4063848972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda20076950 0x7fda20112400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.363+0000 7fda24e00640 1 --2- 172.21.15.62:0/4063848972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20076030 0x7fda20076410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.363+0000 7fda24e00640 1 -- 172.21.15.62:0/4063848972 >> 172.21.15.62:0/4063848972 conn(0x7fda200ff7b0 msgr2=0x7fda20101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.363+0000 7fda24e00640 1 -- 172.21.15.62:0/4063848972 shutdown_connections 2024-04-02T14:46:26.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.364+0000 7fda24e00640 1 -- 172.21.15.62:0/4063848972 wait complete. 2024-04-02T14:46:26.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.366+0000 7fda24e00640 1 Processor -- start 2024-04-02T14:46:26.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.366+0000 7fda24e00640 1 -- start start 2024-04-02T14:46:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.366+0000 7fda24e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20076030 0x7fda20111f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda24e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda20076950 0x7fda2010afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda24e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda20112940 0x7fda2010b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda24e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda201185e0 con 0x7fda20076030 2024-04-02T14:46:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda24e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fda20118460 con 0x7fda20112940 2024-04-02T14:46:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda24e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda20118760 con 0x7fda20076950 2024-04-02T14:46:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda1de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20076030 0x7fda20111f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda17e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda20076950 0x7fda2010afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda1de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20076030 0x7fda20111f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34134/0 (socket says 172.21.15.62:34134) 2024-04-02T14:46:26.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda1de00640 1 -- 172.21.15.62:0/2709136637 learned_addr learned my addr 172.21.15.62:0/2709136637 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:26.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda1e800640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda20112940 0x7fda2010b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda17e00640 1 -- 172.21.15.62:0/2709136637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda20112940 msgr2=0x7fda2010b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda17e00640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda20112940 0x7fda2010b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda17e00640 1 -- 172.21.15.62:0/2709136637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20076030 msgr2=0x7fda20111f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.367+0000 7fda17e00640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20076030 0x7fda20111f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.368+0000 7fda17e00640 1 -- 172.21.15.62:0/2709136637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2010bad0 con 0x7fda20076950 2024-04-02T14:46:26.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.368+0000 7fda17e00640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda20076950 0x7fda2010afd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda0800e9f0 tx=0x7fda0800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.369+0000 7fda16a00640 1 -- 172.21.15.62:0/2709136637 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0800ccc0 con 0x7fda20076950 2024-04-02T14:46:26.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.369+0000 7fda16a00640 1 -- 172.21.15.62:0/2709136637 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda08005020 con 0x7fda20076950 2024-04-02T14:46:26.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.369+0000 7fda16a00640 1 -- 172.21.15.62:0/2709136637 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08004b40 con 0x7fda20076950 2024-04-02T14:46:26.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.370+0000 7fda24e00640 1 -- 172.21.15.62:0/2709136637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2010bdc0 con 0x7fda20076950 2024-04-02T14:46:26.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.370+0000 7fda24e00640 1 -- 172.21.15.62:0/2709136637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2010fd50 con 0x7fda20076950 2024-04-02T14:46:26.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.374+0000 7fda16a00640 1 -- 172.21.15.62:0/2709136637 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fda08004ce0 con 0x7fda20076950 2024-04-02T14:46:26.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.376+0000 7fda27b9f740 1 Processor -- start 2024-04-02T14:46:26.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.378+0000 7fda27b9f740 1 -- start start 2024-04-02T14:46:26.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.378+0000 7fda16a00640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda040a3e70 0x7fda040a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.378+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8f030 0x558ae2e8dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.378+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8e350 0x558ae2e8e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.378+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e976c0 0x558ae2e99ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.378+0000 7fda27b9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558ae2df2310 con 0x558ae2e8e350 2024-04-02T14:46:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.378+0000 7fda27b9f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558ae2d9a690 con 0x558ae2e8f030 2024-04-02T14:46:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.378+0000 7fda27b9f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ae2dee2c0 con 0x558ae2e976c0 2024-04-02T14:46:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.378+0000 7fda16a00640 1 -- 172.21.15.62:0/2709136637 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fda080d0600 con 0x7fda20076950 2024-04-02T14:46:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.379+0000 7fda1de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8f030 0x558ae2e8dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.379+0000 7fda1de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8f030 0x558ae2e8dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37882/0 (socket says 172.21.15.62:37882) 2024-04-02T14:46:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.379+0000 7fda1de00640 1 -- 172.21.15.62:0/2334537847 learned_addr learned my addr 172.21.15.62:0/2334537847 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.379+0000 7fda17e00640 1 --2- 172.21.15.62:0/2334537847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8e350 0x558ae2e8e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.379+0000 7fda1de00640 1 -- 172.21.15.62:0/2334537847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e976c0 msgr2=0x558ae2e99ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.379+0000 7fda1de00640 1 --2- 172.21.15.62:0/2334537847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e976c0 0x558ae2e99ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.380+0000 7fda1de00640 1 -- 172.21.15.62:0/2334537847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8e350 msgr2=0x558ae2e8e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.380+0000 7fda1de00640 1 --2- 172.21.15.62:0/2334537847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8e350 0x558ae2e8e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.380+0000 7fda1de00640 1 -- 172.21.15.62:0/2334537847 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae2bce7e0 con 0x558ae2e8f030 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.380+0000 7fda1e800640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda040a3e70 0x7fda040a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.380+0000 7fda1de00640 1 --2- 172.21.15.62:0/2334537847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8f030 0x558ae2e8dd80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fda10009e10 tx=0x7fda10011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b00150e20cc7c90e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.380+0000 7fda0f400640 1 -- 172.21.15.62:0/2334537847 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1001e070 con 0x558ae2e8f030 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.380+0000 7fda0f400640 1 -- 172.21.15.62:0/2334537847 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda10019b30 con 0x558ae2e8f030 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.380+0000 7fda0f400640 1 -- 172.21.15.62:0/2334537847 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1000fc40 con 0x558ae2e8f030 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.381+0000 7fda27b9f740 1 -- 172.21.15.62:0/2334537847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8f030 msgr2=0x558ae2e8dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.381+0000 7fda27b9f740 1 --2- 172.21.15.62:0/2334537847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8f030 0x558ae2e8dd80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fda10009e10 tx=0x7fda10011810 comp rx=0 tx=0).stop 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.381+0000 7fda1e800640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda040a3e70 0x7fda040a6330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda18009bf0 tx=0x7fda18005d80 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.381+0000 7fda27b9f740 1 -- 172.21.15.62:0/2334537847 shutdown_connections 2024-04-02T14:46:26.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.381+0000 7fda27b9f740 1 --2- 172.21.15.62:0/2334537847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e976c0 0x558ae2e99ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.381+0000 7fda27b9f740 1 --2- 172.21.15.62:0/2334537847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8e350 0x558ae2e8e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.382+0000 7fda27b9f740 1 --2- 172.21.15.62:0/2334537847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8f030 0x558ae2e8dd80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.382+0000 7fda27b9f740 1 -- 172.21.15.62:0/2334537847 >> 172.21.15.62:0/2334537847 conn(0x558ae2e22270 msgr2=0x558ae2e88da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.382+0000 7fda27b9f740 1 -- 172.21.15.62:0/2334537847 shutdown_connections 2024-04-02T14:46:26.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.382+0000 7fda27b9f740 1 -- 172.21.15.62:0/2334537847 wait complete. 2024-04-02T14:46:26.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.383+0000 7fda27b9f740 1 -- start start 2024-04-02T14:46:26.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.383+0000 7fda27b9f740 1 Processor -- start 2024-04-02T14:46:26.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.385+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8e350 0x558ae2e94620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.385+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8f030 0x558ae2e94b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.385+0000 7fda1de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8e350 0x558ae2e94620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.385+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e976c0 0x558ae2e950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.385+0000 7fda27b9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558ae2e9a530 con 0x558ae2e976c0 2024-04-02T14:46:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.385+0000 7fda27b9f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558ae2dada60 con 0x558ae2e8f030 2024-04-02T14:46:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.385+0000 7fda27b9f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ae2e9a6b0 con 0x558ae2e8e350 2024-04-02T14:46:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.385+0000 7fda27b9f740 10 client.0 _ll_register_callbacks cb 0x558ae2db3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.385+0000 7fda1de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8e350 0x558ae2e94620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50612/0 (socket says 172.21.15.62:50612) 2024-04-02T14:46:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.386+0000 7fda1de00640 1 -- 172.21.15.62:0/1604762227 learned_addr learned my addr 172.21.15.62:0/1604762227 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.386+0000 7fda1de00640 1 -- 172.21.15.62:0/1604762227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8f030 msgr2=0x558ae2e94b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T14:46:26.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.386+0000 7fda1de00640 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8f030 0x558ae2e94b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.386+0000 7fda1de00640 1 -- 172.21.15.62:0/1604762227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e976c0 msgr2=0x558ae2e950a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T14:46:26.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.386+0000 7fda1e800640 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e976c0 0x558ae2e950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.387+0000 7fda1de00640 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e976c0 0x558ae2e950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.387+0000 7fda1de00640 1 -- 172.21.15.62:0/1604762227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae2c2ef70 con 0x558ae2e8e350 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.387+0000 7fda1e800640 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e976c0 0x558ae2e950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.387+0000 7fda1de00640 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8e350 0x558ae2e94620 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fda1000f860 tx=0x7fda10019f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13d8c6472a1ffb88 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.387+0000 7fda03e00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.387+0000 7fda03e00640 1 -- 172.21.15.62:0/1604762227 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1001e040 con 0x558ae2e8e350 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.388+0000 7fda03e00640 1 -- 172.21.15.62:0/1604762227 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fda10019430 con 0x558ae2e8e350 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.388+0000 7fda27b9f740 1 -- 172.21.15.62:0/1604762227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda10019b30 con 0x558ae2e8e350 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.388+0000 7fda03e00640 1 -- 172.21.15.62:0/1604762227 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda10031050 con 0x558ae2e8e350 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.388+0000 7fda27b9f740 10 client.26221 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.388+0000 7fda27b9f740 1 -- 172.21.15.62:0/1604762227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ae2ac6170 con 0x558ae2e8e350 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.389+0000 7fda03e00640 1 -- 172.21.15.62:0/1604762227 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda10019b30 con 0x558ae2e8e350 2024-04-02T14:46:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.389+0000 7fda03e00640 1 -- 172.21.15.62:0/1604762227 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fda100046c0 con 0x558ae2e8e350 2024-04-02T14:46:26.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.389+0000 7fda27b9f740 10 client.26221 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:26.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.389+0000 7fda27b9f740 10 client.26221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.389+0000 7fda27b9f740 20 client.26221 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.389+0000 7fda27b9f740 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558ae2ea7a50 0x558ae2ea9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.389+0000 7fda27b9f740 4 client.26221 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.389+0000 7fda27b9f740 1 -- 172.21.15.62:0/1604762227 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ae2dff850 con 0x558ae2ea7a50 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.389+0000 7fda17e00640 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558ae2ea7a50 0x558ae2ea9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.390+0000 7fda17e00640 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558ae2ea7a50 0x558ae2ea9e40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd70543d5d97357f server_cookie=47de51a48a0c175e in_seq=0 out_seq=0 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.390+0000 7fda03e00640 10 client.26221 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda03e00640 1 -- 172.21.15.62:0/1604762227 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ae2dff850 con 0x558ae2ea7a50 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda03e00640 10 client.26221 handle_command_reply: tid=0 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda27b9f740 1 client.26221 shutdown 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda27b9f740 1 -- 172.21.15.62:0/1604762227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8e350 msgr2=0x558ae2e94620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda27b9f740 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8e350 0x558ae2e94620 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fda1000f860 tx=0x7fda10019f10 comp rx=0 tx=0).stop 2024-04-02T14:46:26.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda27b9f740 1 -- 172.21.15.62:0/1604762227 shutdown_connections 2024-04-02T14:46:26.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda27b9f740 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558ae2ea7a50 0x558ae2ea9e40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda27b9f740 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e976c0 0x558ae2e950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda27b9f740 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8f030 0x558ae2e94b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda27b9f740 1 --2- 172.21.15.62:0/1604762227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8e350 0x558ae2e94620 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda27b9f740 1 -- 172.21.15.62:0/1604762227 >> 172.21.15.62:0/1604762227 conn(0x558ae2e22270 msgr2=0x558ae2e8aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda03e00640 0 client.26221 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:26.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.391+0000 7fda27b9f740 1 -- 172.21.15.62:0/1604762227 shutdown_connections 2024-04-02T14:46:26.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.392+0000 7fda27b9f740 1 -- 172.21.15.62:0/1604762227 wait complete. 2024-04-02T14:46:26.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.392+0000 7fda27b9f740 20 client.26221 trim_cache size 0 max 16384 2024-04-02T14:46:26.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.411+0000 7fda27b9f740 1 Processor -- start 2024-04-02T14:46:26.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.411+0000 7fda27b9f740 1 -- start start 2024-04-02T14:46:26.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.411+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8e350 0x558ae2e96d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.411+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8f030 0x558ae2ea3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.411+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e976c0 0x558ae2eaf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda27b9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda201bc590 con 0x558ae2e8f030 2024-04-02T14:46:26.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda27b9f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fda201be1f0 con 0x558ae2e8e350 2024-04-02T14:46:26.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda27b9f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda201bcc60 con 0x558ae2e976c0 2024-04-02T14:46:26.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda1de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8e350 0x558ae2e96d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda17e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8f030 0x558ae2ea3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda1de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8e350 0x558ae2e96d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37908/0 (socket says 172.21.15.62:37908) 2024-04-02T14:46:26.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda1de00640 1 -- 172.21.15.62:0/48749667 learned_addr learned my addr 172.21.15.62:0/48749667 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:26.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda1e800640 1 --2- 172.21.15.62:0/48749667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e976c0 0x558ae2eaf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda1de00640 1 -- 172.21.15.62:0/48749667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e976c0 msgr2=0x558ae2eaf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda1de00640 1 --2- 172.21.15.62:0/48749667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e976c0 0x558ae2eaf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda1de00640 1 -- 172.21.15.62:0/48749667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8f030 msgr2=0x558ae2ea3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda1de00640 1 --2- 172.21.15.62:0/48749667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8f030 0x558ae2ea3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda1de00640 1 -- 172.21.15.62:0/48749667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae2eaa470 con 0x558ae2e8e350 2024-04-02T14:46:26.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.412+0000 7fda1de00640 1 --2- 172.21.15.62:0/48749667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8e350 0x558ae2e96d50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fda10004a90 tx=0x7fda10009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=163946503b89dd38 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.413+0000 7fda0ea00640 1 -- 172.21.15.62:0/48749667 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10015070 con 0x558ae2e8e350 2024-04-02T14:46:26.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.413+0000 7fda0ea00640 1 -- 172.21.15.62:0/48749667 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda10028070 con 0x558ae2e8e350 2024-04-02T14:46:26.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.413+0000 7fda0ea00640 1 -- 172.21.15.62:0/48749667 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1001e040 con 0x558ae2e8e350 2024-04-02T14:46:26.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.413+0000 7fda27b9f740 1 -- 172.21.15.62:0/48749667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8e350 msgr2=0x558ae2e96d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.413+0000 7fda27b9f740 1 --2- 172.21.15.62:0/48749667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8e350 0x558ae2e96d50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fda10004a90 tx=0x7fda10009760 comp rx=0 tx=0).stop 2024-04-02T14:46:26.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.413+0000 7fda27b9f740 1 -- 172.21.15.62:0/48749667 shutdown_connections 2024-04-02T14:46:26.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.413+0000 7fda27b9f740 1 --2- 172.21.15.62:0/48749667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e976c0 0x558ae2eaf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.413+0000 7fda27b9f740 1 --2- 172.21.15.62:0/48749667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8f030 0x558ae2ea3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.414+0000 7fda27b9f740 1 --2- 172.21.15.62:0/48749667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e8e350 0x558ae2e96d50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.414+0000 7fda27b9f740 1 -- 172.21.15.62:0/48749667 >> 172.21.15.62:0/48749667 conn(0x558ae2e22270 msgr2=0x558ae2e8aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.414+0000 7fda27b9f740 1 -- 172.21.15.62:0/48749667 shutdown_connections 2024-04-02T14:46:26.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.414+0000 7fda27b9f740 1 -- 172.21.15.62:0/48749667 wait complete. 2024-04-02T14:46:26.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.414+0000 7fda27b9f740 1 -- start start 2024-04-02T14:46:26.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.414+0000 7fda27b9f740 1 Processor -- start 2024-04-02T14:46:26.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8e350 0x558ae2e91a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8f030 0x558ae2e91f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda27b9f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e976c0 0x558ae2f0cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda27b9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda201bda60 con 0x558ae2e8e350 2024-04-02T14:46:26.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda27b9f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fda201bd330 con 0x558ae2e976c0 2024-04-02T14:46:26.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda27b9f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda201ba570 con 0x558ae2e8f030 2024-04-02T14:46:26.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda27b9f740 10 client.0 _ll_register_callbacks cb 0x7fda1001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:26.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda1e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e976c0 0x558ae2f0cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda1de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8e350 0x558ae2e91a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda1e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e976c0 0x558ae2f0cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37914/0 (socket says 172.21.15.62:37914) 2024-04-02T14:46:26.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda1e800640 1 -- 172.21.15.62:0/610706475 learned_addr learned my addr 172.21.15.62:0/610706475 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:26.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda17e00640 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8f030 0x558ae2e91f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda1e800640 1 -- 172.21.15.62:0/610706475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8f030 msgr2=0x558ae2e91f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda1e800640 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8f030 0x558ae2e91f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda1e800640 1 -- 172.21.15.62:0/610706475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8e350 msgr2=0x558ae2e91a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda1e800640 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8e350 0x558ae2e91a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda1e800640 1 -- 172.21.15.62:0/610706475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae2f0d450 con 0x558ae2e976c0 2024-04-02T14:46:26.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.415+0000 7fda1de00640 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8e350 0x558ae2e91a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:26.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.416+0000 7fda1e800640 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e976c0 0x558ae2f0cf10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fda2006af90 tx=0x7fda1802c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d2f38a539a38057 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.416+0000 7fda03e00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:26.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.416+0000 7fda03e00640 1 -- 172.21.15.62:0/610706475 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1804f020 con 0x558ae2e976c0 2024-04-02T14:46:26.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.416+0000 7fda03e00640 1 -- 172.21.15.62:0/610706475 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fda18003f80 con 0x558ae2e976c0 2024-04-02T14:46:26.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.416+0000 7fda27b9f740 1 -- 172.21.15.62:0/610706475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda10028070 con 0x558ae2e976c0 2024-04-02T14:46:26.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.416+0000 7fda03e00640 1 -- 172.21.15.62:0/610706475 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda1805a050 con 0x558ae2e976c0 2024-04-02T14:46:26.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.416+0000 7fda27b9f740 10 client.26264 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:26.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.416+0000 7fda27b9f740 1 -- 172.21.15.62:0/610706475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ae2f0e790 con 0x558ae2e976c0 2024-04-02T14:46:26.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.417+0000 7fda03e00640 1 -- 172.21.15.62:0/610706475 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda10028070 con 0x558ae2e976c0 2024-04-02T14:46:26.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.417+0000 7fda03e00640 1 -- 172.21.15.62:0/610706475 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fda1803fab0 con 0x558ae2e976c0 2024-04-02T14:46:26.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.417+0000 7fda27b9f740 10 client.26264 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:26.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.417+0000 7fda27b9f740 10 client.26264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:26.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.417+0000 7fda27b9f740 20 client.26264 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:26.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.417+0000 7fda27b9f740 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558ae2f0f040 0x558ae2f11460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.417+0000 7fda27b9f740 4 client.26264 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:46:26.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.417+0000 7fda27b9f740 1 -- 172.21.15.62:0/610706475 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558ae2df9ba0 con 0x558ae2f0f040 2024-04-02T14:46:26.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.417+0000 7fda1de00640 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558ae2f0f040 0x558ae2f11460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.418+0000 7fda1de00640 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558ae2f0f040 0x558ae2f11460 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd7dccec2648185 server_cookie=2c6c01909c061622 in_seq=0 out_seq=0 2024-04-02T14:46:26.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.418+0000 7fda03e00640 10 client.26264 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:26.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda03e00640 1 -- 172.21.15.62:0/610706475 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558ae2df9ba0 con 0x558ae2f0f040 2024-04-02T14:46:26.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda03e00640 10 client.26264 handle_command_reply: tid=0 2024-04-02T14:46:26.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 1 client.26264 shutdown 2024-04-02T14:46:26.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 1 -- 172.21.15.62:0/610706475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e976c0 msgr2=0x558ae2f0cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e976c0 0x558ae2f0cf10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fda2006af90 tx=0x7fda1802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 1 -- 172.21.15.62:0/610706475 shutdown_connections 2024-04-02T14:46:26.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558ae2f0f040 0x558ae2f11460 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558ae2e976c0 0x558ae2f0cf10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ae2e8f030 0x558ae2e91f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 1 --2- 172.21.15.62:0/610706475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ae2e8e350 0x558ae2e91a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 1 -- 172.21.15.62:0/610706475 >> 172.21.15.62:0/610706475 conn(0x558ae2e22270 msgr2=0x558ae2e89da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda03e00640 0 client.26264 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:26.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 1 -- 172.21.15.62:0/610706475 shutdown_connections 2024-04-02T14:46:26.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 1 -- 172.21.15.62:0/610706475 wait complete. 2024-04-02T14:46:26.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.419+0000 7fda27b9f740 20 client.26264 trim_cache size 0 max 16384 2024-04-02T14:46:26.432 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:46:26.432 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-02T14:46:26.432 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "5f9a02c4-ab9c-4524-bc64-fadb550988f6", 2024-04-02T14:46:26.432 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-02T14:46:26.432 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:46:26.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 -- 172.21.15.62:0/2709136637 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda040a3e70 msgr2=0x7fda040a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda040a3e70 0x7fda040a6330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda18009bf0 tx=0x7fda18005d80 comp rx=0 tx=0).stop 2024-04-02T14:46:26.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 -- 172.21.15.62:0/2709136637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda20076950 msgr2=0x7fda2010afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda20076950 0x7fda2010afd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda0800e9f0 tx=0x7fda0800eeb0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 -- 172.21.15.62:0/2709136637 shutdown_connections 2024-04-02T14:46:26.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda040a3e70 0x7fda040a6330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda20112940 0x7fda2010b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda20076950 0x7fda2010afd0 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda0800e9f0 tx=0x7fda0800eeb0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 --2- 172.21.15.62:0/2709136637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20076030 0x7fda20111f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 -- 172.21.15.62:0/2709136637 >> 172.21.15.62:0/2709136637 conn(0x7fda200ff7b0 msgr2=0x7fda20101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 -- 172.21.15.62:0/2709136637 shutdown_connections 2024-04-02T14:46:26.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.420+0000 7fda24e00640 1 -- 172.21.15.62:0/2709136637 wait complete. 2024-04-02T14:46:26.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:46:26.436 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:46:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.659+0000 7f96ba200640 1 -- 172.21.15.62:0/2026229152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b4108ec0 msgr2=0x7f96b4110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.659+0000 7f96ba200640 1 --2- 172.21.15.62:0/2026229152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b4108ec0 0x7f96b4110a90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009ca0 tx=0x7f96ac02c100 comp rx=0 tx=0).stop 2024-04-02T14:46:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.659+0000 7f96ba200640 1 -- 172.21.15.62:0/2026229152 shutdown_connections 2024-04-02T14:46:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.659+0000 7f96ba200640 1 --2- 172.21.15.62:0/2026229152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b4108ec0 0x7f96b4110a90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.659+0000 7f96ba200640 1 --2- 172.21.15.62:0/2026229152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96b41047a0 0x7f96b41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.659+0000 7f96ba200640 1 --2- 172.21.15.62:0/2026229152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96b4103df0 0x7f96b41041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.659+0000 7f96ba200640 1 -- 172.21.15.62:0/2026229152 >> 172.21.15.62:0/2026229152 conn(0x7f96b40ff610 msgr2=0x7f96b4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.659+0000 7f96ba200640 1 -- 172.21.15.62:0/2026229152 shutdown_connections 2024-04-02T14:46:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.659+0000 7f96ba200640 1 -- 172.21.15.62:0/2026229152 wait complete. 2024-04-02T14:46:26.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.660+0000 7f96ba200640 1 Processor -- start 2024-04-02T14:46:26.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.661+0000 7f96ba200640 1 -- start start 2024-04-02T14:46:26.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.661+0000 7f96ba200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b4103df0 0x7f96b419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.661+0000 7f96ba200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96b4108ec0 0x7f96b419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.661+0000 7f96ba200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96b419edc0 0x7f96b419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.661+0000 7f96ba200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4114090 con 0x7f96b4108ec0 2024-04-02T14:46:26.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.661+0000 7f96ba200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4113f10 con 0x7f96b4103df0 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.661+0000 7f96ba200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4114210 con 0x7f96b419edc0 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.661+0000 7f96b8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b4103df0 0x7f96b419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.661+0000 7f96b8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b4103df0 0x7f96b419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37936/0 (socket says 172.21.15.62:37936) 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.661+0000 7f96b8e00640 1 -- 172.21.15.62:0/1308908129 learned_addr learned my addr 172.21.15.62:0/1308908129 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.661+0000 7f96b3e00640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96b4108ec0 0x7f96b419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.662+0000 7f96b9800640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96b419edc0 0x7f96b419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.662+0000 7f96b8e00640 1 -- 172.21.15.62:0/1308908129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96b419edc0 msgr2=0x7f96b419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.662+0000 7f96b8e00640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96b419edc0 0x7f96b419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.662+0000 7f96b8e00640 1 -- 172.21.15.62:0/1308908129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96b4108ec0 msgr2=0x7f96b419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.662+0000 7f96b8e00640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96b4108ec0 0x7f96b419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.662+0000 7f96b8e00640 1 -- 172.21.15.62:0/1308908129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b419f8b0 con 0x7f96b4103df0 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.662+0000 7f96b8e00640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b4103df0 0x7f96b419a550 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f96a800ea60 tx=0x7f96a800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.662+0000 7f96b1600640 1 -- 172.21.15.62:0/1308908129 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a800cde0 con 0x7f96b4103df0 2024-04-02T14:46:26.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.663+0000 7f96ba200640 1 -- 172.21.15.62:0/1308908129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b419fba0 con 0x7f96b4103df0 2024-04-02T14:46:26.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.663+0000 7f96b1600640 1 -- 172.21.15.62:0/1308908129 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f96a80035f0 con 0x7f96b4103df0 2024-04-02T14:46:26.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.663+0000 7f96b1600640 1 -- 172.21.15.62:0/1308908129 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8010720 con 0x7f96b4103df0 2024-04-02T14:46:26.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.663+0000 7f96ba200640 1 -- 172.21.15.62:0/1308908129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b41a73e0 con 0x7f96b4103df0 2024-04-02T14:46:26.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.666+0000 7f96be58f740 1 Processor -- start 2024-04-02T14:46:26.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.667+0000 7f96be58f740 1 -- start start 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.667+0000 7f96be58f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a3960 0x5608287a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.667+0000 7f96be58f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a4280 0x5608287ac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.667+0000 7f96be58f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287ace60 0x5608287af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.667+0000 7f96be58f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560828707770 con 0x5608287a3960 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.667+0000 7f96be58f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5608286af700 con 0x5608287ace60 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.667+0000 7f96be58f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560828703720 con 0x5608287a4280 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.667+0000 7f96b1600640 1 -- 172.21.15.62:0/1308908129 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f96a80037a0 con 0x7f96b4103df0 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.668+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287ace60 0x5608287af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.668+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287ace60 0x5608287af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37940/0 (socket says 172.21.15.62:37940) 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.668+0000 7f96b3e00640 1 -- 172.21.15.62:0/929731896 learned_addr learned my addr 172.21.15.62:0/929731896 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.668+0000 7f96b8e00640 1 --2- 172.21.15.62:0/929731896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a4280 0x5608287ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.668+0000 7f96b9800640 1 --2- 172.21.15.62:0/929731896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a3960 0x5608287a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.668+0000 7f96b3e00640 1 -- 172.21.15.62:0/929731896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a4280 msgr2=0x5608287ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.668+0000 7f96b3e00640 1 --2- 172.21.15.62:0/929731896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a4280 0x5608287ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.669+0000 7f96b3e00640 1 -- 172.21.15.62:0/929731896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a3960 msgr2=0x5608287a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.669+0000 7f96b3e00640 1 --2- 172.21.15.62:0/929731896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a3960 0x5608287a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.669+0000 7f96b3e00640 1 -- 172.21.15.62:0/929731896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608284e57d0 con 0x5608287ace60 2024-04-02T14:46:26.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.669+0000 7f96b8e00640 1 --2- 172.21.15.62:0/929731896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a4280 0x5608287ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:26.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.669+0000 7f96b9800640 1 --2- 172.21.15.62:0/929731896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a3960 0x5608287a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:26.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.669+0000 7f96b3e00640 1 --2- 172.21.15.62:0/929731896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287ace60 0x5608287af250 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009c20 tx=0x7f96a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9a0d5ce55e9f953 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.669+0000 7f96a6000640 1 -- 172.21.15.62:0/929731896 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0009dd0 con 0x5608287ace60 2024-04-02T14:46:26.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.669+0000 7f96a6000640 1 -- 172.21.15.62:0/929731896 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f96a0005020 con 0x5608287ace60 2024-04-02T14:46:26.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.669+0000 7f96a6000640 1 -- 172.21.15.62:0/929731896 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0004ba0 con 0x5608287ace60 2024-04-02T14:46:26.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.670+0000 7f96b1600640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f969c0a3da0 0x7f969c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.670+0000 7f96b1600640 1 -- 172.21.15.62:0/1308908129 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f96a80d0ad0 con 0x7f96b4103df0 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.670+0000 7f96b9800640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f969c0a3da0 0x7f969c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.670+0000 7f96be58f740 1 -- 172.21.15.62:0/929731896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287ace60 msgr2=0x5608287af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.670+0000 7f96be58f740 1 --2- 172.21.15.62:0/929731896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287ace60 0x5608287af250 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009c20 tx=0x7f96a0011810 comp rx=0 tx=0).stop 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.671+0000 7f96be58f740 1 -- 172.21.15.62:0/929731896 shutdown_connections 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.671+0000 7f96be58f740 1 --2- 172.21.15.62:0/929731896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287ace60 0x5608287af250 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.671+0000 7f96be58f740 1 --2- 172.21.15.62:0/929731896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a4280 0x5608287ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.671+0000 7f96be58f740 1 --2- 172.21.15.62:0/929731896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a3960 0x5608287a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.671+0000 7f96be58f740 1 -- 172.21.15.62:0/929731896 >> 172.21.15.62:0/929731896 conn(0x56082879e090 msgr2=0x56082879e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.671+0000 7f96be58f740 1 -- 172.21.15.62:0/929731896 shutdown_connections 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.671+0000 7f96b9800640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f969c0a3da0 0x7f969c0a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f96ac005e30 tx=0x7f96ac02e040 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.671+0000 7f96be58f740 1 -- 172.21.15.62:0/929731896 wait complete. 2024-04-02T14:46:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.671+0000 7f96be58f740 1 -- start start 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.671+0000 7f96be58f740 1 Processor -- start 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.672+0000 7f96be58f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a3960 0x5608287a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.672+0000 7f96be58f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a4280 0x5608287aa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.672+0000 7f96be58f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287ace60 0x5608287aa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.672+0000 7f96be58f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5608287afc50 con 0x5608287ace60 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.672+0000 7f96be58f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5608286c2ac0 con 0x5608287a4280 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.672+0000 7f96be58f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5608287afdd0 con 0x5608287a3960 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.672+0000 7f96be58f740 10 client.0 _ll_register_callbacks cb 0x5608286c5180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.672+0000 7f96b8e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a3960 0x5608287a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.672+0000 7f96b8e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a3960 0x5608287a9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50680/0 (socket says 172.21.15.62:50680) 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.672+0000 7f96b8e00640 1 -- 172.21.15.62:0/2085191789 learned_addr learned my addr 172.21.15.62:0/2085191789 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.672+0000 7f96b3e00640 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a4280 0x5608287aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.673+0000 7f96b8e00640 1 -- 172.21.15.62:0/2085191789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a4280 msgr2=0x5608287aa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.673+0000 7f96b8e00640 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a4280 0x5608287aa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.673+0000 7f96b9800640 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287ace60 0x5608287aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.673+0000 7f96b8e00640 1 -- 172.21.15.62:0/2085191789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287ace60 msgr2=0x5608287aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.673+0000 7f96b8e00640 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287ace60 0x5608287aa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.673+0000 7f96b8e00640 1 -- 172.21.15.62:0/2085191789 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082841b3f0 con 0x5608287a3960 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.673+0000 7f96b9800640 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287ace60 0x5608287aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.673+0000 7f96b8e00640 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a3960 0x5608287a9ec0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f96a800ea30 tx=0x7f96a80985e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c00d5ec65ee52e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.673+0000 7f969aa00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.673+0000 7f969aa00640 1 -- 172.21.15.62:0/2085191789 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80d7060 con 0x5608287a3960 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.673+0000 7f969aa00640 1 -- 172.21.15.62:0/2085191789 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f96a8004000 con 0x5608287a3960 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.674+0000 7f96be58f740 1 -- 172.21.15.62:0/2085191789 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a0005020 con 0x5608287a3960 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.674+0000 7f969aa00640 1 -- 172.21.15.62:0/2085191789 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96a809c5a0 con 0x5608287a3960 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.674+0000 7f96be58f740 10 client.26239 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.674+0000 7f96be58f740 1 -- 172.21.15.62:0/2085191789 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5608283db0f0 con 0x5608287a3960 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.674+0000 7f969aa00640 1 -- 172.21.15.62:0/2085191789 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f96a0005020 con 0x5608287a3960 2024-04-02T14:46:26.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.674+0000 7f969aa00640 1 -- 172.21.15.62:0/2085191789 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f96a80ce490 con 0x5608287a3960 2024-04-02T14:46:26.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.674+0000 7f96be58f740 10 client.26239 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:26.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.674+0000 7f96be58f740 10 client.26239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:26.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.674+0000 7f96be58f740 20 client.26239 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:26.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.674+0000 7f96be58f740 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5608287bcea0 0x5608287bf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.675+0000 7f96be58f740 4 client.26239 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:26.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.675+0000 7f96be58f740 1 -- 172.21.15.62:0/2085191789 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608286a8630 con 0x5608287bcea0 2024-04-02T14:46:26.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.675+0000 7f96b3e00640 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5608287bcea0 0x5608287bf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.676+0000 7f96b3e00640 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5608287bcea0 0x5608287bf300 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e432fe840fa442 server_cookie=e7bf0401bc48302 in_seq=0 out_seq=0 2024-04-02T14:46:26.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.676+0000 7f969aa00640 10 client.26239 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:26.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f969aa00640 1 -- 172.21.15.62:0/2085191789 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608286a8630 con 0x5608287bcea0 2024-04-02T14:46:26.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f969aa00640 10 client.26239 handle_command_reply: tid=0 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f96be58f740 1 client.26239 shutdown 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f96be58f740 1 -- 172.21.15.62:0/2085191789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a3960 msgr2=0x5608287a9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f96be58f740 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a3960 0x5608287a9ec0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f96a800ea30 tx=0x7f96a80985e0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f96be58f740 1 -- 172.21.15.62:0/2085191789 shutdown_connections 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f96be58f740 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5608287bcea0 0x5608287bf300 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f96be58f740 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287ace60 0x5608287aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f96be58f740 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a4280 0x5608287aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f96be58f740 1 --2- 172.21.15.62:0/2085191789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287a3960 0x5608287a9ec0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f969aa00640 0 client.26239 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.677+0000 7f96be58f740 1 -- 172.21.15.62:0/2085191789 >> 172.21.15.62:0/2085191789 conn(0x56082879e090 msgr2=0x56082879fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.678+0000 7f96be58f740 1 -- 172.21.15.62:0/2085191789 shutdown_connections 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.678+0000 7f96be58f740 1 -- 172.21.15.62:0/2085191789 wait complete. 2024-04-02T14:46:26.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.678+0000 7f96be58f740 20 client.26239 trim_cache size 0 max 16384 2024-04-02T14:46:26.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.702+0000 7f96be58f740 1 Processor -- start 2024-04-02T14:46:26.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.702+0000 7f96be58f740 1 -- start start 2024-04-02T14:46:26.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.703+0000 7f96be58f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a3960 0x5608287b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.703+0000 7f96be58f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a4280 0x5608287ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.703+0000 7f96be58f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 0x56082879fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.703+0000 7f96be58f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41af7f0 con 0x5608287a4280 2024-04-02T14:46:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.703+0000 7f96be58f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b1450 con 0x5608287a3960 2024-04-02T14:46:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.703+0000 7f96be58f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41afec0 con 0x5608287ace60 2024-04-02T14:46:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.703+0000 7f96b8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a3960 0x5608287b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.703+0000 7f96b9800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 0x56082879fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.703+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a4280 0x5608287ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.703+0000 7f96b8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a3960 0x5608287b8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37962/0 (socket says 172.21.15.62:37962) 2024-04-02T14:46:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.703+0000 7f96b8e00640 1 -- 172.21.15.62:0/497578221 learned_addr learned my addr 172.21.15.62:0/497578221 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.704+0000 7f96b8e00640 1 -- 172.21.15.62:0/497578221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 msgr2=0x56082879fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.704+0000 7f96b8e00640 1 --2- 172.21.15.62:0/497578221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 0x56082879fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.704+0000 7f96b8e00640 1 -- 172.21.15.62:0/497578221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a4280 msgr2=0x5608287ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.704+0000 7f96b8e00640 1 --2- 172.21.15.62:0/497578221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a4280 0x5608287ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.704+0000 7f96b8e00640 1 -- 172.21.15.62:0/497578221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608287a3100 con 0x5608287a3960 2024-04-02T14:46:26.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.704+0000 7f96b3e00640 1 --2- 172.21.15.62:0/497578221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a4280 0x5608287ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:26.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.704+0000 7f96b8e00640 1 --2- 172.21.15.62:0/497578221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a3960 0x5608287b8b40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f96a80a51a0 tx=0x7f96a80a5720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b05f05ad1b61ed1c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.704+0000 7f96a5600640 1 -- 172.21.15.62:0/497578221 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80d7070 con 0x5608287a3960 2024-04-02T14:46:26.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.704+0000 7f96a5600640 1 -- 172.21.15.62:0/497578221 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f96a80b5070 con 0x5608287a3960 2024-04-02T14:46:26.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.704+0000 7f96a5600640 1 -- 172.21.15.62:0/497578221 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a809c3c0 con 0x5608287a3960 2024-04-02T14:46:26.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.705+0000 7f96be58f740 1 -- 172.21.15.62:0/497578221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a3960 msgr2=0x5608287b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.705+0000 7f96be58f740 1 --2- 172.21.15.62:0/497578221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a3960 0x5608287b8b40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f96a80a51a0 tx=0x7f96a80a5720 comp rx=0 tx=0).stop 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.705+0000 7f96be58f740 1 -- 172.21.15.62:0/497578221 shutdown_connections 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.705+0000 7f96be58f740 1 --2- 172.21.15.62:0/497578221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 0x56082879fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.705+0000 7f96be58f740 1 --2- 172.21.15.62:0/497578221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a4280 0x5608287ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.705+0000 7f96be58f740 1 --2- 172.21.15.62:0/497578221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a3960 0x5608287b8b40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.705+0000 7f96be58f740 1 -- 172.21.15.62:0/497578221 >> 172.21.15.62:0/497578221 conn(0x56082879e090 msgr2=0x56082879f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.706+0000 7f96be58f740 1 -- 172.21.15.62:0/497578221 shutdown_connections 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.706+0000 7f96be58f740 1 -- 172.21.15.62:0/497578221 wait complete. 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.706+0000 7f96be58f740 1 -- start start 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.706+0000 7f96be58f740 1 Processor -- start 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96be58f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a3960 0x5608287a7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96be58f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a4280 0x5608287a7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96be58f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 0x560828822480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96be58f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b0cc0 con 0x5608287a3960 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96be58f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b0590 con 0x5608287a4280 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96be58f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41ad7f0 con 0x5608287ace60 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96be58f740 10 client.0 _ll_register_callbacks cb 0x7f96a80d4270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96b9800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 0x560828822480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a4280 0x5608287a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96b9800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 0x560828822480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50698/0 (socket says 172.21.15.62:50698) 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96b9800640 1 -- 172.21.15.62:0/585527996 learned_addr learned my addr 172.21.15.62:0/585527996 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96b8e00640 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a3960 0x5608287a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96b9800640 1 -- 172.21.15.62:0/585527996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a4280 msgr2=0x5608287a7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96b9800640 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a4280 0x5608287a7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96b9800640 1 -- 172.21.15.62:0/585527996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a3960 msgr2=0x5608287a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.707+0000 7f96b9800640 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a3960 0x5608287a7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.708+0000 7f96b9800640 1 -- 172.21.15.62:0/585527996 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608288229c0 con 0x5608287ace60 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.708+0000 7f96b9800640 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 0x560828822480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f96ac02e010 tx=0x7f96ac02c100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900bb1791f349746 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.708+0000 7f969aa00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:46:26.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.708+0000 7f969aa00640 1 -- 172.21.15.62:0/585527996 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac051070 con 0x5608287ace60 2024-04-02T14:46:26.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.708+0000 7f969aa00640 1 -- 172.21.15.62:0/585527996 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f96ac004040 con 0x5608287ace60 2024-04-02T14:46:26.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.708+0000 7f96be58f740 1 -- 172.21.15.62:0/585527996 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a80b5070 con 0x5608287ace60 2024-04-02T14:46:26.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.709+0000 7f969aa00640 1 -- 172.21.15.62:0/585527996 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96ac05d050 con 0x5608287ace60 2024-04-02T14:46:26.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.709+0000 7f96be58f740 10 client.26245 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:26.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.709+0000 7f96be58f740 1 -- 172.21.15.62:0/585527996 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560828823c40 con 0x5608287ace60 2024-04-02T14:46:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.709+0000 7f969aa00640 1 -- 172.21.15.62:0/585527996 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f96a80b5070 con 0x5608287ace60 2024-04-02T14:46:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.709+0000 7f969aa00640 1 -- 172.21.15.62:0/585527996 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f96ac040ad0 con 0x5608287ace60 2024-04-02T14:46:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.709+0000 7f96be58f740 10 client.26245 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.709+0000 7f96be58f740 10 client.26245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.709+0000 7f96be58f740 20 client.26245 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.710+0000 7f96be58f740 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x560828824480 0x5608288268a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.710+0000 7f96be58f740 4 client.26245 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:46:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.710+0000 7f96be58f740 1 -- 172.21.15.62:0/585527996 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560828714cb0 con 0x560828824480 2024-04-02T14:46:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.710+0000 7f96b8e00640 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x560828824480 0x5608288268a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.710+0000 7f96b8e00640 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x560828824480 0x5608288268a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a8b4e33f54a9a81 server_cookie=4587399b4b9245b8 in_seq=0 out_seq=0 2024-04-02T14:46:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.710+0000 7f969aa00640 10 client.26245 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f969aa00640 1 -- 172.21.15.62:0/585527996 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x560828714cb0 con 0x560828824480 2024-04-02T14:46:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f969aa00640 10 client.26245 handle_command_reply: tid=0 2024-04-02T14:46:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f96be58f740 1 client.26245 shutdown 2024-04-02T14:46:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f96be58f740 1 -- 172.21.15.62:0/585527996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 msgr2=0x560828822480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f96be58f740 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 0x560828822480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f96ac02e010 tx=0x7f96ac02c100 comp rx=0 tx=0).stop 2024-04-02T14:46:26.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f96be58f740 1 -- 172.21.15.62:0/585527996 shutdown_connections 2024-04-02T14:46:26.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f96be58f740 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x560828824480 0x5608288268a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f96be58f740 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608287ace60 0x560828822480 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f96be58f740 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5608287a4280 0x5608287a7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f96be58f740 1 --2- 172.21.15.62:0/585527996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608287a3960 0x5608287a7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f969aa00640 0 client.26245 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:26.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.711+0000 7f96be58f740 1 -- 172.21.15.62:0/585527996 >> 172.21.15.62:0/585527996 conn(0x56082879e090 msgr2=0x5608287aa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.712+0000 7f96be58f740 1 -- 172.21.15.62:0/585527996 shutdown_connections 2024-04-02T14:46:26.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.712+0000 7f96be58f740 1 -- 172.21.15.62:0/585527996 wait complete. 2024-04-02T14:46:26.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.712+0000 7f96be58f740 20 client.26245 trim_cache size 0 max 16384 2024-04-02T14:46:26.716 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:46:26.716 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "scrub active (560 inodes in the stack)", 2024-04-02T14:46:26.716 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-02T14:46:26.716 INFO:teuthology.orchestra.run.smithi062.stdout: "5f9a02c4-ab9c-4524-bc64-fadb550988f6": { 2024-04-02T14:46:26.716 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-04-02T14:46:26.716 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "5f9a02c4-ab9c-4524-bc64-fadb550988f6", 2024-04-02T14:46:26.716 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-04-02T14:46:26.716 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:46:26.716 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:46:26.717 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:46:26.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.712+0000 7f96ba200640 1 -- 172.21.15.62:0/1308908129 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f969c0a3da0 msgr2=0x7f969c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.712+0000 7f96ba200640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f969c0a3da0 0x7f969c0a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f96ac005e30 tx=0x7f96ac02e040 comp rx=0 tx=0).stop 2024-04-02T14:46:26.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.712+0000 7f96ba200640 1 -- 172.21.15.62:0/1308908129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b4103df0 msgr2=0x7f96b419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:26.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.712+0000 7f96ba200640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b4103df0 0x7f96b419a550 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f96a800ea60 tx=0x7f96a800ef20 comp rx=0 tx=0).stop 2024-04-02T14:46:26.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.713+0000 7f96ba200640 1 -- 172.21.15.62:0/1308908129 shutdown_connections 2024-04-02T14:46:26.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.713+0000 7f96ba200640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f969c0a3da0 0x7f969c0a6260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.713+0000 7f96ba200640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96b419edc0 0x7f96b419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.713+0000 7f96ba200640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96b4108ec0 0x7f96b419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.713+0000 7f96ba200640 1 --2- 172.21.15.62:0/1308908129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b4103df0 0x7f96b419a550 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:26.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.713+0000 7f96ba200640 1 -- 172.21.15.62:0/1308908129 >> 172.21.15.62:0/1308908129 conn(0x7f96b40ff610 msgr2=0x7f96b4100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:26.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.713+0000 7f96ba200640 1 -- 172.21.15.62:0/1308908129 shutdown_connections 2024-04-02T14:46:26.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:26.713+0000 7f96ba200640 1 -- 172.21.15.62:0/1308908129 wait complete. 2024-04-02T14:46:26.731 INFO:tasks.cephfs.filesystem:scrub status for tag:5f9a02c4-ab9c-4524-bc64-fadb550988f6 - {'path': '/', 'tag': '5f9a02c4-ab9c-4524-bc64-fadb550988f6', 'options': 'recursive,force'} 2024-04-02T14:46:27.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: pgmap v76: 97 pgs: 97 active+clean; 153 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 805 op/s 2024-04-02T14:46:27.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:27.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:27.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:27.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:27 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: pgmap v76: 97 pgs: 97 active+clean; 153 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 805 op/s 2024-04-02T14:46:27.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:27.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:27.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:27.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:27.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:27.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:27.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:27.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:27.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:27.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:27 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: pgmap v76: 97 pgs: 97 active+clean; 153 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 805 op/s 2024-04-02T14:46:27.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:27.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:27.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:27.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:27.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:27 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub queued for path: / 2024-04-02T14:46:28.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:28.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:28.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:28.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:28.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:28.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:46:28.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:28 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub queued for path: / 2024-04-02T14:46:28.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:28.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:28.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:28.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:28.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:28 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub queued for path: / 2024-04-02T14:46:28.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:28.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:28.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:28.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:46:28.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:28.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:28 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:46:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.203+0000 7f305be00640 1 -- 172.21.15.62:0/1266078068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30541048d0 msgr2=0x7f3054108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.203+0000 7f305be00640 1 --2- 172.21.15.62:0/1266078068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30541048d0 0x7f3054108910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3040009900 tx=0x7f304002bc30 comp rx=0 tx=0).stop 2024-04-02T14:46:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.204+0000 7f305be00640 1 -- 172.21.15.62:0/1266078068 shutdown_connections 2024-04-02T14:46:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.204+0000 7f305be00640 1 --2- 172.21.15.62:0/1266078068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3054108ff0 0x7f3054110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.204+0000 7f305be00640 1 --2- 172.21.15.62:0/1266078068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30541048d0 0x7f3054108910 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.204+0000 7f305be00640 1 --2- 172.21.15.62:0/1266078068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3054103f20 0x7f3054104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.204+0000 7f305be00640 1 -- 172.21.15.62:0/1266078068 >> 172.21.15.62:0/1266078068 conn(0x7f30540ff780 msgr2=0x7f3054101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.204+0000 7f305be00640 1 -- 172.21.15.62:0/1266078068 shutdown_connections 2024-04-02T14:46:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.204+0000 7f305be00640 1 -- 172.21.15.62:0/1266078068 wait complete. 2024-04-02T14:46:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.205+0000 7f305be00640 1 Processor -- start 2024-04-02T14:46:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.205+0000 7f305be00640 1 -- start start 2024-04-02T14:46:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.205+0000 7f305be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3054103f20 0x7f305419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.205+0000 7f305be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3054108ff0 0x7f305419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.205+0000 7f305be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f305419eed0 0x7f305419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.205+0000 7f305be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3054114170 con 0x7f3054108ff0 2024-04-02T14:46:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.205+0000 7f305be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3054113ff0 con 0x7f305419eed0 2024-04-02T14:46:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.205+0000 7f305be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30541142f0 con 0x7f3054103f20 2024-04-02T14:46:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.206+0000 7f3058a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3054108ff0 0x7f305419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.206+0000 7f3058a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3054108ff0 0x7f305419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34252/0 (socket says 172.21.15.62:34252) 2024-04-02T14:46:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.206+0000 7f3058a00640 1 -- 172.21.15.62:0/442312777 learned_addr learned my addr 172.21.15.62:0/442312777 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.206+0000 7f3059400640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3054103f20 0x7f305419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.206+0000 7f3059e00640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f305419eed0 0x7f305419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.206+0000 7f3059400640 1 -- 172.21.15.62:0/442312777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f305419eed0 msgr2=0x7f305419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.206+0000 7f3059400640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f305419eed0 0x7f305419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.206+0000 7f3059400640 1 -- 172.21.15.62:0/442312777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3054108ff0 msgr2=0x7f305419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.206+0000 7f3059400640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3054108ff0 0x7f305419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.206+0000 7f3059400640 1 -- 172.21.15.62:0/442312777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305419f9c0 con 0x7f3054103f20 2024-04-02T14:46:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.206+0000 7f3058a00640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3054108ff0 0x7f305419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.207+0000 7f3059400640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3054103f20 0x7f305419a610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f304800e9f0 tx=0x7f304800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.207+0000 7f3052000640 1 -- 172.21.15.62:0/442312777 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304800ccc0 con 0x7f3054103f20 2024-04-02T14:46:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.207+0000 7f305be00640 1 -- 172.21.15.62:0/442312777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305419fcb0 con 0x7f3054103f20 2024-04-02T14:46:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.207+0000 7f3052000640 1 -- 172.21.15.62:0/442312777 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3048005020 con 0x7f3054103f20 2024-04-02T14:46:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.207+0000 7f3052000640 1 -- 172.21.15.62:0/442312777 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048004b40 con 0x7f3054103f20 2024-04-02T14:46:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.207+0000 7f305be00640 1 -- 172.21.15.62:0/442312777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30541a7450 con 0x7f3054103f20 2024-04-02T14:46:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.210+0000 7f305eb87740 1 Processor -- start 2024-04-02T14:46:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f305eb87740 1 -- start start 2024-04-02T14:46:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f305eb87740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 0x55f85fa8a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f3052000640 1 -- 172.21.15.62:0/442312777 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3048004ce0 con 0x7f3054103f20 2024-04-02T14:46:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f305eb87740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 0x55f85fa92e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f305eb87740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fa957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f305eb87740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f85f9eed70 con 0x55f85fa933b0 2024-04-02T14:46:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f305eb87740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f85f9972a0 con 0x55f85fa89eb0 2024-04-02T14:46:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f305eb87740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f85f9ead20 con 0x55f85fa8a7d0 2024-04-02T14:46:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f3059e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 0x55f85fa8a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f3059e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 0x55f85fa8a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:38006/0 (socket says 172.21.15.62:38006) 2024-04-02T14:46:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f3059e00640 1 -- 172.21.15.62:0/2919769238 learned_addr learned my addr 172.21.15.62:0/2919769238 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.211+0000 7f3059400640 1 --2- 172.21.15.62:0/2919769238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 0x55f85fa92e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.212+0000 7f3058a00640 1 --2- 172.21.15.62:0/2919769238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fa957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.212+0000 7f3059e00640 1 -- 172.21.15.62:0/2919769238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 msgr2=0x55f85fa92e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.212+0000 7f3059e00640 1 --2- 172.21.15.62:0/2919769238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 0x55f85fa92e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.212+0000 7f3059e00640 1 -- 172.21.15.62:0/2919769238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 msgr2=0x55f85fa957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.212+0000 7f3059e00640 1 --2- 172.21.15.62:0/2919769238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fa957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.212+0000 7f3059e00640 1 -- 172.21.15.62:0/2919769238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85f9ab840 con 0x55f85fa89eb0 2024-04-02T14:46:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.212+0000 7f3058a00640 1 --2- 172.21.15.62:0/2919769238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fa957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.213+0000 7f3059e00640 1 --2- 172.21.15.62:0/2919769238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 0x55f85fa8a290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f304c009f50 tx=0x7f304c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba976c446c2f25e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.213+0000 7f3052000640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f303c0a3dc0 0x7f303c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.213+0000 7f3052000640 1 -- 172.21.15.62:0/442312777 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f30480d07b0 con 0x7f3054103f20 2024-04-02T14:46:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.214+0000 7f3046600640 1 -- 172.21.15.62:0/2919769238 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c004d20 con 0x55f85fa89eb0 2024-04-02T14:46:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.214+0000 7f3046600640 1 -- 172.21.15.62:0/2919769238 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f304c004050 con 0x55f85fa89eb0 2024-04-02T14:46:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.214+0000 7f3046600640 1 -- 172.21.15.62:0/2919769238 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c005260 con 0x55f85fa89eb0 2024-04-02T14:46:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.214+0000 7f3059400640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f303c0a3dc0 0x7f303c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.215+0000 7f305eb87740 1 -- 172.21.15.62:0/2919769238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 msgr2=0x55f85fa8a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.215+0000 7f305eb87740 1 --2- 172.21.15.62:0/2919769238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 0x55f85fa8a290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f304c009f50 tx=0x7f304c0118d0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.215+0000 7f305eb87740 1 -- 172.21.15.62:0/2919769238 shutdown_connections 2024-04-02T14:46:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.215+0000 7f305eb87740 1 --2- 172.21.15.62:0/2919769238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fa957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.215+0000 7f305eb87740 1 --2- 172.21.15.62:0/2919769238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 0x55f85fa92e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.215+0000 7f305eb87740 1 --2- 172.21.15.62:0/2919769238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 0x55f85fa8a290 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.215+0000 7f305eb87740 1 -- 172.21.15.62:0/2919769238 >> 172.21.15.62:0/2919769238 conn(0x55f85fa845e0 msgr2=0x55f85fa849c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.216+0000 7f305eb87740 1 -- 172.21.15.62:0/2919769238 shutdown_connections 2024-04-02T14:46:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.216+0000 7f305eb87740 1 -- 172.21.15.62:0/2919769238 wait complete. 2024-04-02T14:46:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.216+0000 7f3059400640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f303c0a3dc0 0x7f303c0a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55f85fa89980 tx=0x7f30480d6530 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.216+0000 7f305eb87740 1 -- start start 2024-04-02T14:46:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.217+0000 7f305eb87740 1 Processor -- start 2024-04-02T14:46:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.217+0000 7f305eb87740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa89eb0 0x55f85fa903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.217+0000 7f305eb87740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.217+0000 7f305eb87740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fa90e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.217+0000 7f305eb87740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f85fa961a0 con 0x55f85fa933b0 2024-04-02T14:46:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.217+0000 7f305eb87740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f85f9aa660 con 0x55f85fa8a7d0 2024-04-02T14:46:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.217+0000 7f305eb87740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f85fa96320 con 0x55f85fa89eb0 2024-04-02T14:46:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.218+0000 7f305eb87740 10 client.0 _ll_register_callbacks cb 0x55f85f9aa050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.218+0000 7f3058a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.218+0000 7f3058a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:38022/0 (socket says 172.21.15.62:38022) 2024-04-02T14:46:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.218+0000 7f3058a00640 1 -- 172.21.15.62:0/305980709 learned_addr learned my addr 172.21.15.62:0/305980709 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.218+0000 7f3059e00640 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa89eb0 0x55f85fa903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.218+0000 7f3059e00640 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fa90e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.218+0000 7f3058a00640 1 -- 172.21.15.62:0/305980709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa89eb0 msgr2=0x55f85fa903e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.218+0000 7f3058a00640 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa89eb0 0x55f85fa903e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.218+0000 7f3058a00640 1 -- 172.21.15.62:0/305980709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 msgr2=0x55f85fa90e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.218+0000 7f3058a00640 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fa90e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.218+0000 7f3058a00640 1 -- 172.21.15.62:0/305980709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85f9b9e80 con 0x55f85fa8a7d0 2024-04-02T14:46:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.219+0000 7f3059e00640 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fa90e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:46:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.219+0000 7f3058a00640 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3040000c00 tx=0x7f304002cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=342f7d08e2b6786e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.219+0000 7f303aa00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.219+0000 7f303aa00640 1 -- 172.21.15.62:0/305980709 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040043070 con 0x55f85fa8a7d0 2024-04-02T14:46:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.219+0000 7f305eb87740 1 -- 172.21.15.62:0/305980709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f304c004050 con 0x55f85fa8a7d0 2024-04-02T14:46:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.219+0000 7f303aa00640 1 -- 172.21.15.62:0/305980709 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f3040004250 con 0x55f85fa8a7d0 2024-04-02T14:46:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.220+0000 7f303aa00640 1 -- 172.21.15.62:0/305980709 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3040004450 con 0x55f85fa8a7d0 2024-04-02T14:46:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.220+0000 7f305eb87740 10 client.26294 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.220+0000 7f305eb87740 1 -- 172.21.15.62:0/305980709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f85f703210 con 0x55f85fa8a7d0 2024-04-02T14:46:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.220+0000 7f303aa00640 1 -- 172.21.15.62:0/305980709 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f304c004050 con 0x55f85fa8a7d0 2024-04-02T14:46:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.220+0000 7f303aa00640 1 -- 172.21.15.62:0/305980709 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f30400348b0 con 0x55f85fa8a7d0 2024-04-02T14:46:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.220+0000 7f305eb87740 10 client.26294 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.220+0000 7f305eb87740 10 client.26294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.220+0000 7f305eb87740 20 client.26294 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.220+0000 7f305eb87740 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f85faac430 0x55f85faae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.220+0000 7f305eb87740 4 client.26294 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.220+0000 7f305eb87740 1 -- 172.21.15.62:0/305980709 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f85f99ecd0 con 0x55f85faac430 2024-04-02T14:46:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.221+0000 7f3059e00640 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f85faac430 0x55f85faae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.221+0000 7f3059e00640 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f85faac430 0x55f85faae890 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bec511bdfd3212a server_cookie=8e19827ea128d15e in_seq=0 out_seq=0 2024-04-02T14:46:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.221+0000 7f303aa00640 10 client.26294 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f303aa00640 1 -- 172.21.15.62:0/305980709 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f85f99ecd0 con 0x55f85faac430 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f303aa00640 10 client.26294 handle_command_reply: tid=0 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 1 client.26294 shutdown 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 1 -- 172.21.15.62:0/305980709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 msgr2=0x55f85fa90920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3040000c00 tx=0x7f304002cd60 comp rx=0 tx=0).stop 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 1 -- 172.21.15.62:0/305980709 shutdown_connections 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f85faac430 0x55f85faae890 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fa90e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90920 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 1 --2- 172.21.15.62:0/305980709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa89eb0 0x55f85fa903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 1 -- 172.21.15.62:0/305980709 >> 172.21.15.62:0/305980709 conn(0x55f85fa845e0 msgr2=0x55f85fa86330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f303aa00640 0 client.26294 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 1 -- 172.21.15.62:0/305980709 shutdown_connections 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 1 -- 172.21.15.62:0/305980709 wait complete. 2024-04-02T14:46:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.225+0000 7f305eb87740 20 client.26294 trim_cache size 0 max 16384 2024-04-02T14:46:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.254+0000 7f305eb87740 1 Processor -- start 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.254+0000 7f305eb87740 1 -- start start 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.254+0000 7f305eb87740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 0x55f85fa92b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.254+0000 7f305eb87740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.254+0000 7f305eb87740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fab3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.254+0000 7f305eb87740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f30541ab5d0 con 0x55f85fa933b0 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.254+0000 7f305eb87740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f30541ad230 con 0x55f85fa89eb0 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.254+0000 7f305eb87740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30541abca0 con 0x55f85fa8a7d0 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3058a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3058a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50748/0 (socket says 172.21.15.62:50748) 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3058a00640 1 -- 172.21.15.62:0/3872471142 learned_addr learned my addr 172.21.15.62:0/3872471142 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3059e00640 1 --2- 172.21.15.62:0/3872471142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 0x55f85fa92b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3059e00640 1 --2- 172.21.15.62:0/3872471142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fab3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3058a00640 1 -- 172.21.15.62:0/3872471142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 msgr2=0x55f85fa92b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3058a00640 1 --2- 172.21.15.62:0/3872471142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 0x55f85fa92b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3058a00640 1 -- 172.21.15.62:0/3872471142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 msgr2=0x55f85fab3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3058a00640 1 --2- 172.21.15.62:0/3872471142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fab3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3058a00640 1 -- 172.21.15.62:0/3872471142 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85f7cf620 con 0x55f85fa8a7d0 2024-04-02T14:46:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3059e00640 1 --2- 172.21.15.62:0/3872471142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 0x55f85fa92b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.255+0000 7f3058a00640 1 --2- 172.21.15.62:0/3872471142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f304002cb00 tx=0x7f304002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=873d629201318a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.256+0000 7f3045c00640 1 -- 172.21.15.62:0/3872471142 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040056030 con 0x55f85fa8a7d0 2024-04-02T14:46:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.256+0000 7f3045c00640 1 -- 172.21.15.62:0/3872471142 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3040004250 con 0x55f85fa8a7d0 2024-04-02T14:46:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.256+0000 7f3045c00640 1 -- 172.21.15.62:0/3872471142 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040043040 con 0x55f85fa8a7d0 2024-04-02T14:46:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.257+0000 7f305eb87740 1 -- 172.21.15.62:0/3872471142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 msgr2=0x55f85fa90c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.257+0000 7f305eb87740 1 --2- 172.21.15.62:0/3872471142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f304002cb00 tx=0x7f304002bc30 comp rx=0 tx=0).stop 2024-04-02T14:46:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.257+0000 7f305eb87740 1 -- 172.21.15.62:0/3872471142 shutdown_connections 2024-04-02T14:46:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.257+0000 7f305eb87740 1 --2- 172.21.15.62:0/3872471142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fab3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.257+0000 7f305eb87740 1 --2- 172.21.15.62:0/3872471142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa8a7d0 0x55f85fa90c80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.257+0000 7f305eb87740 1 --2- 172.21.15.62:0/3872471142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa89eb0 0x55f85fa92b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.257+0000 7f305eb87740 1 -- 172.21.15.62:0/3872471142 >> 172.21.15.62:0/3872471142 conn(0x55f85fa845e0 msgr2=0x55f85faa7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.257+0000 7f305eb87740 1 -- 172.21.15.62:0/3872471142 shutdown_connections 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.257+0000 7f305eb87740 1 -- 172.21.15.62:0/3872471142 wait complete. 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.257+0000 7f305eb87740 1 -- start start 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.257+0000 7f305eb87740 1 Processor -- start 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f305eb87740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa89eb0 0x55f85fab1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f305eb87740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fab1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f305eb87740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fab2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f305eb87740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f30541acaa0 con 0x55f85fa933b0 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f305eb87740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f30541ac370 con 0x55f85fa8a7d0 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f305eb87740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30541a95d0 con 0x55f85fa89eb0 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f305eb87740 10 client.0 _ll_register_callbacks cb 0x7f304002c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f3058a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fab1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f3058a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fab1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:38044/0 (socket says 172.21.15.62:38044) 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f3058a00640 1 -- 172.21.15.62:0/1449685267 learned_addr learned my addr 172.21.15.62:0/1449685267 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f3059e00640 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa89eb0 0x55f85fab1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f3059e00640 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fab2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.258+0000 7f3058a00640 1 -- 172.21.15.62:0/1449685267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa89eb0 msgr2=0x55f85fab1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f3058a00640 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa89eb0 0x55f85fab1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f3058a00640 1 -- 172.21.15.62:0/1449685267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 msgr2=0x55f85fab2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f3058a00640 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fab2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f3058a00640 1 -- 172.21.15.62:0/1449685267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85fb11810 con 0x55f85fa8a7d0 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f3059e00640 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fab2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f3058a00640 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fab1e50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3040004420 tx=0x7f3040000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f505ff5de56cb8a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f303aa00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f303aa00640 1 -- 172.21.15.62:0/1449685267 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304005f060 con 0x55f85fa8a7d0 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f303aa00640 1 -- 172.21.15.62:0/1449685267 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f3040051db0 con 0x55f85fa8a7d0 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f305eb87740 1 -- 172.21.15.62:0/1449685267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3040004250 con 0x55f85fa8a7d0 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f303aa00640 1 -- 172.21.15.62:0/1449685267 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3040043070 con 0x55f85fa8a7d0 2024-04-02T14:46:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f305eb87740 10 client.26300 fetch_fsmap learned FSMap version 32 2024-04-02T14:46:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.259+0000 7f305eb87740 1 -- 172.21.15.62:0/1449685267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f85fb12a60 con 0x55f85fa8a7d0 2024-04-02T14:46:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.260+0000 7f303aa00640 1 -- 172.21.15.62:0/1449685267 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3040004250 con 0x55f85fa8a7d0 2024-04-02T14:46:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.260+0000 7f303aa00640 1 -- 172.21.15.62:0/1449685267 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f3040064580 con 0x55f85fa8a7d0 2024-04-02T14:46:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.260+0000 7f305eb87740 10 client.26300 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T14:46:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.260+0000 7f305eb87740 10 client.26300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.260+0000 7f305eb87740 20 client.26300 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.260+0000 7f305eb87740 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f85fb131d0 0x55f85fb155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.260+0000 7f305eb87740 4 client.26300 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:46:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.260+0000 7f305eb87740 1 -- 172.21.15.62:0/1449685267 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f85f9fc2b0 con 0x55f85fb131d0 2024-04-02T14:46:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.260+0000 7f3059e00640 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f85fb131d0 0x55f85fb155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.261+0000 7f3059e00640 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f85fb131d0 0x55f85fb155f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99107355a9c8c169 server_cookie=4d9c026dc458125b in_seq=0 out_seq=0 2024-04-02T14:46:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:29.261+0000 7f303aa00640 10 client.26300 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:29.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:29 smithi062 ceph-mon[32116]: pgmap v77: 97 pgs: 97 active+clean; 153 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-02T14:46:29.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:29 smithi113 ceph-mon[36135]: pgmap v77: 97 pgs: 97 active+clean; 153 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-02T14:46:29.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:29 smithi204 ceph-mon[35948]: pgmap v77: 97 pgs: 97 active+clean; 153 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-02T14:46:30.242 INFO:tasks.workunit.client.0.smithi062.stderr:+ sync 2024-04-02T14:46:31.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:31 smithi062 ceph-mon[32116]: pgmap v78: 97 pgs: 97 active+clean; 153 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-02T14:46:31.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:31 smithi113 ceph-mon[36135]: pgmap v78: 97 pgs: 97 active+clean; 153 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-02T14:46:31.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:31 smithi204 ceph-mon[35948]: pgmap v78: 97 pgs: 97 active+clean; 153 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-02T14:46:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.851+0000 7f303aa00640 1 -- 172.21.15.62:0/1449685267 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f85f9fc2b0 con 0x55f85fb131d0 2024-04-02T14:46:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.851+0000 7f303aa00640 10 client.26300 handle_command_reply: tid=0 2024-04-02T14:46:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.851+0000 7f305eb87740 1 client.26300 shutdown 2024-04-02T14:46:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.852+0000 7f305eb87740 1 -- 172.21.15.62:0/1449685267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 msgr2=0x55f85fab1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.852+0000 7f305eb87740 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fab1e50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3040004420 tx=0x7f3040000bc0 comp rx=0 tx=0).stop 2024-04-02T14:46:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.852+0000 7f305eb87740 1 -- 172.21.15.62:0/1449685267 shutdown_connections 2024-04-02T14:46:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.852+0000 7f305eb87740 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55f85fb131d0 0x55f85fb155f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.852+0000 7f305eb87740 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f85fa933b0 0x55f85fab2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.852+0000 7f305eb87740 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f85fa8a7d0 0x55f85fab1e50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.852+0000 7f305eb87740 1 --2- 172.21.15.62:0/1449685267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f85fa89eb0 0x55f85fab1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.852+0000 7f305eb87740 1 -- 172.21.15.62:0/1449685267 >> 172.21.15.62:0/1449685267 conn(0x55f85fa845e0 msgr2=0x55f85fa82db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.852+0000 7f303aa00640 0 client.26300 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.852+0000 7f305eb87740 1 -- 172.21.15.62:0/1449685267 shutdown_connections 2024-04-02T14:46:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.853+0000 7f305eb87740 1 -- 172.21.15.62:0/1449685267 wait complete. 2024-04-02T14:46:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.853+0000 7f305eb87740 20 client.26300 trim_cache size 0 max 16384 2024-04-02T14:46:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:46:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:46:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:46:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:46:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.854+0000 7f305be00640 1 -- 172.21.15.62:0/442312777 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f303c0a3dc0 msgr2=0x7f303c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.854+0000 7f305be00640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f303c0a3dc0 0x7f303c0a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55f85fa89980 tx=0x7f30480d6530 comp rx=0 tx=0).stop 2024-04-02T14:46:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.854+0000 7f305be00640 1 -- 172.21.15.62:0/442312777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3054103f20 msgr2=0x7f305419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.854+0000 7f305be00640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3054103f20 0x7f305419a610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f304800e9f0 tx=0x7f304800eeb0 comp rx=0 tx=0).stop 2024-04-02T14:46:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.854+0000 7f305be00640 1 -- 172.21.15.62:0/442312777 shutdown_connections 2024-04-02T14:46:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.854+0000 7f305be00640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f303c0a3dc0 0x7f303c0a6280 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.854+0000 7f305be00640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f305419eed0 0x7f305419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.854+0000 7f305be00640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3054108ff0 0x7f305419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.854+0000 7f305be00640 1 --2- 172.21.15.62:0/442312777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3054103f20 0x7f305419a610 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.854+0000 7f305be00640 1 -- 172.21.15.62:0/442312777 >> 172.21.15.62:0/442312777 conn(0x7f30540ff780 msgr2=0x7f3054101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.855+0000 7f305be00640 1 -- 172.21.15.62:0/442312777 shutdown_connections 2024-04-02T14:46:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:31.855+0000 7f305be00640 1 -- 172.21.15.62:0/442312777 wait complete. 2024-04-02T14:46:33.002 INFO:tasks.workunit.client.0.smithi062.stderr:+ tar c linux-4.0.5 2024-04-02T14:46:33.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:33 smithi062 ceph-mon[32116]: pgmap v79: 97 pgs: 97 active+clean; 175 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 1.18k op/s 2024-04-02T14:46:33.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:33 smithi113 ceph-mon[36135]: pgmap v79: 97 pgs: 97 active+clean; 175 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 1.18k op/s 2024-04-02T14:46:33.747 INFO:tasks.workunit.client.0.smithi062.stderr:+ rm -rf linux-4.0.5 2024-04-02T14:46:33.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:33 smithi204 ceph-mon[35948]: pgmap v79: 97 pgs: 97 active+clean; 175 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 1.18k op/s 2024-04-02T14:46:35.556 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:35 smithi204 ceph-mon[35948]: pgmap v80: 97 pgs: 97 active+clean; 175 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 1.05k op/s 2024-04-02T14:46:35.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:35 smithi062 ceph-mon[32116]: pgmap v80: 97 pgs: 97 active+clean; 175 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 1.05k op/s 2024-04-02T14:46:35.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:35 smithi113 ceph-mon[36135]: pgmap v80: 97 pgs: 97 active+clean; 175 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 1.05k op/s 2024-04-02T14:46:37.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:37 smithi062 ceph-mon[32116]: pgmap v81: 97 pgs: 97 active+clean; 158 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 1.12k op/s 2024-04-02T14:46:37.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:37 smithi113 ceph-mon[36135]: pgmap v81: 97 pgs: 97 active+clean; 158 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 1.12k op/s 2024-04-02T14:46:37.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:37 smithi204 ceph-mon[35948]: pgmap v81: 97 pgs: 97 active+clean; 158 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 1.12k op/s 2024-04-02T14:46:39.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:39 smithi062 ceph-mon[32116]: pgmap v82: 97 pgs: 97 active+clean; 160 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 1.05k op/s 2024-04-02T14:46:39.616 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:39 smithi113 ceph-mon[36135]: pgmap v82: 97 pgs: 97 active+clean; 160 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 1.05k op/s 2024-04-02T14:46:39.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:39 smithi204 ceph-mon[35948]: pgmap v82: 97 pgs: 97 active+clean; 160 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 1.05k op/s 2024-04-02T14:46:40.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:40 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:40.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:40 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:40.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:40 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:41.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:41 smithi113 ceph-mon[36135]: pgmap v83: 97 pgs: 97 active+clean; 160 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 1.05k op/s 2024-04-02T14:46:41.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:41 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:41.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:41 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:46:41.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:41 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:46:41.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:41 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:46:41.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:41 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:46:41.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:41 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:46:41.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:41 smithi204 ceph-mon[35948]: pgmap v83: 97 pgs: 97 active+clean; 160 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 1.05k op/s 2024-04-02T14:46:41.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:41 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:41.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:41 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:46:41.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:41 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:46:41.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:41 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:46:41.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:41 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:46:41.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:41 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:46:41.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:41 smithi062 ceph-mon[32116]: pgmap v83: 97 pgs: 97 active+clean; 160 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 1.05k op/s 2024-04-02T14:46:41.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:41 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:41.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:41 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:46:41.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:41 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:46:41.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:41 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:46:41.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:41 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:46:41.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:41 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:46:43.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:43 smithi113 ceph-mon[36135]: pgmap v84: 97 pgs: 97 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 1.19k op/s 2024-04-02T14:46:43.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:43 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:43.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:43 smithi204 ceph-mon[35948]: pgmap v84: 97 pgs: 97 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 1.19k op/s 2024-04-02T14:46:43.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:43 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:43.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:43 smithi062 ceph-mon[32116]: pgmap v84: 97 pgs: 97 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 1.19k op/s 2024-04-02T14:46:43.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:43 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.012+0000 7f8ab5200640 1 -- 172.21.15.62:0/196677022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ab0069180 msgr2=0x7f8ab00695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.012+0000 7f8ab5200640 1 --2- 172.21.15.62:0/196677022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ab0069180 0x7f8ab00695e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009cd0 tx=0x7f8a9802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.012+0000 7f8ab5200640 1 -- 172.21.15.62:0/196677022 shutdown_connections 2024-04-02T14:46:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.012+0000 7f8ab5200640 1 --2- 172.21.15.62:0/196677022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab0069b20 0x7f8ab010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.012+0000 7f8ab5200640 1 --2- 172.21.15.62:0/196677022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ab0069180 0x7f8ab00695e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.012+0000 7f8ab5200640 1 --2- 172.21.15.62:0/196677022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0107b60 0x7f8ab0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.012+0000 7f8ab5200640 1 -- 172.21.15.62:0/196677022 >> 172.21.15.62:0/196677022 conn(0x7f8ab0077850 msgr2=0x7f8ab0077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.012+0000 7f8ab5200640 1 -- 172.21.15.62:0/196677022 shutdown_connections 2024-04-02T14:46:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.012+0000 7f8ab5200640 1 -- 172.21.15.62:0/196677022 wait complete. 2024-04-02T14:46:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.014+0000 7f8ab5200640 1 Processor -- start 2024-04-02T14:46:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.014+0000 7f8ab5200640 1 -- start start 2024-04-02T14:46:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.014+0000 7f8ab5200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab0069180 0x7f8ab0100d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.014+0000 7f8ab5200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ab0069b20 0x7f8ab0101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.014+0000 7f8ab5200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0107b60 0x7f8ab01017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.014+0000 7f8ab5200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab0111fa0 con 0x7f8ab0107b60 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.014+0000 7f8ab5200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab0111e20 con 0x7f8ab0069b20 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.014+0000 7f8ab5200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab0112120 con 0x7f8ab0069180 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.014+0000 7f8aad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ab0069b20 0x7f8ab0101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.014+0000 7f8aad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ab0069b20 0x7f8ab0101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:46800/0 (socket says 172.21.15.62:46800) 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.014+0000 7f8aad400640 1 -- 172.21.15.62:0/460959741 learned_addr learned my addr 172.21.15.62:0/460959741 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.015+0000 7f8aade00640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab0069180 0x7f8ab0100d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.015+0000 7f8aae800640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0107b60 0x7f8ab01017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.015+0000 7f8aad400640 1 -- 172.21.15.62:0/460959741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab0069180 msgr2=0x7f8ab0100d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.015+0000 7f8aad400640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab0069180 0x7f8ab0100d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.015+0000 7f8aad400640 1 -- 172.21.15.62:0/460959741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0107b60 msgr2=0x7f8ab01017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.015+0000 7f8aad400640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0107b60 0x7f8ab01017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.015+0000 7f8aad400640 1 -- 172.21.15.62:0/460959741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab01a68a0 con 0x7f8ab0069b20 2024-04-02T14:46:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.015+0000 7f8aae800640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0107b60 0x7f8ab01017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.015+0000 7f8aad400640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ab0069b20 0x7f8ab0101260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8a98004af0 tx=0x7f8a98034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.015+0000 7f8aa6a00640 1 -- 172.21.15.62:0/460959741 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98039040 con 0x7f8ab0069b20 2024-04-02T14:46:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.016+0000 7f8ab5200640 1 -- 172.21.15.62:0/460959741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab01a6b30 con 0x7f8ab0069b20 2024-04-02T14:46:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.016+0000 7f8aa6a00640 1 -- 172.21.15.62:0/460959741 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a98044070 con 0x7f8ab0069b20 2024-04-02T14:46:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.016+0000 7f8aa6a00640 1 -- 172.21.15.62:0/460959741 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98004440 con 0x7f8ab0069b20 2024-04-02T14:46:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.016+0000 7f8ab5200640 1 -- 172.21.15.62:0/460959741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab01a7020 con 0x7f8ab0069b20 2024-04-02T14:46:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.019+0000 7f8aa6a00640 1 -- 172.21.15.62:0/460959741 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8a98032870 con 0x7f8ab0069b20 2024-04-02T14:46:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.019+0000 7f8ab7e2a740 1 Processor -- start 2024-04-02T14:46:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.019+0000 7f8ab7e2a740 1 -- start start 2024-04-02T14:46:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb418e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb4196e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ecb40f2d70 con 0x55ecb418deb0 2024-04-02T14:46:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ecb409b2a0 con 0x55ecb418e7d0 2024-04-02T14:46:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ecb40eed20 con 0x55ecb41973b0 2024-04-02T14:46:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8aae800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb418e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8aade00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb4196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8aae800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb418e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55342/0 (socket says 172.21.15.62:55342) 2024-04-02T14:46:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8aae800640 1 -- 172.21.15.62:0/2415969407 learned_addr learned my addr 172.21.15.62:0/2415969407 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8aad400640 1 --2- 172.21.15.62:0/2415969407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8aade00640 1 -- 172.21.15.62:0/2415969407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 msgr2=0x55ecb41997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8aade00640 1 --2- 172.21.15.62:0/2415969407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb41997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8aade00640 1 -- 172.21.15.62:0/2415969407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 msgr2=0x55ecb418e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8aade00640 1 --2- 172.21.15.62:0/2415969407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb418e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8aade00640 1 -- 172.21.15.62:0/2415969407 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb40af840 con 0x55ecb418e7d0 2024-04-02T14:46:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.020+0000 7f8aae800640 1 --2- 172.21.15.62:0/2415969407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb418e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.021+0000 7f8aa6a00640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a900a3dc0 0x7f8a900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.021+0000 7f8aade00640 1 --2- 172.21.15.62:0/2415969407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb4196e70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009c20 tx=0x7f8aa0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=874c64bf33a53289 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.021+0000 7f8aa6a00640 1 -- 172.21.15.62:0/460959741 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8a980f2ce0 con 0x7f8ab0069b20 2024-04-02T14:46:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.021+0000 7f8aae800640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a900a3dc0 0x7f8a900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.021+0000 7f8a9f400640 1 -- 172.21.15.62:0/2415969407 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0004050 con 0x55ecb418e7d0 2024-04-02T14:46:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.021+0000 7f8a9f400640 1 -- 172.21.15.62:0/2415969407 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8aa0005520 con 0x55ecb418e7d0 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.021+0000 7f8a9f400640 1 -- 172.21.15.62:0/2415969407 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0004e00 con 0x55ecb418e7d0 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.022+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/2415969407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 msgr2=0x55ecb4196e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.022+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/2415969407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb4196e70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009c20 tx=0x7f8aa0011810 comp rx=0 tx=0).stop 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.022+0000 7f8aae800640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a900a3dc0 0x7f8a900a6280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800bfd0 tx=0x7f8aa800b5b0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.022+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/2415969407 shutdown_connections 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.022+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/2415969407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb41997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.022+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/2415969407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb4196e70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.022+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/2415969407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb418e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.022+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/2415969407 >> 172.21.15.62:0/2415969407 conn(0x55ecb41885e0 msgr2=0x55ecb41889c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.022+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/2415969407 shutdown_connections 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.022+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/2415969407 wait complete. 2024-04-02T14:46:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.023+0000 7f8ab7e2a740 1 -- start start 2024-04-02T14:46:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.023+0000 7f8ab7e2a740 1 Processor -- start 2024-04-02T14:46:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.023+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb4194400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.023+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb4194940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.023+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb4194e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.023+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ecb419a1a0 con 0x55ecb418deb0 2024-04-02T14:46:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.023+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ecb40ae660 con 0x55ecb418e7d0 2024-04-02T14:46:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.023+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ecb419a320 con 0x55ecb41973b0 2024-04-02T14:46:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8ab7e2a740 10 client.0 _ll_register_callbacks cb 0x55ecb40ae050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8aade00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb4194400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8aad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb4194940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8aad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb4194940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:46816/0 (socket says 172.21.15.62:46816) 2024-04-02T14:46:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8aad400640 1 -- 172.21.15.62:0/3556117889 learned_addr learned my addr 172.21.15.62:0/3556117889 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8aae800640 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb4194e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8aad400640 1 -- 172.21.15.62:0/3556117889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 msgr2=0x55ecb4194e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8aad400640 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb4194e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8aad400640 1 -- 172.21.15.62:0/3556117889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 msgr2=0x55ecb4194400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8aad400640 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb4194400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8aad400640 1 -- 172.21.15.62:0/3556117889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb40bde80 con 0x55ecb418e7d0 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8aad400640 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb4194940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a980f0590 tx=0x7f8a98005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69154220c0fd72ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8a97e00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.024+0000 7f8a97e00640 1 -- 172.21.15.62:0/3556117889 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a980be8c0 con 0x55ecb418e7d0 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8a97e00640 1 -- 172.21.15.62:0/3556117889 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8a980047e0 con 0x55ecb418e7d0 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/3556117889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa0005520 con 0x55ecb418e7d0 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8a97e00640 1 -- 172.21.15.62:0/3556117889 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8a980f08b0 con 0x55ecb418e7d0 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8ab7e2a740 10 client.26312 fetch_fsmap learned FSMap version 35 2024-04-02T14:46:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/3556117889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ecb3e07210 con 0x55ecb418e7d0 2024-04-02T14:46:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8a97e00640 1 -- 172.21.15.62:0/3556117889 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8aa0005520 con 0x55ecb418e7d0 2024-04-02T14:46:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8a97e00640 1 -- 172.21.15.62:0/3556117889 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 35) ==== 8390+0+0 (secure 0 0 0) 0x7f8a9803c030 con 0x55ecb418e7d0 2024-04-02T14:46:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8ab7e2a740 10 client.26312 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T14:46:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8ab7e2a740 10 client.26312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8ab7e2a740 20 client.26312 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ecb41a77c0 0x55ecb41a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8ab7e2a740 4 client.26312 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.025+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/3556117889 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecb40a2cd0 con 0x55ecb41a77c0 2024-04-02T14:46:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.026+0000 7f8aade00640 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ecb41a77c0 0x55ecb41a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.026+0000 7f8aade00640 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ecb41a77c0 0x55ecb41a9c60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab4dd4a458804a server_cookie=eda7775fe48267a9 in_seq=0 out_seq=0 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.026+0000 7f8a97e00640 10 client.26312 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.027+0000 7f8a97e00640 1 -- 172.21.15.62:0/3556117889 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecb40a2cd0 con 0x55ecb41a77c0 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.027+0000 7f8a97e00640 10 client.26312 handle_command_reply: tid=0 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.027+0000 7f8ab7e2a740 1 client.26312 shutdown 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.027+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/3556117889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 msgr2=0x55ecb4194940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.027+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb4194940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a980f0590 tx=0x7f8a98005ef0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.028+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/3556117889 shutdown_connections 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.028+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ecb41a77c0 0x55ecb41a9c60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.028+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb4194e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.028+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb4194940 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.028+0000 7f8a97e00640 0 client.26312 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.028+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/3556117889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb4194400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.028+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/3556117889 >> 172.21.15.62:0/3556117889 conn(0x55ecb41885e0 msgr2=0x55ecb418a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.028+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/3556117889 shutdown_connections 2024-04-02T14:46:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.028+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/3556117889 wait complete. 2024-04-02T14:46:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.028+0000 7f8ab7e2a740 20 client.26312 trim_cache size 0 max 16384 2024-04-02T14:46:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.044+0000 7f8ab7e2a740 1 Processor -- start 2024-04-02T14:46:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.044+0000 7f8ab7e2a740 1 -- start start 2024-04-02T14:46:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.044+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb418deb0 0x55ecb4196b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.044+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418e7d0 0x55ecb41a3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.044+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb41973b0 0x55ecb41af570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.044+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab01af8b0 con 0x55ecb418e7d0 2024-04-02T14:46:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.044+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab01b1570 con 0x55ecb41973b0 2024-04-02T14:46:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.044+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab01affb0 con 0x55ecb418deb0 2024-04-02T14:46:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aae800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb41973b0 0x55ecb41af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aad400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418e7d0 0x55ecb41a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aae800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb41973b0 0x55ecb41af570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:46832/0 (socket says 172.21.15.62:46832) 2024-04-02T14:46:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aae800640 1 -- 172.21.15.62:0/799695287 learned_addr learned my addr 172.21.15.62:0/799695287 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aade00640 1 --2- 172.21.15.62:0/799695287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb418deb0 0x55ecb4196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aae800640 1 -- 172.21.15.62:0/799695287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb418deb0 msgr2=0x55ecb4196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aae800640 1 --2- 172.21.15.62:0/799695287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb418deb0 0x55ecb4196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aae800640 1 -- 172.21.15.62:0/799695287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418e7d0 msgr2=0x55ecb41a3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aae800640 1 --2- 172.21.15.62:0/799695287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418e7d0 0x55ecb41a3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aae800640 1 -- 172.21.15.62:0/799695287 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb3ed3620 con 0x55ecb41973b0 2024-04-02T14:46:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aade00640 1 --2- 172.21.15.62:0/799695287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb418deb0 0x55ecb4196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aad400640 1 --2- 172.21.15.62:0/799695287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418e7d0 0x55ecb41a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.045+0000 7f8aae800640 1 --2- 172.21.15.62:0/799695287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb41973b0 0x55ecb41af570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800b9a0 tx=0x7f8aa8015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef37bbb0ab5254a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.046+0000 7f8a9ea00640 1 -- 172.21.15.62:0/799695287 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8010cd0 con 0x55ecb41973b0 2024-04-02T14:46:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.046+0000 7f8a9ea00640 1 -- 172.21.15.62:0/799695287 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8aa8010e70 con 0x55ecb41973b0 2024-04-02T14:46:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.046+0000 7f8a9ea00640 1 -- 172.21.15.62:0/799695287 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa801e470 con 0x55ecb41973b0 2024-04-02T14:46:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.046+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/799695287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb41973b0 msgr2=0x55ecb41af570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.046+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/799695287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb41973b0 0x55ecb41af570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800b9a0 tx=0x7f8aa8015000 comp rx=0 tx=0).stop 2024-04-02T14:46:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.046+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/799695287 shutdown_connections 2024-04-02T14:46:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.046+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/799695287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb41973b0 0x55ecb41af570 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.046+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/799695287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418e7d0 0x55ecb41a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.046+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/799695287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb418deb0 0x55ecb4196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.046+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/799695287 >> 172.21.15.62:0/799695287 conn(0x55ecb41885e0 msgr2=0x55ecb418a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.047+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/799695287 shutdown_connections 2024-04-02T14:46:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.047+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/799695287 wait complete. 2024-04-02T14:46:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.047+0000 7f8ab7e2a740 1 -- start start 2024-04-02T14:46:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.047+0000 7f8ab7e2a740 1 Processor -- start 2024-04-02T14:46:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.047+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb4191270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.047+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb41917b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8ab7e2a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb4191cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab01b0de0 con 0x55ecb418deb0 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab01b06b0 con 0x55ecb418e7d0 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8ab7e2a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab01ad890 con 0x55ecb41973b0 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8ab7e2a740 10 client.0 _ll_register_callbacks cb 0x7f8a98005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aade00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb4191270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb41917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb41917b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:46838/0 (socket says 172.21.15.62:46838) 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aad400640 1 -- 172.21.15.62:0/935081693 learned_addr learned my addr 172.21.15.62:0/935081693 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aade00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb4191270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55368/0 (socket says 172.21.15.62:55368) 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aae800640 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb4191cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aad400640 1 -- 172.21.15.62:0/935081693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 msgr2=0x55ecb4191cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aad400640 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb4191cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aad400640 1 -- 172.21.15.62:0/935081693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 msgr2=0x55ecb4191270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aad400640 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb4191270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aad400640 1 -- 172.21.15.62:0/935081693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb420cfe0 con 0x55ecb418e7d0 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.048+0000 7f8aade00640 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb4191270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8aad400640 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb41917b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a980bb700 tx=0x7f8a980d8370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=924fa4910c3e2689 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8a97e00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8a97e00640 1 -- 172.21.15.62:0/935081693 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a980f5070 con 0x55ecb418e7d0 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8a97e00640 1 -- 172.21.15.62:0/935081693 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8a980f0b70 con 0x55ecb418e7d0 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/935081693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa8010e70 con 0x55ecb418e7d0 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8a97e00640 1 -- 172.21.15.62:0/935081693 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8a980be6e0 con 0x55ecb418e7d0 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8ab7e2a740 10 client.26318 fetch_fsmap learned FSMap version 35 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/935081693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ecb420e290 con 0x55ecb418e7d0 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8a97e00640 1 -- 172.21.15.62:0/935081693 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8aa8010e70 con 0x55ecb418e7d0 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8a97e00640 1 -- 172.21.15.62:0/935081693 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 35) ==== 8390+0+0 (secure 0 0 0) 0x7f8a98004b40 con 0x55ecb418e7d0 2024-04-02T14:46:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8ab7e2a740 10 client.26318 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T14:46:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8ab7e2a740 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8ab7e2a740 20 client.26318 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ecb41a77c0 0x55ecb42108f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8ab7e2a740 4 client.26318 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:46:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.049+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/935081693 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ecb41002b0 con 0x55ecb41a77c0 2024-04-02T14:46:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.050+0000 7f8aade00640 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ecb41a77c0 0x55ecb42108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.050+0000 7f8aade00640 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ecb41a77c0 0x55ecb42108f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77c0502c06d697 server_cookie=d44d86e90a75a49c in_seq=0 out_seq=0 2024-04-02T14:46:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.050+0000 7f8a97e00640 10 client.26318 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8a97e00640 1 -- 172.21.15.62:0/935081693 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ecb41002b0 con 0x55ecb41a77c0 2024-04-02T14:46:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8a97e00640 10 client.26318 handle_command_reply: tid=0 2024-04-02T14:46:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8ab7e2a740 1 client.26318 shutdown 2024-04-02T14:46:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/935081693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 msgr2=0x55ecb41917b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb41917b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a980bb700 tx=0x7f8a980d8370 comp rx=0 tx=0).stop 2024-04-02T14:46:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/935081693 shutdown_connections 2024-04-02T14:46:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55ecb41a77c0 0x55ecb42108f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecb41973b0 0x55ecb4191cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ecb418e7d0 0x55ecb41917b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8ab7e2a740 1 --2- 172.21.15.62:0/935081693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb418deb0 0x55ecb4191270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8a97e00640 0 client.26318 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.417+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/935081693 >> 172.21.15.62:0/935081693 conn(0x55ecb41885e0 msgr2=0x55ecb41943a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.418+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/935081693 shutdown_connections 2024-04-02T14:46:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.418+0000 7f8ab7e2a740 1 -- 172.21.15.62:0/935081693 wait complete. 2024-04-02T14:46:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.418+0000 7f8ab7e2a740 20 client.26318 trim_cache size 0 max 16384 2024-04-02T14:46:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:46:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:46:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:46:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.419+0000 7f8ab5200640 1 -- 172.21.15.62:0/460959741 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a900a3dc0 msgr2=0x7f8a900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.419+0000 7f8ab5200640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a900a3dc0 0x7f8a900a6280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800bfd0 tx=0x7f8aa800b5b0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.419+0000 7f8ab5200640 1 -- 172.21.15.62:0/460959741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ab0069b20 msgr2=0x7f8ab0101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.419+0000 7f8ab5200640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ab0069b20 0x7f8ab0101260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8a98004af0 tx=0x7f8a98034670 comp rx=0 tx=0).stop 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.419+0000 7f8ab5200640 1 -- 172.21.15.62:0/460959741 shutdown_connections 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.419+0000 7f8ab5200640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f8a900a3dc0 0x7f8a900a6280 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.419+0000 7f8ab5200640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0107b60 0x7f8ab01017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.419+0000 7f8ab5200640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ab0069b20 0x7f8ab0101260 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.419+0000 7f8ab5200640 1 --2- 172.21.15.62:0/460959741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab0069180 0x7f8ab0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.419+0000 7f8ab5200640 1 -- 172.21.15.62:0/460959741 >> 172.21.15.62:0/460959741 conn(0x7f8ab0077850 msgr2=0x7f8ab00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.420+0000 7f8ab5200640 1 -- 172.21.15.62:0/460959741 shutdown_connections 2024-04-02T14:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:45.420+0000 7f8ab5200640 1 -- 172.21.15.62:0/460959741 wait complete. 2024-04-02T14:46:45.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:45 smithi113 ceph-mon[36135]: pgmap v85: 97 pgs: 97 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 245 op/s 2024-04-02T14:46:45.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:45 smithi204 ceph-mon[35948]: pgmap v85: 97 pgs: 97 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 245 op/s 2024-04-02T14:46:45.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:45 smithi062 ceph-mon[32116]: pgmap v85: 97 pgs: 97 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 245 op/s 2024-04-02T14:46:47.615 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:47 smithi113 ceph-mon[36135]: pgmap v86: 97 pgs: 97 active+clean; 139 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 535 op/s 2024-04-02T14:46:47.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:47 smithi204 ceph-mon[35948]: pgmap v86: 97 pgs: 97 active+clean; 139 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 535 op/s 2024-04-02T14:46:47.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:47 smithi062 ceph-mon[32116]: pgmap v86: 97 pgs: 97 active+clean; 139 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 535 op/s 2024-04-02T14:46:48.039 INFO:tasks.workunit.client.0.smithi062.stderr:+ tar xf doc.tar 2024-04-02T14:46:49.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:49 smithi204 ceph-mon[35948]: pgmap v87: 97 pgs: 97 active+clean; 137 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 520 op/s 2024-04-02T14:46:49.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:49 smithi062 ceph-mon[32116]: pgmap v87: 97 pgs: 97 active+clean; 137 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 520 op/s 2024-04-02T14:46:49.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:49 smithi113 ceph-mon[36135]: pgmap v87: 97 pgs: 97 active+clean; 137 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 520 op/s 2024-04-02T14:46:51.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:51 smithi204 ceph-mon[35948]: pgmap v88: 97 pgs: 97 active+clean; 137 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 481 op/s 2024-04-02T14:46:51.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:51 smithi062 ceph-mon[32116]: pgmap v88: 97 pgs: 97 active+clean; 137 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 481 op/s 2024-04-02T14:46:51.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:51 smithi113 ceph-mon[36135]: pgmap v88: 97 pgs: 97 active+clean; 137 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 481 op/s 2024-04-02T14:46:53.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:53 smithi062 ceph-mon[32116]: pgmap v89: 97 pgs: 97 active+clean; 134 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 590 op/s 2024-04-02T14:46:53.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:53 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:53.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:53 smithi204 ceph-mon[35948]: pgmap v89: 97 pgs: 97 active+clean; 134 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 590 op/s 2024-04-02T14:46:53.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:53 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:53.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:53 smithi113 ceph-mon[36135]: pgmap v89: 97 pgs: 97 active+clean; 134 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 590 op/s 2024-04-02T14:46:53.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:53 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:46:55.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:55 smithi204 ceph-mon[35948]: pgmap v90: 97 pgs: 97 active+clean; 134 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 446 op/s 2024-04-02T14:46:55.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:55 smithi062 ceph-mon[32116]: pgmap v90: 97 pgs: 97 active+clean; 134 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 446 op/s 2024-04-02T14:46:55.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:55 smithi113 ceph-mon[36135]: pgmap v90: 97 pgs: 97 active+clean; 134 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 446 op/s 2024-04-02T14:46:56.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:46:56.733 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:46:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.974+0000 7fcdc9200640 1 -- 172.21.15.62:0/1872841355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc410d3e0 msgr2=0x7fcdc4114fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.974+0000 7fcdc9200640 1 --2- 172.21.15.62:0/1872841355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc410d3e0 0x7fcdc4114fb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009ca0 tx=0x7fcdbc02c100 comp rx=0 tx=0).stop 2024-04-02T14:46:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.974+0000 7fcdc9200640 1 -- 172.21.15.62:0/1872841355 shutdown_connections 2024-04-02T14:46:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.974+0000 7fcdc9200640 1 --2- 172.21.15.62:0/1872841355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc410d3e0 0x7fcdc4114fb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.974+0000 7fcdc9200640 1 --2- 172.21.15.62:0/1872841355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc4108cc0 0x7fcdc410cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.974+0000 7fcdc9200640 1 --2- 172.21.15.62:0/1872841355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc4108310 0x7fcdc41086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.974+0000 7fcdc9200640 1 -- 172.21.15.62:0/1872841355 >> 172.21.15.62:0/1872841355 conn(0x7fcdc4076040 msgr2=0x7fcdc4078460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.974+0000 7fcdc9200640 1 -- 172.21.15.62:0/1872841355 shutdown_connections 2024-04-02T14:46:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.974+0000 7fcdc9200640 1 -- 172.21.15.62:0/1872841355 wait complete. 2024-04-02T14:46:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.975+0000 7fcdc9200640 1 Processor -- start 2024-04-02T14:46:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.975+0000 7fcdc9200640 1 -- start start 2024-04-02T14:46:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.976+0000 7fcdc9200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc4108310 0x7fcdc41a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.976+0000 7fcdc9200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc410d3e0 0x7fcdc41a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.976+0000 7fcdc9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc419e8d0 0x7fcdc419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.976+0000 7fcdc9200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41185d0 con 0x7fcdc419e8d0 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.976+0000 7fcdc9200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc4118450 con 0x7fcdc410d3e0 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.976+0000 7fcdc9200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc4118750 con 0x7fcdc4108310 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.976+0000 7fcdc2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc419e8d0 0x7fcdc419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.976+0000 7fcdc1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc4108310 0x7fcdc41a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.976+0000 7fcdc1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc4108310 0x7fcdc41a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:36218/0 (socket says 172.21.15.62:36218) 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.976+0000 7fcdc1e00640 1 -- 172.21.15.62:0/425505926 learned_addr learned my addr 172.21.15.62:0/425505926 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.976+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc410d3e0 0x7fcdc41a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.977+0000 7fcdc1e00640 1 -- 172.21.15.62:0/425505926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc410d3e0 msgr2=0x7fcdc41a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.977+0000 7fcdc1e00640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc410d3e0 0x7fcdc41a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.977+0000 7fcdc1e00640 1 -- 172.21.15.62:0/425505926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc419e8d0 msgr2=0x7fcdc419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.977+0000 7fcdc1e00640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc419e8d0 0x7fcdc419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.977+0000 7fcdc1e00640 1 -- 172.21.15.62:0/425505926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc419f3f0 con 0x7fcdc4108310 2024-04-02T14:46:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.977+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc410d3e0 0x7fcdc41a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.977+0000 7fcdc1e00640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc4108310 0x7fcdc41a46d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e9f0 tx=0x7fcdb400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.977+0000 7fcdbaa00640 1 -- 172.21.15.62:0/425505926 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400cd80 con 0x7fcdc4108310 2024-04-02T14:46:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.977+0000 7fcdbaa00640 1 -- 172.21.15.62:0/425505926 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcdb4010930 con 0x7fcdc4108310 2024-04-02T14:46:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.977+0000 7fcdc9200640 1 -- 172.21.15.62:0/425505926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc419f6e0 con 0x7fcdc4108310 2024-04-02T14:46:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.977+0000 7fcdbaa00640 1 -- 172.21.15.62:0/425505926 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4018b00 con 0x7fcdc4108310 2024-04-02T14:46:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.978+0000 7fcdc9200640 1 -- 172.21.15.62:0/425505926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc41ab8e0 con 0x7fcdc4108310 2024-04-02T14:46:56.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdcbe4e740 1 Processor -- start 2024-04-02T14:46:56.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdcbe4e740 1 -- start start 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdbaa00640 1 -- 172.21.15.62:0/425505926 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fcdb4018ca0 con 0x7fcdc4108310 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70e7b0 0x55cd6f70eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70f0d0 0x55cd6f717770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f71a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cd6f672770 con 0x55cd6f717cb0 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cd6f61a700 con 0x55cd6f70e7b0 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cd6f66e720 con 0x55cd6f70f0d0 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdc2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70e7b0 0x55cd6f70eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdc2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70e7b0 0x55cd6f70eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55508/0 (socket says 172.21.15.62:55508) 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.981+0000 7fcdc2800640 1 -- 172.21.15.62:0/919104362 learned_addr learned my addr 172.21.15.62:0/919104362 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.982+0000 7fcdc1e00640 1 --2- 172.21.15.62:0/919104362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70f0d0 0x55cd6f717770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.982+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/919104362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f71a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.982+0000 7fcdc2800640 1 -- 172.21.15.62:0/919104362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70f0d0 msgr2=0x55cd6f717770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.982+0000 7fcdc2800640 1 --2- 172.21.15.62:0/919104362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70f0d0 0x55cd6f717770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.982+0000 7fcdc2800640 1 -- 172.21.15.62:0/919104362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 msgr2=0x55cd6f71a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.982+0000 7fcdc2800640 1 --2- 172.21.15.62:0/919104362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f71a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.982+0000 7fcdc2800640 1 -- 172.21.15.62:0/919104362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd6f4507d0 con 0x55cd6f70e7b0 2024-04-02T14:46:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.982+0000 7fcdc2800640 1 --2- 172.21.15.62:0/919104362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70e7b0 0x55cd6f70eb90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc02c710 tx=0x7fcdbc005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b099f0f3d2e4cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:56.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.983+0000 7fcdbaa00640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fcda80a3d20 0x7fcda80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:56.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.983+0000 7fcdb3400640 1 -- 172.21.15.62:0/919104362 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc043070 con 0x55cd6f70e7b0 2024-04-02T14:46:56.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.983+0000 7fcdb3400640 1 -- 172.21.15.62:0/919104362 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcdbc032d40 con 0x55cd6f70e7b0 2024-04-02T14:46:56.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.983+0000 7fcdb3400640 1 -- 172.21.15.62:0/919104362 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc039040 con 0x55cd6f70e7b0 2024-04-02T14:46:56.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.983+0000 7fcdbaa00640 1 -- 172.21.15.62:0/425505926 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fcdb40d0b90 con 0x7fcdc4108310 2024-04-02T14:46:56.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.983+0000 7fcdc1e00640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fcda80a3d20 0x7fcda80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:56.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.983+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/919104362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70e7b0 msgr2=0x55cd6f70eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:56.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.983+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/919104362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70e7b0 0x55cd6f70eb90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc02c710 tx=0x7fcdbc005f80 comp rx=0 tx=0).stop 2024-04-02T14:46:56.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.984+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/919104362 shutdown_connections 2024-04-02T14:46:56.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.984+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/919104362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f71a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.984+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/919104362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70f0d0 0x55cd6f717770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.984+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/919104362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70e7b0 0x55cd6f70eb90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.984+0000 7fcdc1e00640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fcda80a3d20 0x7fcda80a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55cd6f70e280 tx=0x7fcdb4018520 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.984+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/919104362 >> 172.21.15.62:0/919104362 conn(0x55cd6f708ee0 msgr2=0x55cd6f7092c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.984+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/919104362 shutdown_connections 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.984+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/919104362 wait complete. 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.984+0000 7fcdcbe4e740 1 -- start start 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.984+0000 7fcdcbe4e740 1 Processor -- start 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70e7b0 0x55cd6f714cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f715200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f715740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cd6f71aa40 con 0x55cd6f717cb0 2024-04-02T14:46:56.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cd6f62dac0 con 0x55cd6f70f0d0 2024-04-02T14:46:56.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cd6f71abc0 con 0x55cd6f70e7b0 2024-04-02T14:46:56.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdcbe4e740 10 client.0 _ll_register_callbacks cb 0x55cd6f630180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:56.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdc2800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70e7b0 0x55cd6f714cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:56.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdc2800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70e7b0 0x55cd6f714cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:36228/0 (socket says 172.21.15.62:36228) 2024-04-02T14:46:56.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdc2800640 1 -- 172.21.15.62:0/2236194459 learned_addr learned my addr 172.21.15.62:0/2236194459 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdc2800640 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f715740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.985+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f715200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcdbbe00640 1 -- 172.21.15.62:0/2236194459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70e7b0 msgr2=0x55cd6f714cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70e7b0 0x55cd6f714cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcdbbe00640 1 -- 172.21.15.62:0/2236194459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 msgr2=0x55cd6f715740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f715740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcdbbe00640 1 -- 172.21.15.62:0/2236194459 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd6f3863f0 con 0x55cd6f70f0d0 2024-04-02T14:46:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f715200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcdc41a0070 tx=0x7fcdac012e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24ae7721e1653d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcda7e00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcda7e00640 1 -- 172.21.15.62:0/2236194459 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac004050 con 0x55cd6f70f0d0 2024-04-02T14:46:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcda7e00640 1 -- 172.21.15.62:0/2236194459 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fcdac0041f0 con 0x55cd6f70f0d0 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2236194459 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdbc032d40 con 0x55cd6f70f0d0 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcda7e00640 1 -- 172.21.15.62:0/2236194459 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcdac0043f0 con 0x55cd6f70f0d0 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcdcbe4e740 10 client.26330 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.986+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2236194459 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cd6f3460f0 con 0x55cd6f70f0d0 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.987+0000 7fcda7e00640 1 -- 172.21.15.62:0/2236194459 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcdbc032d40 con 0x55cd6f70f0d0 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.987+0000 7fcda7e00640 1 -- 172.21.15.62:0/2236194459 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fcdac025430 con 0x55cd6f70f0d0 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.987+0000 7fcdcbe4e740 10 client.26330 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.987+0000 7fcdcbe4e740 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.987+0000 7fcdcbe4e740 20 client.26330 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.987+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55cd6f727c70 0x55cd6f72a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.987+0000 7fcdcbe4e740 4 client.26330 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.987+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2236194459 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd6f613630 con 0x55cd6f727c70 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.987+0000 7fcdc2800640 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55cd6f727c70 0x55cd6f72a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.988+0000 7fcdc2800640 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55cd6f727c70 0x55cd6f72a0b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c9504502449e6bf server_cookie=f77c4c8fa51fa19f in_seq=0 out_seq=0 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.988+0000 7fcda7e00640 10 client.26330 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.988+0000 7fcda7e00640 1 -- 172.21.15.62:0/2236194459 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd6f613630 con 0x55cd6f727c70 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.988+0000 7fcda7e00640 10 client.26330 handle_command_reply: tid=0 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.988+0000 7fcdcbe4e740 1 client.26330 shutdown 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2236194459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 msgr2=0x55cd6f715200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f715200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcdc41a0070 tx=0x7fcdac012e70 comp rx=0 tx=0).stop 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2236194459 shutdown_connections 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55cd6f727c70 0x55cd6f72a0b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f715740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f715200 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/2236194459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70e7b0 0x55cd6f714cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2236194459 >> 172.21.15.62:0/2236194459 conn(0x55cd6f708ee0 msgr2=0x55cd6f70ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcda7e00640 0 client.26330 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2236194459 shutdown_connections 2024-04-02T14:46:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2236194459 wait complete. 2024-04-02T14:46:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:56.989+0000 7fcdcbe4e740 20 client.26330 trim_cache size 0 max 16384 2024-04-02T14:46:57.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.008+0000 7fcdcbe4e740 1 Processor -- start 2024-04-02T14:46:57.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.009+0000 7fcdcbe4e740 1 -- start start 2024-04-02T14:46:57.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.009+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f70e7b0 0x55cd6f723900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.009+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f715560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.009+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f717cb0 0x55cd6f72f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.009+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41b3e40 con 0x55cd6f70e7b0 2024-04-02T14:46:57.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.009+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41b5b30 con 0x55cd6f70f0d0 2024-04-02T14:46:57.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.009+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41b4570 con 0x55cd6f717cb0 2024-04-02T14:46:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.009+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f715560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.009+0000 7fcdc2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f70e7b0 0x55cd6f723900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.009+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f715560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55516/0 (socket says 172.21.15.62:55516) 2024-04-02T14:46:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.009+0000 7fcdbbe00640 1 -- 172.21.15.62:0/2945884732 learned_addr learned my addr 172.21.15.62:0/2945884732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.010+0000 7fcdbbe00640 1 -- 172.21.15.62:0/2945884732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f717cb0 msgr2=0x55cd6f72f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.010+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/2945884732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f717cb0 0x55cd6f72f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.010+0000 7fcdbbe00640 1 -- 172.21.15.62:0/2945884732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f70e7b0 msgr2=0x55cd6f723900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.010+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/2945884732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f70e7b0 0x55cd6f723900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.010+0000 7fcdbbe00640 1 -- 172.21.15.62:0/2945884732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd6f7267f0 con 0x55cd6f70f0d0 2024-04-02T14:46:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.010+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/2945884732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f715560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00a010 tx=0x7fcdac012460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929794def101875f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.010+0000 7fcdb2a00640 1 -- 172.21.15.62:0/2945884732 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac025430 con 0x55cd6f70f0d0 2024-04-02T14:46:57.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.010+0000 7fcdb2a00640 1 -- 172.21.15.62:0/2945884732 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcdac0255d0 con 0x55cd6f70f0d0 2024-04-02T14:46:57.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.010+0000 7fcdb2a00640 1 -- 172.21.15.62:0/2945884732 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac005460 con 0x55cd6f70f0d0 2024-04-02T14:46:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.011+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2945884732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 msgr2=0x55cd6f715560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.011+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/2945884732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f715560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00a010 tx=0x7fcdac012460 comp rx=0 tx=0).stop 2024-04-02T14:46:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.011+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2945884732 shutdown_connections 2024-04-02T14:46:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.011+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/2945884732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f717cb0 0x55cd6f72f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.011+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/2945884732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f715560 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.011+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/2945884732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f70e7b0 0x55cd6f723900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.011+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2945884732 >> 172.21.15.62:0/2945884732 conn(0x55cd6f708ee0 msgr2=0x55cd6f70a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.011+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2945884732 shutdown_connections 2024-04-02T14:46:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.011+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/2945884732 wait complete. 2024-04-02T14:46:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.012+0000 7fcdcbe4e740 1 -- start start 2024-04-02T14:46:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.012+0000 7fcdcbe4e740 1 Processor -- start 2024-04-02T14:46:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.012+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70e7b0 0x55cd6f71b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.012+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f71be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.012+0000 7fcdcbe4e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f71c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.012+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41b53a0 con 0x55cd6f717cb0 2024-04-02T14:46:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.012+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41b4c70 con 0x55cd6f70f0d0 2024-04-02T14:46:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.012+0000 7fcdcbe4e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41b1e40 con 0x55cd6f70e7b0 2024-04-02T14:46:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.012+0000 7fcdcbe4e740 10 client.0 _ll_register_callbacks cb 0x7fcdac009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.012+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f71be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f71be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55528/0 (socket says 172.21.15.62:55528) 2024-04-02T14:46:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdbbe00640 1 -- 172.21.15.62:0/1729740419 learned_addr learned my addr 172.21.15.62:0/1729740419 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdc2800640 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70e7b0 0x55cd6f71b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdbbe00640 1 -- 172.21.15.62:0/1729740419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70e7b0 msgr2=0x55cd6f71b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70e7b0 0x55cd6f71b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdc2800640 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f71c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdbbe00640 1 -- 172.21.15.62:0/1729740419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 msgr2=0x55cd6f71c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f71c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdbbe00640 1 -- 172.21.15.62:0/1729740419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd6f78d2c0 con 0x55cd6f70f0d0 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdc2800640 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f71c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdbbe00640 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f71be50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcdac005080 tx=0x7fcdac005410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e9d342e0b25a478 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcda7e00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcda7e00640 1 -- 172.21.15.62:0/1729740419 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac03b060 con 0x55cd6f70f0d0 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/1729740419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdac0255d0 con 0x55cd6f70f0d0 2024-04-02T14:46:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.013+0000 7fcda7e00640 1 -- 172.21.15.62:0/1729740419 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fcdac005600 con 0x55cd6f70f0d0 2024-04-02T14:46:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.014+0000 7fcda7e00640 1 -- 172.21.15.62:0/1729740419 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcdac025430 con 0x55cd6f70f0d0 2024-04-02T14:46:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.014+0000 7fcdcbe4e740 10 client.26336 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.014+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/1729740419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cd6f78e5d0 con 0x55cd6f70f0d0 2024-04-02T14:46:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.014+0000 7fcda7e00640 1 -- 172.21.15.62:0/1729740419 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcdac0255d0 con 0x55cd6f70f0d0 2024-04-02T14:46:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.014+0000 7fcda7e00640 1 -- 172.21.15.62:0/1729740419 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fcdac040460 con 0x55cd6f70f0d0 2024-04-02T14:46:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.014+0000 7fcdcbe4e740 10 client.26336 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.014+0000 7fcdcbe4e740 10 client.26336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.014+0000 7fcdcbe4e740 20 client.26336 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.014+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55cd6f78ec70 0x55cd6f791090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.014+0000 7fcdcbe4e740 4 client.26336 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:46:57.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.014+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/1729740419 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd6f67fcb0 con 0x55cd6f78ec70 2024-04-02T14:46:57.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcdc2800640 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55cd6f78ec70 0x55cd6f791090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcdc2800640 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55cd6f78ec70 0x55cd6f791090 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1cf0af458770bc1 server_cookie=95b0380e71a204c7 in_seq=0 out_seq=0 2024-04-02T14:46:57.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcda7e00640 10 client.26336 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:57.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcda7e00640 1 -- 172.21.15.62:0/1729740419 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cd6f67fcb0 con 0x55cd6f78ec70 2024-04-02T14:46:57.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcda7e00640 10 client.26336 handle_command_reply: tid=0 2024-04-02T14:46:57.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcdcbe4e740 1 client.26336 shutdown 2024-04-02T14:46:57.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/1729740419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 msgr2=0x55cd6f71be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f71be50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcdac005080 tx=0x7fcdac005410 comp rx=0 tx=0).stop 2024-04-02T14:46:57.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/1729740419 shutdown_connections 2024-04-02T14:46:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55cd6f78ec70 0x55cd6f791090 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd6f717cb0 0x55cd6f71c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd6f70f0d0 0x55cd6f71be50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcdcbe4e740 1 --2- 172.21.15.62:0/1729740419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd6f70e7b0 0x55cd6f71b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcda7e00640 0 client.26336 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.015+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/1729740419 >> 172.21.15.62:0/1729740419 conn(0x55cd6f708ee0 msgr2=0x55cd6f70a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.016+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/1729740419 shutdown_connections 2024-04-02T14:46:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.016+0000 7fcdcbe4e740 1 -- 172.21.15.62:0/1729740419 wait complete. 2024-04-02T14:46:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.016+0000 7fcdcbe4e740 20 client.26336 trim_cache size 0 max 16384 2024-04-02T14:46:57.015 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:46:57.016 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-02T14:46:57.016 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-02T14:46:57.016 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:46:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.016+0000 7fcdc9200640 1 -- 172.21.15.62:0/425505926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fcda80a3d20 msgr2=0x7fcda80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.016+0000 7fcdc9200640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fcda80a3d20 0x7fcda80a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55cd6f70e280 tx=0x7fcdb4018520 comp rx=0 tx=0).stop 2024-04-02T14:46:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.016+0000 7fcdc9200640 1 -- 172.21.15.62:0/425505926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc4108310 msgr2=0x7fcdc41a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.016+0000 7fcdc9200640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc4108310 0x7fcdc41a46d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e9f0 tx=0x7fcdb400eeb0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.017+0000 7fcdc9200640 1 -- 172.21.15.62:0/425505926 shutdown_connections 2024-04-02T14:46:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.017+0000 7fcdc9200640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fcda80a3d20 0x7fcda80a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.017+0000 7fcdc9200640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc419e8d0 0x7fcdc419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.017+0000 7fcdc9200640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc410d3e0 0x7fcdc41a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.017+0000 7fcdc9200640 1 --2- 172.21.15.62:0/425505926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc4108310 0x7fcdc41a46d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.017+0000 7fcdc9200640 1 -- 172.21.15.62:0/425505926 >> 172.21.15.62:0/425505926 conn(0x7fcdc4076040 msgr2=0x7fcdc4076bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.017+0000 7fcdc9200640 1 -- 172.21.15.62:0/425505926 shutdown_connections 2024-04-02T14:46:57.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.017+0000 7fcdc9200640 1 -- 172.21.15.62:0/425505926 wait complete. 2024-04-02T14:46:57.028 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:46:57.028 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:46:57.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.239+0000 7fee29a00640 1 -- 172.21.15.62:0/1644074289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee240691a0 msgr2=0x7fee24069600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.239+0000 7fee29a00640 1 --2- 172.21.15.62:0/1644074289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee240691a0 0x7fee24069600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009ca0 tx=0x7fee0c02c100 comp rx=0 tx=0).stop 2024-04-02T14:46:57.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.239+0000 7fee29a00640 1 -- 172.21.15.62:0/1644074289 shutdown_connections 2024-04-02T14:46:57.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.239+0000 7fee29a00640 1 --2- 172.21.15.62:0/1644074289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee2410e1b0 0x7fee241105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.239+0000 7fee29a00640 1 --2- 172.21.15.62:0/1644074289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee240691a0 0x7fee24069600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.239+0000 7fee29a00640 1 --2- 172.21.15.62:0/1644074289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee24102810 0x7fee24068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.239+0000 7fee29a00640 1 -- 172.21.15.62:0/1644074289 >> 172.21.15.62:0/1644074289 conn(0x7fee24076410 msgr2=0x7fee24076820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:57.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.240+0000 7fee29a00640 1 -- 172.21.15.62:0/1644074289 shutdown_connections 2024-04-02T14:46:57.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.240+0000 7fee29a00640 1 -- 172.21.15.62:0/1644074289 wait complete. 2024-04-02T14:46:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.241+0000 7fee29a00640 1 Processor -- start 2024-04-02T14:46:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.241+0000 7fee29a00640 1 -- start start 2024-04-02T14:46:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.241+0000 7fee29a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee240691a0 0x7fee2419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.241+0000 7fee29a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee24102810 0x7fee2419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.241+0000 7fee29a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee2410e1b0 0x7fee2419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.241+0000 7fee29a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fee24114020 con 0x7fee240691a0 2024-04-02T14:46:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.241+0000 7fee29a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fee24113ea0 con 0x7fee2410e1b0 2024-04-02T14:46:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.241+0000 7fee29a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fee241141a0 con 0x7fee24102810 2024-04-02T14:46:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.242+0000 7fee23e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee240691a0 0x7fee2419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.242+0000 7fee23e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee240691a0 0x7fee2419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42070/0 (socket says 172.21.15.62:42070) 2024-04-02T14:46:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.242+0000 7fee23e00640 1 -- 172.21.15.62:0/2832065442 learned_addr learned my addr 172.21.15.62:0/2832065442 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:57.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.242+0000 7fee29000640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee2410e1b0 0x7fee2419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.242+0000 7fee23400640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee24102810 0x7fee2419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.242+0000 7fee23e00640 1 -- 172.21.15.62:0/2832065442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee24102810 msgr2=0x7fee2419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.242+0000 7fee23e00640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee24102810 0x7fee2419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.242+0000 7fee23e00640 1 -- 172.21.15.62:0/2832065442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee2410e1b0 msgr2=0x7fee2419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.242+0000 7fee23e00640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee2410e1b0 0x7fee2419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.242+0000 7fee23e00640 1 -- 172.21.15.62:0/2832065442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2419f4c0 con 0x7fee240691a0 2024-04-02T14:46:57.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.242+0000 7fee29000640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee2410e1b0 0x7fee2419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:57.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.243+0000 7fee23e00640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee240691a0 0x7fee2419a520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fee1400d950 tx=0x7fee1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:57.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.243+0000 7fee20c00640 1 -- 172.21.15.62:0/2832065442 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14017030 con 0x7fee240691a0 2024-04-02T14:46:57.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.243+0000 7fee20c00640 1 -- 172.21.15.62:0/2832065442 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fee14004020 con 0x7fee240691a0 2024-04-02T14:46:57.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.243+0000 7fee29a00640 1 -- 172.21.15.62:0/2832065442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2419f7b0 con 0x7fee240691a0 2024-04-02T14:46:57.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.243+0000 7fee20c00640 1 -- 172.21.15.62:0/2832065442 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14004a00 con 0x7fee240691a0 2024-04-02T14:46:57.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.243+0000 7fee29a00640 1 -- 172.21.15.62:0/2832065442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee241a7040 con 0x7fee240691a0 2024-04-02T14:46:57.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.244+0000 7fee29a00640 1 -- 172.21.15.62:0/2832065442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee24068810 con 0x7fee240691a0 2024-04-02T14:46:57.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.246+0000 7fee20c00640 1 -- 172.21.15.62:0/2832065442 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fee140041d0 con 0x7fee240691a0 2024-04-02T14:46:57.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.248+0000 7fee20c00640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fee040a3d20 0x7fee040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.248+0000 7fee20c00640 1 -- 172.21.15.62:0/2832065442 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fee140d0110 con 0x7fee240691a0 2024-04-02T14:46:57.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.248+0000 7fee23400640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fee040a3d20 0x7fee040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.249+0000 7fee23400640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fee040a3d20 0x7fee040a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fee0c004af0 tx=0x7fee0c005c50 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.254+0000 7fee20c00640 1 -- 172.21.15.62:0/2832065442 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fee1409c800 con 0x7fee240691a0 2024-04-02T14:46:57.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.507+0000 7fee29a00640 1 -- 172.21.15.62:0/2832065442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee2419b820 con 0x7fee240691a0 2024-04-02T14:46:57.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.509+0000 7fee20c00640 1 -- 172.21.15.62:0/2832065442 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11950 (secure 0 0 0) 0x7fee1409c100 con 0x7fee240691a0 2024-04-02T14:46:57.504 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:46:57.504 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:46:51.746158+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584,24535,24533,24553,24565]},"id":1}]} 2024-04-02T14:46:57.505 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 36 2024-04-02T14:46:57.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 -- 172.21.15.62:0/2832065442 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fee040a3d20 msgr2=0x7fee040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fee040a3d20 0x7fee040a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fee0c004af0 tx=0x7fee0c005c50 comp rx=0 tx=0).stop 2024-04-02T14:46:57.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 -- 172.21.15.62:0/2832065442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee240691a0 msgr2=0x7fee2419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee240691a0 0x7fee2419a520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fee1400d950 tx=0x7fee1400de10 comp rx=0 tx=0).stop 2024-04-02T14:46:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 -- 172.21.15.62:0/2832065442 shutdown_connections 2024-04-02T14:46:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fee040a3d20 0x7fee040a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee2410e1b0 0x7fee2419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee24102810 0x7fee2419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 --2- 172.21.15.62:0/2832065442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee240691a0 0x7fee2419a520 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 -- 172.21.15.62:0/2832065442 >> 172.21.15.62:0/2832065442 conn(0x7fee24076410 msgr2=0x7fee240ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 -- 172.21.15.62:0/2832065442 shutdown_connections 2024-04-02T14:46:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.512+0000 7fee29a00640 1 -- 172.21.15.62:0/2832065442 wait complete. 2024-04-02T14:46:57.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:46:57.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.757+0000 7f51ea400640 1 -- 172.21.15.62:0/1117458958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e41010f0 msgr2=0x7f51e410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.757+0000 7f51ea400640 1 --2- 172.21.15.62:0/1117458958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e41010f0 0x7f51e410dd20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009ca0 tx=0x7f51d402c100 comp rx=0 tx=0).stop 2024-04-02T14:46:57.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.758+0000 7f51ea400640 1 -- 172.21.15.62:0/1117458958 shutdown_connections 2024-04-02T14:46:57.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.758+0000 7f51ea400640 1 --2- 172.21.15.62:0/1117458958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51e410e2f0 0x7f51e41106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.758+0000 7f51ea400640 1 --2- 172.21.15.62:0/1117458958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e41010f0 0x7f51e410dd20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.758+0000 7f51ea400640 1 --2- 172.21.15.62:0/1117458958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51e4100740 0x7f51e4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.758+0000 7f51ea400640 1 -- 172.21.15.62:0/1117458958 >> 172.21.15.62:0/1117458958 conn(0x7f51e40fc350 msgr2=0x7f51e40fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:57.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.758+0000 7f51ea400640 1 -- 172.21.15.62:0/1117458958 shutdown_connections 2024-04-02T14:46:57.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.758+0000 7f51ea400640 1 -- 172.21.15.62:0/1117458958 wait complete. 2024-04-02T14:46:57.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:57 smithi204 ceph-mon[35948]: pgmap v91: 97 pgs: 97 active+clean; 139 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 467 op/s 2024-04-02T14:46:57.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.759+0000 7f51ea400640 1 Processor -- start 2024-04-02T14:46:57.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.759+0000 7f51ea400640 1 -- start start 2024-04-02T14:46:57.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.759+0000 7f51ea400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51e4100740 0x7f51e419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.759+0000 7f51ea400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51e41010f0 0x7f51e419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.759+0000 7f51ea400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e410e2f0 0x7f51e419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.759+0000 7f51ea400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51e41140c0 con 0x7f51e41010f0 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.759+0000 7f51ea400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f51e4113f40 con 0x7f51e410e2f0 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.759+0000 7f51ea400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51e4114240 con 0x7f51e4100740 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.759+0000 7f51e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51e41010f0 0x7f51e419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.760+0000 7f51e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51e41010f0 0x7f51e419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42090/0 (socket says 172.21.15.62:42090) 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.760+0000 7f51e3e00640 1 -- 172.21.15.62:0/2670848577 learned_addr learned my addr 172.21.15.62:0/2670848577 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.760+0000 7f51e9a00640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e410e2f0 0x7f51e419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.760+0000 7f51e9000640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51e4100740 0x7f51e419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.760+0000 7f51e3e00640 1 -- 172.21.15.62:0/2670848577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51e4100740 msgr2=0x7f51e419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.760+0000 7f51e3e00640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51e4100740 0x7f51e419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.760+0000 7f51e3e00640 1 -- 172.21.15.62:0/2670848577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e410e2f0 msgr2=0x7f51e419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.760+0000 7f51e3e00640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e410e2f0 0x7f51e419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.760+0000 7f51e3e00640 1 -- 172.21.15.62:0/2670848577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e419f530 con 0x7f51e41010f0 2024-04-02T14:46:57.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.760+0000 7f51e3e00640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51e41010f0 0x7f51e419aa80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f51d4005e30 tx=0x7f51d40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:57.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.761+0000 7f51e1600640 1 -- 172.21.15.62:0/2670848577 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d4039040 con 0x7f51e41010f0 2024-04-02T14:46:57.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.761+0000 7f51e1600640 1 -- 172.21.15.62:0/2670848577 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f51d4044070 con 0x7f51e41010f0 2024-04-02T14:46:57.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.761+0000 7f51ea400640 1 -- 172.21.15.62:0/2670848577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e419f7c0 con 0x7f51e41010f0 2024-04-02T14:46:57.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.761+0000 7f51e1600640 1 -- 172.21.15.62:0/2670848577 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d40043a0 con 0x7f51e41010f0 2024-04-02T14:46:57.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.761+0000 7f51ea400640 1 -- 172.21.15.62:0/2670848577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e41a70b0 con 0x7f51e41010f0 2024-04-02T14:46:57.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.764+0000 7f51ee61d740 1 Processor -- start 2024-04-02T14:46:57.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.764+0000 7f51e1600640 1 -- 172.21.15.62:0/2670848577 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f51d4003a90 con 0x7f51e41010f0 2024-04-02T14:46:57.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.764+0000 7f51ee61d740 1 -- start start 2024-04-02T14:46:57.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 0x559d36e168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e16ec0 0x559d36e17340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e20230 0x559d36e22620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51ee61d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559d36d57400 con 0x559d36e17ba0 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51ee61d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559d36d3bf10 con 0x559d36e16ec0 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51ee61d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559d36d7b6f0 con 0x559d36e20230 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51e9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 0x559d36e168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51e9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 0x559d36e168f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42104/0 (socket says 172.21.15.62:42104) 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51e9a00640 1 -- 172.21.15.62:0/407756563 learned_addr learned my addr 172.21.15.62:0/407756563 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51e3e00640 1 --2- 172.21.15.62:0/407756563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e20230 0x559d36e22620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51e9000640 1 --2- 172.21.15.62:0/407756563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e16ec0 0x559d36e17340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51e9a00640 1 -- 172.21.15.62:0/407756563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e20230 msgr2=0x559d36e22620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51e9a00640 1 --2- 172.21.15.62:0/407756563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e20230 0x559d36e22620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51e9a00640 1 -- 172.21.15.62:0/407756563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e16ec0 msgr2=0x559d36e17340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51e9a00640 1 --2- 172.21.15.62:0/407756563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e16ec0 0x559d36e17340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.765+0000 7f51e9a00640 1 -- 172.21.15.62:0/407756563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d36a8f3f0 con 0x559d36e17ba0 2024-04-02T14:46:57.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.766+0000 7f51e9000640 1 --2- 172.21.15.62:0/407756563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e16ec0 0x559d36e17340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:57.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.766+0000 7f51e9a00640 1 --2- 172.21.15.62:0/407756563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 0x559d36e168f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009b80 tx=0x7f51dc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20669475db9faeff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:57.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.766+0000 7f51da000640 1 -- 172.21.15.62:0/407756563 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc004030 con 0x559d36e17ba0 2024-04-02T14:46:57.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.766+0000 7f51da000640 1 -- 172.21.15.62:0/407756563 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f51dc0041d0 con 0x559d36e17ba0 2024-04-02T14:46:57.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.766+0000 7f51da000640 1 -- 172.21.15.62:0/407756563 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc019750 con 0x559d36e17ba0 2024-04-02T14:46:57.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.766+0000 7f51e1600640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f51c80a3d20 0x7f51c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.766+0000 7f51e1600640 1 -- 172.21.15.62:0/2670848577 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f51d40f2bf0 con 0x7f51e41010f0 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.767+0000 7f51e9000640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f51c80a3d20 0x7f51c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.767+0000 7f51ee61d740 1 -- 172.21.15.62:0/407756563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 msgr2=0x559d36e168f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.767+0000 7f51ee61d740 1 --2- 172.21.15.62:0/407756563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 0x559d36e168f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009b80 tx=0x7f51dc011810 comp rx=0 tx=0).stop 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.767+0000 7f51ee61d740 1 -- 172.21.15.62:0/407756563 shutdown_connections 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.767+0000 7f51ee61d740 1 --2- 172.21.15.62:0/407756563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e20230 0x559d36e22620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.767+0000 7f51ee61d740 1 --2- 172.21.15.62:0/407756563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e16ec0 0x559d36e17340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.767+0000 7f51ee61d740 1 --2- 172.21.15.62:0/407756563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 0x559d36e168f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.767+0000 7f51ee61d740 1 -- 172.21.15.62:0/407756563 >> 172.21.15.62:0/407756563 conn(0x559d36ddda50 msgr2=0x559d36e11910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.767+0000 7f51ee61d740 1 -- 172.21.15.62:0/407756563 shutdown_connections 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.767+0000 7f51e9000640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f51c80a3d20 0x7f51c80a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x559d36e180f0 tx=0x7f51cc005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.767+0000 7f51ee61d740 1 -- 172.21.15.62:0/407756563 wait complete. 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.768+0000 7f51ee61d740 1 -- start start 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.768+0000 7f51ee61d740 1 Processor -- start 2024-04-02T14:46:57.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.768+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e1d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.768+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 0x559d36e1d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.768+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 0x559d36e1dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51ee61d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559d36d776a0 con 0x559d36e20230 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51ee61d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559d36d23980 con 0x559d36e17ba0 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51ee61d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559d36d36cb0 con 0x559d36e16ec0 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51ee61d740 10 client.0 _ll_register_callbacks cb 0x559d36d366a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51e9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 0x559d36e1dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51e3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 0x559d36e1d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51e9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 0x559d36e1dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42118/0 (socket says 172.21.15.62:42118) 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51e9a00640 1 -- 172.21.15.62:0/1742223426 learned_addr learned my addr 172.21.15.62:0/1742223426 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51e9000640 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e1d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51e9a00640 1 -- 172.21.15.62:0/1742223426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 msgr2=0x559d36e1d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51e9a00640 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e1d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51e9a00640 1 -- 172.21.15.62:0/1742223426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 msgr2=0x559d36e1d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51e9a00640 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 0x559d36e1d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51e9a00640 1 -- 172.21.15.62:0/1742223426 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d36b597d0 con 0x559d36e20230 2024-04-02T14:46:57.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.769+0000 7f51e3e00640 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 0x559d36e1d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:57.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.770+0000 7f51e9a00640 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 0x559d36e1dc30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0055f0 tx=0x7f51dc005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3feddd87335b67f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:57.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.770+0000 7f51d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:46:57.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.770+0000 7f51d2a00640 1 -- 172.21.15.62:0/1742223426 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc028020 con 0x559d36e20230 2024-04-02T14:46:57.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.770+0000 7f51d2a00640 1 -- 172.21.15.62:0/1742223426 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f51dc0035f0 con 0x559d36e20230 2024-04-02T14:46:57.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.770+0000 7f51ee61d740 1 -- 172.21.15.62:0/1742223426 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51dc0041d0 con 0x559d36e20230 2024-04-02T14:46:57.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.770+0000 7f51d2a00640 1 -- 172.21.15.62:0/1742223426 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f51dc033050 con 0x559d36e20230 2024-04-02T14:46:57.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.770+0000 7f51ee61d740 10 client.16218 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:57.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.770+0000 7f51ee61d740 1 -- 172.21.15.62:0/1742223426 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559d36e2f090 con 0x559d36e20230 2024-04-02T14:46:57.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.771+0000 7f51d2a00640 1 -- 172.21.15.62:0/1742223426 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f51dc0041d0 con 0x559d36e20230 2024-04-02T14:46:57.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.771+0000 7f51d2a00640 1 -- 172.21.15.62:0/1742223426 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f51dc018070 con 0x559d36e20230 2024-04-02T14:46:57.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.771+0000 7f51ee61d740 10 client.16218 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:57.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.771+0000 7f51ee61d740 10 client.16218 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:46:57.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.771+0000 7f51ee61d740 20 client.16218 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:57.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.771+0000 7f51ee61d740 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x559d36e30490 0x559d36e32880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.771+0000 7f51ee61d740 4 client.16218 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.771+0000 7f51ee61d740 1 -- 172.21.15.62:0/1742223426 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d36d3b040 con 0x559d36e30490 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.771+0000 7f51e9000640 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x559d36e30490 0x559d36e32880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.772+0000 7f51e9000640 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x559d36e30490 0x559d36e32880 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6d1eedf2deb5d96b server_cookie=6ed0624faa8cd364 in_seq=0 out_seq=0 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.772+0000 7f51d2a00640 10 client.16218 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51d2a00640 1 -- 172.21.15.62:0/1742223426 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d36d3b040 con 0x559d36e30490 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51d2a00640 10 client.16218 handle_command_reply: tid=0 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51ee61d740 1 client.16218 shutdown 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51ee61d740 1 -- 172.21.15.62:0/1742223426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 msgr2=0x559d36e1dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51ee61d740 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 0x559d36e1dc30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0055f0 tx=0x7f51dc005620 comp rx=0 tx=0).stop 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51ee61d740 1 -- 172.21.15.62:0/1742223426 shutdown_connections 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51ee61d740 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x559d36e30490 0x559d36e32880 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51ee61d740 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 0x559d36e1dc30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51ee61d740 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 0x559d36e1d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51ee61d740 1 --2- 172.21.15.62:0/1742223426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e1d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51d2a00640 0 client.16218 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:46:57.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51ee61d740 1 -- 172.21.15.62:0/1742223426 >> 172.21.15.62:0/1742223426 conn(0x559d36ddda50 msgr2=0x559d36e13660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:57.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51ee61d740 1 -- 172.21.15.62:0/1742223426 shutdown_connections 2024-04-02T14:46:57.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.773+0000 7f51ee61d740 1 -- 172.21.15.62:0/1742223426 wait complete. 2024-04-02T14:46:57.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.774+0000 7f51ee61d740 20 client.16218 trim_cache size 0 max 16384 2024-04-02T14:46:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.789+0000 7f51ee61d740 1 Processor -- start 2024-04-02T14:46:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.789+0000 7f51ee61d740 1 -- start start 2024-04-02T14:46:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.789+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e2bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.789+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 0x559d36e1c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.789+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e20230 0x559d36e1da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.789+0000 7f51ee61d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51e41af820 con 0x559d36e17ba0 2024-04-02T14:46:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.789+0000 7f51ee61d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f51e41b1510 con 0x559d36e20230 2024-04-02T14:46:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.789+0000 7f51ee61d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51e41aff50 con 0x559d36e16ec0 2024-04-02T14:46:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e9a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e20230 0x559d36e1da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 0x559d36e1c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e9a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e20230 0x559d36e1da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55586/0 (socket says 172.21.15.62:55586) 2024-04-02T14:46:57.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e9a00640 1 -- 172.21.15.62:0/638044158 learned_addr learned my addr 172.21.15.62:0/638044158 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:57.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e9000640 1 --2- 172.21.15.62:0/638044158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e2bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e9a00640 1 -- 172.21.15.62:0/638044158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 msgr2=0x559d36e2bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e9a00640 1 --2- 172.21.15.62:0/638044158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e2bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e9a00640 1 -- 172.21.15.62:0/638044158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 msgr2=0x559d36e1c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e9a00640 1 --2- 172.21.15.62:0/638044158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 0x559d36e1c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e9a00640 1 -- 172.21.15.62:0/638044158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d36e15ef0 con 0x559d36e20230 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e3e00640 1 --2- 172.21.15.62:0/638044158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 0x559d36e1c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.790+0000 7f51e9a00640 1 --2- 172.21.15.62:0/638044158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e20230 0x559d36e1da50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f51dc001eb0 tx=0x7f51dc009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18ad32f5c324f813 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51d9600640 1 -- 172.21.15.62:0/638044158 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc028050 con 0x559d36e20230 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51d9600640 1 -- 172.21.15.62:0/638044158 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f51dc009780 con 0x559d36e20230 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51d9600640 1 -- 172.21.15.62:0/638044158 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc01f040 con 0x559d36e20230 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51ee61d740 1 -- 172.21.15.62:0/638044158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e20230 msgr2=0x559d36e1da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51ee61d740 1 --2- 172.21.15.62:0/638044158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e20230 0x559d36e1da50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f51dc001eb0 tx=0x7f51dc009d30 comp rx=0 tx=0).stop 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51ee61d740 1 -- 172.21.15.62:0/638044158 shutdown_connections 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51ee61d740 1 --2- 172.21.15.62:0/638044158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e20230 0x559d36e1da50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51ee61d740 1 --2- 172.21.15.62:0/638044158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e17ba0 0x559d36e1c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51ee61d740 1 --2- 172.21.15.62:0/638044158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e2bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51ee61d740 1 -- 172.21.15.62:0/638044158 >> 172.21.15.62:0/638044158 conn(0x559d36ddda50 msgr2=0x559d36e12d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51ee61d740 1 -- 172.21.15.62:0/638044158 shutdown_connections 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.791+0000 7f51ee61d740 1 -- 172.21.15.62:0/638044158 wait complete. 2024-04-02T14:46:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.792+0000 7f51ee61d740 1 -- start start 2024-04-02T14:46:57.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.792+0000 7f51ee61d740 1 Processor -- start 2024-04-02T14:46:57.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.792+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.792+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 0x559d36e236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.792+0000 7f51ee61d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 0x559d36e948d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.792+0000 7f51ee61d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51e41b0d80 con 0x559d36e20230 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.792+0000 7f51ee61d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f51e41b0650 con 0x559d36e17ba0 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.792+0000 7f51ee61d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51e41ad820 con 0x559d36e16ec0 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51ee61d740 10 client.0 _ll_register_callbacks cb 0x7f51dc0054b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51e3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 0x559d36e236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51e3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 0x559d36e236e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55598/0 (socket says 172.21.15.62:55598) 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51e3e00640 1 -- 172.21.15.62:0/2312756652 learned_addr learned my addr 172.21.15.62:0/2312756652 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51e9000640 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51e9a00640 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 0x559d36e948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51e3e00640 1 -- 172.21.15.62:0/2312756652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 msgr2=0x559d36e231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51e3e00640 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51e3e00640 1 -- 172.21.15.62:0/2312756652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 msgr2=0x559d36e948d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51e3e00640 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 0x559d36e948d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51e3e00640 1 -- 172.21.15.62:0/2312756652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d36e94e10 con 0x559d36e17ba0 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51e3e00640 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 0x559d36e236e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f51d40bb150 tx=0x7f51d4034880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ece06a854f68775 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.793+0000 7f51d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.794+0000 7f51d2a00640 1 -- 172.21.15.62:0/2312756652 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d40bf150 con 0x559d36e17ba0 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.794+0000 7f51ee61d740 1 -- 172.21.15.62:0/2312756652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51dc009780 con 0x559d36e17ba0 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.794+0000 7f51d2a00640 1 -- 172.21.15.62:0/2312756652 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f51d40c74d0 con 0x559d36e17ba0 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.794+0000 7f51d2a00640 1 -- 172.21.15.62:0/2312756652 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f51d40bf3f0 con 0x559d36e17ba0 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.794+0000 7f51ee61d740 10 client.26360 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.794+0000 7f51ee61d740 1 -- 172.21.15.62:0/2312756652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559d36e95ff0 con 0x559d36e17ba0 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.794+0000 7f51d2a00640 1 -- 172.21.15.62:0/2312756652 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f51dc009780 con 0x559d36e17ba0 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.794+0000 7f51d2a00640 1 -- 172.21.15.62:0/2312756652 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f51d403c030 con 0x559d36e17ba0 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.794+0000 7f51ee61d740 10 client.26360 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.794+0000 7f51ee61d740 10 client.26360 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:46:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.795+0000 7f51ee61d740 20 client.26360 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.795+0000 7f51ee61d740 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x559d36e96780 0x559d36e98ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.795+0000 7f51ee61d740 4 client.26360 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.795+0000 7f51ee61d740 1 -- 172.21.15.62:0/2312756652 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d36d88c30 con 0x559d36e96780 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.795+0000 7f51e9000640 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x559d36e96780 0x559d36e98ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.795+0000 7f51e9000640 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x559d36e96780 0x559d36e98ba0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2192842fae0302c server_cookie=cd0bbcca47035334 in_seq=0 out_seq=0 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.795+0000 7f51d2a00640 10 client.26360 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51d2a00640 1 -- 172.21.15.62:0/2312756652 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d36d88c30 con 0x559d36e96780 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51d2a00640 10 client.26360 handle_command_reply: tid=0 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 1 client.26360 shutdown 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 1 -- 172.21.15.62:0/2312756652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 msgr2=0x559d36e236e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 0x559d36e236e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f51d40bb150 tx=0x7f51d4034880 comp rx=0 tx=0).stop 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 1 -- 172.21.15.62:0/2312756652 shutdown_connections 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x559d36e96780 0x559d36e98ba0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d36e20230 0x559d36e948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559d36e17ba0 0x559d36e236e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51d2a00640 0 client.26360 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:46:57.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 1 --2- 172.21.15.62:0/2312756652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d36e16ec0 0x559d36e231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 1 -- 172.21.15.62:0/2312756652 >> 172.21.15.62:0/2312756652 conn(0x559d36ddda50 msgr2=0x559d36e1da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:57.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 1 -- 172.21.15.62:0/2312756652 shutdown_connections 2024-04-02T14:46:57.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 1 -- 172.21.15.62:0/2312756652 wait complete. 2024-04-02T14:46:57.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.796+0000 7f51ee61d740 20 client.26360 trim_cache size 0 max 16384 2024-04-02T14:46:57.796 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:46:57.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 -- 172.21.15.62:0/2670848577 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f51c80a3d20 msgr2=0x7f51c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f51c80a3d20 0x7f51c80a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x559d36e180f0 tx=0x7f51cc005f90 comp rx=0 tx=0).stop 2024-04-02T14:46:57.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 -- 172.21.15.62:0/2670848577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51e41010f0 msgr2=0x7f51e419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:57.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51e41010f0 0x7f51e419aa80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f51d4005e30 tx=0x7f51d40093d0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 -- 172.21.15.62:0/2670848577 shutdown_connections 2024-04-02T14:46:57.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f51c80a3d20 0x7f51c80a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e410e2f0 0x7f51e419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51e41010f0 0x7f51e419aa80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 --2- 172.21.15.62:0/2670848577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51e4100740 0x7f51e419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:57.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 -- 172.21.15.62:0/2670848577 >> 172.21.15.62:0/2670848577 conn(0x7f51e40fc350 msgr2=0x7f51e40fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:57.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 -- 172.21.15.62:0/2670848577 shutdown_connections 2024-04-02T14:46:57.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:57.797+0000 7f51ea400640 1 -- 172.21.15.62:0/2670848577 wait complete. 2024-04-02T14:46:57.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:57 smithi062 ceph-mon[32116]: pgmap v91: 97 pgs: 97 active+clean; 139 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 467 op/s 2024-04-02T14:46:57.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:46:57.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:57 smithi113 ceph-mon[36135]: pgmap v91: 97 pgs: 97 active+clean; 139 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 467 op/s 2024-04-02T14:46:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.027+0000 7fda77200640 1 -- 172.21.15.62:0/753301005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda780747c0 msgr2=0x7fda78112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.027+0000 7fda77200640 1 --2- 172.21.15.62:0/753301005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda780747c0 0x7fda78112410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fda60009900 tx=0x7fda6002bcc0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.027+0000 7fda77200640 1 -- 172.21.15.62:0/753301005 shutdown_connections 2024-04-02T14:46:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.027+0000 7fda77200640 1 --2- 172.21.15.62:0/753301005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda78112950 0x7fda78114d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.027+0000 7fda77200640 1 --2- 172.21.15.62:0/753301005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda780747c0 0x7fda78112410 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.027+0000 7fda77200640 1 --2- 172.21.15.62:0/753301005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda78073ea0 0x7fda78074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.027+0000 7fda77200640 1 -- 172.21.15.62:0/753301005 >> 172.21.15.62:0/753301005 conn(0x7fda780fd470 msgr2=0x7fda780ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.027+0000 7fda77200640 1 -- 172.21.15.62:0/753301005 shutdown_connections 2024-04-02T14:46:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.027+0000 7fda77200640 1 -- 172.21.15.62:0/753301005 wait complete. 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.028+0000 7fda77200640 1 Processor -- start 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.028+0000 7fda77200640 1 -- start start 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda77200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda78073ea0 0x7fda781a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda77200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda780747c0 0x7fda781a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda77200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda78112950 0x7fda7819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda77200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda78118470 con 0x7fda78073ea0 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda77200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fda781182f0 con 0x7fda78112950 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda77200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda781185f0 con 0x7fda780747c0 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda75e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda78073ea0 0x7fda781a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda76800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda78112950 0x7fda7819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda76800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda78112950 0x7fda7819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55606/0 (socket says 172.21.15.62:55606) 2024-04-02T14:46:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda76800640 1 -- 172.21.15.62:0/127207061 learned_addr learned my addr 172.21.15.62:0/127207061 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda75e00640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda78073ea0 0x7fda781a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42152/0 (socket says 172.21.15.62:42152) 2024-04-02T14:46:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda6fe00640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda780747c0 0x7fda781a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda76800640 1 -- 172.21.15.62:0/127207061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda780747c0 msgr2=0x7fda781a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda76800640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda780747c0 0x7fda781a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda76800640 1 -- 172.21.15.62:0/127207061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda78073ea0 msgr2=0x7fda781a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda76800640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda78073ea0 0x7fda781a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda76800640 1 -- 172.21.15.62:0/127207061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7819efc0 con 0x7fda78112950 2024-04-02T14:46:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.029+0000 7fda75e00640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda78073ea0 0x7fda781a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.030+0000 7fda76800640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda78112950 0x7fda7819e7f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fda7000eb40 tx=0x7fda7000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.030+0000 7fda6ea00640 1 -- 172.21.15.62:0/127207061 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70004b20 con 0x7fda78112950 2024-04-02T14:46:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.030+0000 7fda6ea00640 1 -- 172.21.15.62:0/127207061 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda70004cc0 con 0x7fda78112950 2024-04-02T14:46:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.030+0000 7fda77200640 1 -- 172.21.15.62:0/127207061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7819f2b0 con 0x7fda78112950 2024-04-02T14:46:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.030+0000 7fda6ea00640 1 -- 172.21.15.62:0/127207061 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700108f0 con 0x7fda78112950 2024-04-02T14:46:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.030+0000 7fda77200640 1 -- 172.21.15.62:0/127207061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda781ab4d0 con 0x7fda78112950 2024-04-02T14:46:58.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.033+0000 7fda7fe02740 1 Processor -- start 2024-04-02T14:46:58.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.033+0000 7fda6ea00640 1 -- 172.21.15.62:0/127207061 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fda70005020 con 0x7fda78112950 2024-04-02T14:46:58.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.034+0000 7fda7fe02740 1 -- start start 2024-04-02T14:46:58.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.034+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.034+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f76c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.034+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f777620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.034+0000 7fda7fe02740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ed0f6ac400 con 0x55ed0f76bec0 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.034+0000 7fda7fe02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ed0f690f10 con 0x55ed0f76cba0 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.034+0000 7fda7fe02740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ed0f6d06f0 con 0x55ed0f775230 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda6fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda6fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55622/0 (socket says 172.21.15.62:55622) 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda6fe00640 1 -- 172.21.15.62:0/1002875057 learned_addr learned my addr 172.21.15.62:0/1002875057 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda75e00640 1 --2- 172.21.15.62:0/1002875057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f76c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda6fe00640 1 --2- 172.21.15.62:0/1002875057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f777620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda6fe00640 1 -- 172.21.15.62:0/1002875057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 msgr2=0x55ed0f777620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda6fe00640 1 --2- 172.21.15.62:0/1002875057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f777620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda6fe00640 1 -- 172.21.15.62:0/1002875057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 msgr2=0x55ed0f76c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda6fe00640 1 --2- 172.21.15.62:0/1002875057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f76c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda6fe00640 1 -- 172.21.15.62:0/1002875057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed0f3e43f0 con 0x55ed0f76cba0 2024-04-02T14:46:58.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda75e00640 1 --2- 172.21.15.62:0/1002875057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f76c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:58.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda6fe00640 1 --2- 172.21.15.62:0/1002875057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76b8f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fda60009900 tx=0x7fda60005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce8ab09dac7b0a60 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.035+0000 7fda6ea00640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda5c0a3d20 0x7fda5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.036+0000 7fda6ea00640 1 -- 172.21.15.62:0/127207061 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fda700d0e70 con 0x7fda78112950 2024-04-02T14:46:58.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.036+0000 7fda67400640 1 -- 172.21.15.62:0/1002875057 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60048070 con 0x55ed0f76cba0 2024-04-02T14:46:58.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.036+0000 7fda67400640 1 -- 172.21.15.62:0/1002875057 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda60004200 con 0x55ed0f76cba0 2024-04-02T14:46:58.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.036+0000 7fda67400640 1 -- 172.21.15.62:0/1002875057 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60004920 con 0x55ed0f76cba0 2024-04-02T14:46:58.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.036+0000 7fda75e00640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda5c0a3d20 0x7fda5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.036+0000 7fda7fe02740 1 -- 172.21.15.62:0/1002875057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 msgr2=0x55ed0f76b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.036+0000 7fda7fe02740 1 --2- 172.21.15.62:0/1002875057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76b8f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fda60009900 tx=0x7fda60005b50 comp rx=0 tx=0).stop 2024-04-02T14:46:58.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.037+0000 7fda75e00640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda5c0a3d20 0x7fda5c0a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fda68009fd0 tx=0x7fda68005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.037+0000 7fda7fe02740 1 -- 172.21.15.62:0/1002875057 shutdown_connections 2024-04-02T14:46:58.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.037+0000 7fda7fe02740 1 --2- 172.21.15.62:0/1002875057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f777620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.037+0000 7fda7fe02740 1 --2- 172.21.15.62:0/1002875057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f76c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.037+0000 7fda7fe02740 1 --2- 172.21.15.62:0/1002875057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76b8f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.037+0000 7fda7fe02740 1 -- 172.21.15.62:0/1002875057 >> 172.21.15.62:0/1002875057 conn(0x55ed0f732a50 msgr2=0x55ed0f766910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.037+0000 7fda7fe02740 1 -- 172.21.15.62:0/1002875057 shutdown_connections 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.037+0000 7fda7fe02740 1 -- 172.21.15.62:0/1002875057 wait complete. 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.037+0000 7fda7fe02740 1 -- start start 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.037+0000 7fda7fe02740 1 Processor -- start 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.038+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f772240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.038+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f76cba0 0x55ed0f772780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.038+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f775230 0x55ed0f772cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.038+0000 7fda7fe02740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ed0f6cc6a0 con 0x55ed0f76bec0 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.038+0000 7fda7fe02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ed0f678980 con 0x55ed0f775230 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.038+0000 7fda7fe02740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ed0f68bcb0 con 0x55ed0f76cba0 2024-04-02T14:46:58.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.038+0000 7fda7fe02740 10 client.0 _ll_register_callbacks cb 0x55ed0f68b6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:58.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.038+0000 7fda75e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f772240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.038+0000 7fda76800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f775230 0x55ed0f772cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.038+0000 7fda76800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f775230 0x55ed0f772cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55624/0 (socket says 172.21.15.62:55624) 2024-04-02T14:46:58.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.038+0000 7fda76800640 1 -- 172.21.15.62:0/4267850605 learned_addr learned my addr 172.21.15.62:0/4267850605 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda6fe00640 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f76cba0 0x55ed0f772780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda76800640 1 -- 172.21.15.62:0/4267850605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f76cba0 msgr2=0x55ed0f772780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda76800640 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f76cba0 0x55ed0f772780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda76800640 1 -- 172.21.15.62:0/4267850605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 msgr2=0x55ed0f772240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda76800640 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f772240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda76800640 1 -- 172.21.15.62:0/4267850605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed0f4ae7d0 con 0x55ed0f775230 2024-04-02T14:46:58.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda75e00640 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f772240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:58.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda6fe00640 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f76cba0 0x55ed0f772780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:46:58.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda76800640 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f775230 0x55ed0f772cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda700992c0 tx=0x7fda7009c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c096a918c5b564 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda5be00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:58.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda5be00640 1 -- 172.21.15.62:0/4267850605 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700b5070 con 0x55ed0f775230 2024-04-02T14:46:58.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda5be00640 1 -- 172.21.15.62:0/4267850605 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fda700994d0 con 0x55ed0f775230 2024-04-02T14:46:58.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.039+0000 7fda7fe02740 1 -- 172.21.15.62:0/4267850605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda60004200 con 0x55ed0f775230 2024-04-02T14:46:58.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.040+0000 7fda5be00640 1 -- 172.21.15.62:0/4267850605 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fda700a1210 con 0x55ed0f775230 2024-04-02T14:46:58.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.040+0000 7fda7fe02740 10 client.26372 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:58.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.040+0000 7fda7fe02740 1 -- 172.21.15.62:0/4267850605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ed0f7840b0 con 0x55ed0f775230 2024-04-02T14:46:58.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.040+0000 7fda5be00640 1 -- 172.21.15.62:0/4267850605 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda60004200 con 0x55ed0f775230 2024-04-02T14:46:58.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.040+0000 7fda5be00640 1 -- 172.21.15.62:0/4267850605 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fda700a5640 con 0x55ed0f775230 2024-04-02T14:46:58.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.040+0000 7fda7fe02740 10 client.26372 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.040+0000 7fda7fe02740 10 client.26372 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.040+0000 7fda7fe02740 20 client.26372 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.040+0000 7fda7fe02740 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55ed0f7855c0 0x55ed0f787a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.040+0000 7fda7fe02740 4 client.26372 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.040+0000 7fda7fe02740 1 -- 172.21.15.62:0/4267850605 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed0f690040 con 0x55ed0f7855c0 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.041+0000 7fda75e00640 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55ed0f7855c0 0x55ed0f787a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.041+0000 7fda75e00640 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55ed0f7855c0 0x55ed0f787a10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=614a429e1f8fc7b6 server_cookie=e628540a2aad271c in_seq=0 out_seq=0 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.041+0000 7fda5be00640 10 client.26372 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda5be00640 1 -- 172.21.15.62:0/4267850605 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed0f690040 con 0x55ed0f7855c0 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda5be00640 10 client.26372 handle_command_reply: tid=0 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda7fe02740 1 client.26372 shutdown 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda7fe02740 1 -- 172.21.15.62:0/4267850605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f775230 msgr2=0x55ed0f772cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda7fe02740 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f775230 0x55ed0f772cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda700992c0 tx=0x7fda7009c880 comp rx=0 tx=0).stop 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda7fe02740 1 -- 172.21.15.62:0/4267850605 shutdown_connections 2024-04-02T14:46:58.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda7fe02740 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55ed0f7855c0 0x55ed0f787a10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda7fe02740 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f775230 0x55ed0f772cc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda5be00640 0 client.26372 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:46:58.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda7fe02740 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f76cba0 0x55ed0f772780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda7fe02740 1 --2- 172.21.15.62:0/4267850605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f772240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.042+0000 7fda7fe02740 1 -- 172.21.15.62:0/4267850605 >> 172.21.15.62:0/4267850605 conn(0x55ed0f732a50 msgr2=0x55ed0f768660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.043+0000 7fda7fe02740 1 -- 172.21.15.62:0/4267850605 shutdown_connections 2024-04-02T14:46:58.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.043+0000 7fda7fe02740 1 -- 172.21.15.62:0/4267850605 wait complete. 2024-04-02T14:46:58.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.043+0000 7fda7fe02740 20 client.26372 trim_cache size 0 max 16384 2024-04-02T14:46:58.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.059+0000 7fda7fe02740 1 Processor -- start 2024-04-02T14:46:58.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda7fe02740 1 -- start start 2024-04-02T14:46:58.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76bec0 0x55ed0f780c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76cba0 0x55ed0f768470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f78d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda7fe02740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda781b3b00 con 0x55ed0f76cba0 2024-04-02T14:46:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda7fe02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fda781b57f0 con 0x55ed0f76bec0 2024-04-02T14:46:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda7fe02740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda781b4230 con 0x55ed0f775230 2024-04-02T14:46:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda75e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76bec0 0x55ed0f780c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda75e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76bec0 0x55ed0f780c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55632/0 (socket says 172.21.15.62:55632) 2024-04-02T14:46:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda75e00640 1 -- 172.21.15.62:0/3015128517 learned_addr learned my addr 172.21.15.62:0/3015128517 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda6fe00640 1 --2- 172.21.15.62:0/3015128517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76cba0 0x55ed0f768470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.060+0000 7fda76800640 1 --2- 172.21.15.62:0/3015128517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f78d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.061+0000 7fda75e00640 1 -- 172.21.15.62:0/3015128517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 msgr2=0x55ed0f78d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.061+0000 7fda75e00640 1 --2- 172.21.15.62:0/3015128517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f78d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.061+0000 7fda75e00640 1 -- 172.21.15.62:0/3015128517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76cba0 msgr2=0x55ed0f768470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.061+0000 7fda75e00640 1 --2- 172.21.15.62:0/3015128517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76cba0 0x55ed0f768470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.061+0000 7fda75e00640 1 -- 172.21.15.62:0/3015128517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed0f76aef0 con 0x55ed0f76bec0 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.061+0000 7fda75e00640 1 --2- 172.21.15.62:0/3015128517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76bec0 0x55ed0f780c70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55ed0f788250 tx=0x7fda6801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71d12eb971622b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.061+0000 7fda66a00640 1 -- 172.21.15.62:0/3015128517 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68030070 con 0x55ed0f76bec0 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.061+0000 7fda66a00640 1 -- 172.21.15.62:0/3015128517 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda6800fde0 con 0x55ed0f76bec0 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.061+0000 7fda66a00640 1 -- 172.21.15.62:0/3015128517 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6801fbe0 con 0x55ed0f76bec0 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.062+0000 7fda7fe02740 1 -- 172.21.15.62:0/3015128517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76bec0 msgr2=0x55ed0f780c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.062+0000 7fda7fe02740 1 --2- 172.21.15.62:0/3015128517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76bec0 0x55ed0f780c70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55ed0f788250 tx=0x7fda6801a000 comp rx=0 tx=0).stop 2024-04-02T14:46:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.062+0000 7fda7fe02740 1 -- 172.21.15.62:0/3015128517 shutdown_connections 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.062+0000 7fda7fe02740 1 --2- 172.21.15.62:0/3015128517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f78d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.062+0000 7fda7fe02740 1 --2- 172.21.15.62:0/3015128517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76cba0 0x55ed0f768470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.062+0000 7fda7fe02740 1 --2- 172.21.15.62:0/3015128517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76bec0 0x55ed0f780c70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.062+0000 7fda7fe02740 1 -- 172.21.15.62:0/3015128517 >> 172.21.15.62:0/3015128517 conn(0x55ed0f732a50 msgr2=0x55ed0f767d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.062+0000 7fda7fe02740 1 -- 172.21.15.62:0/3015128517 shutdown_connections 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.062+0000 7fda7fe02740 1 -- 172.21.15.62:0/3015128517 wait complete. 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.062+0000 7fda7fe02740 1 -- start start 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.062+0000 7fda7fe02740 1 Processor -- start 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f77d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda7fe02740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f76fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda7fe02740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda781b5060 con 0x55ed0f76bec0 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda7fe02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fda781b4930 con 0x55ed0f76cba0 2024-04-02T14:46:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda7fe02740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda781b1ae0 con 0x55ed0f775230 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda7fe02740 10 client.0 _ll_register_callbacks cb 0x7fda7009cd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda6fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda6fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55646/0 (socket says 172.21.15.62:55646) 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda6fe00640 1 -- 172.21.15.62:0/3120792260 learned_addr learned my addr 172.21.15.62:0/3120792260 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda75e00640 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f77d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda76800640 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f76fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda6fe00640 1 -- 172.21.15.62:0/3120792260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 msgr2=0x55ed0f76fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.063+0000 7fda6fe00640 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f76fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda6fe00640 1 -- 172.21.15.62:0/3120792260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 msgr2=0x55ed0f77d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda6fe00640 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f77d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda6fe00640 1 -- 172.21.15.62:0/3120792260 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed0f7702b0 con 0x55ed0f76cba0 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda75e00640 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f77d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda76800640 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f76fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda6fe00640 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76f4e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55ed0f767670 tx=0x7fda6002cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdccc618c693302b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda5be00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda5be00640 1 -- 172.21.15.62:0/3120792260 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60050060 con 0x55ed0f76cba0 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda5be00640 1 -- 172.21.15.62:0/3120792260 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7fda60005450 con 0x55ed0f76cba0 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda7fe02740 1 -- 172.21.15.62:0/3120792260 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda6800fde0 con 0x55ed0f76cba0 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda5be00640 1 -- 172.21.15.62:0/3120792260 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fda60048070 con 0x55ed0f76cba0 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda7fe02740 10 client.26378 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.064+0000 7fda7fe02740 1 -- 172.21.15.62:0/3120792260 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ed0f7ebc10 con 0x55ed0f76cba0 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.065+0000 7fda5be00640 1 -- 172.21.15.62:0/3120792260 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda6800fde0 con 0x55ed0f76cba0 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.065+0000 7fda5be00640 1 -- 172.21.15.62:0/3120792260 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fda60003680 con 0x55ed0f76cba0 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.065+0000 7fda7fe02740 10 client.26378 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.065+0000 7fda7fe02740 10 client.26378 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.065+0000 7fda7fe02740 20 client.26378 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.065+0000 7fda7fe02740 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55ed0f7ebef0 0x55ed0f7ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.065+0000 7fda7fe02740 4 client.26378 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:46:58.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.065+0000 7fda7fe02740 1 -- 172.21.15.62:0/3120792260 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed0f6ddc30 con 0x55ed0f7ebef0 2024-04-02T14:46:58.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.065+0000 7fda75e00640 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55ed0f7ebef0 0x55ed0f7ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.066+0000 7fda75e00640 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55ed0f7ebef0 0x55ed0f7ee310 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c41b232612d7f4a1 server_cookie=c5a6ae9e6d8cf08c in_seq=0 out_seq=0 2024-04-02T14:46:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.066+0000 7fda5be00640 10 client.26378 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:46:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.066+0000 7fda5be00640 1 -- 172.21.15.62:0/3120792260 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed0f6ddc30 con 0x55ed0f7ebef0 2024-04-02T14:46:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.066+0000 7fda5be00640 10 client.26378 handle_command_reply: tid=0 2024-04-02T14:46:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.066+0000 7fda7fe02740 1 client.26378 shutdown 2024-04-02T14:46:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.066+0000 7fda7fe02740 1 -- 172.21.15.62:0/3120792260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 msgr2=0x55ed0f76f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.066+0000 7fda7fe02740 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76f4e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55ed0f767670 tx=0x7fda6002cc90 comp rx=0 tx=0).stop 2024-04-02T14:46:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.067+0000 7fda7fe02740 1 -- 172.21.15.62:0/3120792260 shutdown_connections 2024-04-02T14:46:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.067+0000 7fda7fe02740 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55ed0f7ebef0 0x55ed0f7ee310 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.067+0000 7fda7fe02740 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed0f775230 0x55ed0f76fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.067+0000 7fda7fe02740 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ed0f76cba0 0x55ed0f76f4e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.067+0000 7fda5be00640 0 client.26378 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:46:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.067+0000 7fda7fe02740 1 --2- 172.21.15.62:0/3120792260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed0f76bec0 0x55ed0f77d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.067+0000 7fda7fe02740 1 -- 172.21.15.62:0/3120792260 >> 172.21.15.62:0/3120792260 conn(0x55ed0f732a50 msgr2=0x55ed0f765af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.067+0000 7fda7fe02740 1 -- 172.21.15.62:0/3120792260 shutdown_connections 2024-04-02T14:46:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.067+0000 7fda7fe02740 1 -- 172.21.15.62:0/3120792260 wait complete. 2024-04-02T14:46:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.067+0000 7fda7fe02740 20 client.26378 trim_cache size 0 max 16384 2024-04-02T14:46:58.066 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:46:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 -- 172.21.15.62:0/127207061 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda5c0a3d20 msgr2=0x7fda5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda5c0a3d20 0x7fda5c0a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fda68009fd0 tx=0x7fda68005f90 comp rx=0 tx=0).stop 2024-04-02T14:46:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 -- 172.21.15.62:0/127207061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda78112950 msgr2=0x7fda7819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda78112950 0x7fda7819e7f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fda7000eb40 tx=0x7fda7000c2d0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 -- 172.21.15.62:0/127207061 shutdown_connections 2024-04-02T14:46:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fda5c0a3d20 0x7fda5c0a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda78112950 0x7fda7819e7f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda780747c0 0x7fda781a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 --2- 172.21.15.62:0/127207061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda78073ea0 0x7fda781a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 -- 172.21.15.62:0/127207061 >> 172.21.15.62:0/127207061 conn(0x7fda780fd470 msgr2=0x7fda780febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 -- 172.21.15.62:0/127207061 shutdown_connections 2024-04-02T14:46:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.068+0000 7fda77200640 1 -- 172.21.15.62:0/127207061 wait complete. 2024-04-02T14:46:58.080 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:46:58.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.299+0000 7f3647c00640 1 -- 172.21.15.62:0/3218773652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364010d3e0 msgr2=0x7f3640114fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.299+0000 7f3647c00640 1 --2- 172.21.15.62:0/3218773652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364010d3e0 0x7f3640114fb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3638009be0 tx=0x7f363802bef0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.299+0000 7f3647c00640 1 -- 172.21.15.62:0/3218773652 shutdown_connections 2024-04-02T14:46:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.299+0000 7f3647c00640 1 --2- 172.21.15.62:0/3218773652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364010d3e0 0x7f3640114fb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.299+0000 7f3647c00640 1 --2- 172.21.15.62:0/3218773652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3640108cc0 0x7f364010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.299+0000 7f3647c00640 1 --2- 172.21.15.62:0/3218773652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3640108310 0x7f36401086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.299+0000 7f3647c00640 1 -- 172.21.15.62:0/3218773652 >> 172.21.15.62:0/3218773652 conn(0x7f3640076040 msgr2=0x7f3640078460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.300+0000 7f3647c00640 1 -- 172.21.15.62:0/3218773652 shutdown_connections 2024-04-02T14:46:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.300+0000 7f3647c00640 1 -- 172.21.15.62:0/3218773652 wait complete. 2024-04-02T14:46:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.301+0000 7f3647c00640 1 Processor -- start 2024-04-02T14:46:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.301+0000 7f3647c00640 1 -- start start 2024-04-02T14:46:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.301+0000 7f3647c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3640108310 0x7f36401a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.301+0000 7f3647c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364010d3e0 0x7f36401a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f3647c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f364019e8b0 0x7f364019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f3647c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36401185d0 con 0x7f3640108310 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f3647c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3640118450 con 0x7f364010d3e0 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f3647c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3640118750 con 0x7f364019e8b0 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f363fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364010d3e0 0x7f36401a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f363fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364010d3e0 0x7f36401a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55672/0 (socket says 172.21.15.62:55672) 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f363fe00640 1 -- 172.21.15.62:0/1788461413 learned_addr learned my addr 172.21.15.62:0/1788461413 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f3645200640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3640108310 0x7f36401a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f3645c00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f364019e8b0 0x7f364019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f363fe00640 1 -- 172.21.15.62:0/1788461413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f364019e8b0 msgr2=0x7f364019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f363fe00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f364019e8b0 0x7f364019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f363fe00640 1 -- 172.21.15.62:0/1788461413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3640108310 msgr2=0x7f36401a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f363fe00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3640108310 0x7f36401a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f363fe00640 1 -- 172.21.15.62:0/1788461413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364019f3d0 con 0x7f364010d3e0 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.302+0000 7f3645200640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3640108310 0x7f36401a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:58.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.303+0000 7f363fe00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364010d3e0 0x7f36401a4c40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e9f0 tx=0x7f362c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.303+0000 7f363d600640 1 -- 172.21.15.62:0/1788461413 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c00ccc0 con 0x7f364010d3e0 2024-04-02T14:46:58.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.303+0000 7f363d600640 1 -- 172.21.15.62:0/1788461413 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f362c005520 con 0x7f364010d3e0 2024-04-02T14:46:58.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.303+0000 7f3647c00640 1 -- 172.21.15.62:0/1788461413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364019f6c0 con 0x7f364010d3e0 2024-04-02T14:46:58.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.303+0000 7f363d600640 1 -- 172.21.15.62:0/1788461413 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c010690 con 0x7f364010d3e0 2024-04-02T14:46:58.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.303+0000 7f3647c00640 1 -- 172.21.15.62:0/1788461413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36401ab910 con 0x7f364010d3e0 2024-04-02T14:46:58.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.306+0000 7f363d600640 1 -- 172.21.15.62:0/1788461413 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f362c004f70 con 0x7f364010d3e0 2024-04-02T14:46:58.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.307+0000 7f364a98c740 1 Processor -- start 2024-04-02T14:46:58.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.307+0000 7f364a98c740 1 -- start start 2024-04-02T14:46:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.307+0000 7f364a98c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f42609f0 0x55d8f425f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.307+0000 7f364a98c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f425fd10 0x55d8f4260190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.307+0000 7f364a98c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f4269080 0x55d8f426b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.307+0000 7f364a98c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d8f41a0400 con 0x55d8f42609f0 2024-04-02T14:46:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.307+0000 7f364a98c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d8f4184f10 con 0x55d8f425fd10 2024-04-02T14:46:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.307+0000 7f364a98c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d8f41c46f0 con 0x55d8f4269080 2024-04-02T14:46:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f3645200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f425fd10 0x55d8f4260190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f3645c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f42609f0 0x55d8f425f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f3645200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f425fd10 0x55d8f4260190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55688/0 (socket says 172.21.15.62:55688) 2024-04-02T14:46:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f3645200640 1 -- 172.21.15.62:0/241699440 learned_addr learned my addr 172.21.15.62:0/241699440 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f363d600640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f36280a3da0 0x7f36280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f363fe00640 1 --2- 172.21.15.62:0/241699440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f4269080 0x55d8f426b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f3645200640 1 -- 172.21.15.62:0/241699440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f4269080 msgr2=0x55d8f426b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f3645200640 1 --2- 172.21.15.62:0/241699440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f4269080 0x55d8f426b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f3645200640 1 -- 172.21.15.62:0/241699440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f42609f0 msgr2=0x55d8f425f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f3645200640 1 --2- 172.21.15.62:0/241699440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f42609f0 0x55d8f425f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f3645200640 1 -- 172.21.15.62:0/241699440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f3ed83f0 con 0x55d8f425fd10 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f363d600640 1 -- 172.21.15.62:0/1788461413 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f362c0d0b00 con 0x7f364010d3e0 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f3645c00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f36280a3da0 0x7f36280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.308+0000 7f3645200640 1 --2- 172.21.15.62:0/241699440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f425fd10 0x55d8f4260190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3634009ce0 tx=0x7f36340117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bb6c5d2ea1282d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.309+0000 7f3632000640 1 -- 172.21.15.62:0/241699440 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634012450 con 0x55d8f425fd10 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.309+0000 7f3632000640 1 -- 172.21.15.62:0/241699440 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3634012af0 con 0x55d8f425fd10 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.309+0000 7f3632000640 1 -- 172.21.15.62:0/241699440 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363400fd40 con 0x55d8f425fd10 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.309+0000 7f364a98c740 1 -- 172.21.15.62:0/241699440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f425fd10 msgr2=0x55d8f4260190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.309+0000 7f3645c00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f36280a3da0 0x7f36280a6260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f36380099a0 tx=0x7f3638009500 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.310+0000 7f364a98c740 1 --2- 172.21.15.62:0/241699440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f425fd10 0x55d8f4260190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3634009ce0 tx=0x7f36340117b0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.310+0000 7f364a98c740 1 -- 172.21.15.62:0/241699440 shutdown_connections 2024-04-02T14:46:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.310+0000 7f364a98c740 1 --2- 172.21.15.62:0/241699440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f4269080 0x55d8f426b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.310+0000 7f364a98c740 1 --2- 172.21.15.62:0/241699440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f425fd10 0x55d8f4260190 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.310+0000 7f364a98c740 1 --2- 172.21.15.62:0/241699440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f42609f0 0x55d8f425f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.310+0000 7f364a98c740 1 -- 172.21.15.62:0/241699440 >> 172.21.15.62:0/241699440 conn(0x55d8f4226a50 msgr2=0x55d8f425a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.310+0000 7f364a98c740 1 -- 172.21.15.62:0/241699440 shutdown_connections 2024-04-02T14:46:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.310+0000 7f364a98c740 1 -- 172.21.15.62:0/241699440 wait complete. 2024-04-02T14:46:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.310+0000 7f364a98c740 1 -- start start 2024-04-02T14:46:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f364a98c740 1 Processor -- start 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f364a98c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 0x55d8f4266070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f364a98c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f42665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f364a98c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f4269080 0x55d8f4266af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f364a98c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d8f41c06a0 con 0x55d8f4269080 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f364a98c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d8f416c980 con 0x55d8f42609f0 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f364a98c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d8f417fcb0 con 0x55d8f425fd10 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f364a98c740 10 client.0 _ll_register_callbacks cb 0x55d8f417f6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f363fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f42665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f3645200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 0x55d8f4266070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f363fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f42665b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55696/0 (socket says 172.21.15.62:55696) 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f3645c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f4269080 0x55d8f4266af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.311+0000 7f363fe00640 1 -- 172.21.15.62:0/934394194 learned_addr learned my addr 172.21.15.62:0/934394194 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.312+0000 7f363fe00640 1 -- 172.21.15.62:0/934394194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 msgr2=0x55d8f4266070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.312+0000 7f363fe00640 1 --2- 172.21.15.62:0/934394194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 0x55d8f4266070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.312+0000 7f363fe00640 1 -- 172.21.15.62:0/934394194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f4269080 msgr2=0x55d8f4266af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.312+0000 7f363fe00640 1 --2- 172.21.15.62:0/934394194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f4269080 0x55d8f4266af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.312+0000 7f363fe00640 1 -- 172.21.15.62:0/934394194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f3fa27d0 con 0x55d8f42609f0 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.312+0000 7f363fe00640 1 --2- 172.21.15.62:0/934394194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f42665b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f362c099460 tx=0x7f362c09c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d32c1aa89ffda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.312+0000 7f3626a00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.312+0000 7f3626a00640 1 -- 172.21.15.62:0/934394194 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0a0e40 con 0x55d8f42609f0 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.312+0000 7f3626a00640 1 -- 172.21.15.62:0/934394194 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f362c004830 con 0x55d8f42609f0 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.312+0000 7f364a98c740 1 -- 172.21.15.62:0/934394194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3634012af0 con 0x55d8f42609f0 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.312+0000 7f3626a00640 1 -- 172.21.15.62:0/934394194 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f362c0a5700 con 0x55d8f42609f0 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.313+0000 7f364a98c740 10 client.26390 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.313+0000 7f364a98c740 1 -- 172.21.15.62:0/934394194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d8f4277f80 con 0x55d8f42609f0 2024-04-02T14:46:58.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.313+0000 7f3626a00640 1 -- 172.21.15.62:0/934394194 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3634012af0 con 0x55d8f42609f0 2024-04-02T14:46:58.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.313+0000 7f3626a00640 1 -- 172.21.15.62:0/934394194 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f362c0a5270 con 0x55d8f42609f0 2024-04-02T14:46:58.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.313+0000 7f364a98c740 10 client.26390 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:58.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.313+0000 7f364a98c740 10 client.26390 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:46:58.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.313+0000 7f364a98c740 20 client.26390 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:58.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.313+0000 7f364a98c740 1 --2- 172.21.15.62:0/934394194 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55d8f4279480 0x55d8f427b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.313+0000 7f364a98c740 4 client.26390 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:58.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.313+0000 7f364a98c740 1 -- 172.21.15.62:0/934394194 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8f4184040 con 0x55d8f4279480 2024-04-02T14:46:58.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.314+0000 7f3645200640 1 --2- 172.21.15.62:0/934394194 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55d8f4279480 0x55d8f427b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.314+0000 7f3645200640 1 --2- 172.21.15.62:0/934394194 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55d8f4279480 0x55d8f427b870 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f306221e633f43bc server_cookie=d0fa912420b6c741 in_seq=0 out_seq=0 2024-04-02T14:46:58.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.314+0000 7f3626a00640 10 client.26390 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:46:58.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f3626a00640 1 -- 172.21.15.62:0/934394194 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8f4184040 con 0x55d8f4279480 2024-04-02T14:46:58.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f3626a00640 10 client.26390 handle_command_reply: tid=0 2024-04-02T14:46:58.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f364a98c740 1 client.26390 shutdown 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f364a98c740 1 -- 172.21.15.62:0/934394194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 msgr2=0x55d8f42665b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f364a98c740 1 --2- 172.21.15.62:0/934394194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f42665b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f362c099460 tx=0x7f362c09c4b0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f364a98c740 1 -- 172.21.15.62:0/934394194 shutdown_connections 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f364a98c740 1 --2- 172.21.15.62:0/934394194 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55d8f4279480 0x55d8f427b870 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f364a98c740 1 --2- 172.21.15.62:0/934394194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f4269080 0x55d8f4266af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f364a98c740 1 --2- 172.21.15.62:0/934394194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f42665b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f364a98c740 1 --2- 172.21.15.62:0/934394194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 0x55d8f4266070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f364a98c740 1 -- 172.21.15.62:0/934394194 >> 172.21.15.62:0/934394194 conn(0x55d8f4226a50 msgr2=0x55d8f425c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f3626a00640 0 client.26390 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.315+0000 7f364a98c740 1 -- 172.21.15.62:0/934394194 shutdown_connections 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.316+0000 7f364a98c740 1 -- 172.21.15.62:0/934394194 wait complete. 2024-04-02T14:46:58.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.316+0000 7f364a98c740 20 client.26390 trim_cache size 0 max 16384 2024-04-02T14:46:58.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.334+0000 7f364a98c740 1 Processor -- start 2024-04-02T14:46:58.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.334+0000 7f364a98c740 1 -- start start 2024-04-02T14:46:58.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.334+0000 7f364a98c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f425fd10 0x55d8f4274af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.334+0000 7f364a98c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f4265950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.334+0000 7f364a98c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f4269080 0x55d8f4259720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.334+0000 7f364a98c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36401b3e80 con 0x55d8f425fd10 2024-04-02T14:46:58.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.334+0000 7f364a98c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f36401b5b70 con 0x55d8f42609f0 2024-04-02T14:46:58.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.334+0000 7f364a98c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36401b45b0 con 0x55d8f4269080 2024-04-02T14:46:58.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f363fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f4265950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f363fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f4265950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55702/0 (socket says 172.21.15.62:55702) 2024-04-02T14:46:58.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f363fe00640 1 -- 172.21.15.62:0/1605721664 learned_addr learned my addr 172.21.15.62:0/1605721664 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f3645200640 1 --2- 172.21.15.62:0/1605721664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f425fd10 0x55d8f4274af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f3645c00640 1 --2- 172.21.15.62:0/1605721664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f4269080 0x55d8f4259720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f363fe00640 1 -- 172.21.15.62:0/1605721664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f4269080 msgr2=0x55d8f4259720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f363fe00640 1 --2- 172.21.15.62:0/1605721664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f4269080 0x55d8f4259720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f363fe00640 1 -- 172.21.15.62:0/1605721664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f425fd10 msgr2=0x55d8f4274af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f363fe00640 1 --2- 172.21.15.62:0/1605721664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f425fd10 0x55d8f4274af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f363fe00640 1 -- 172.21.15.62:0/1605721664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f425ed40 con 0x55d8f42609f0 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f3645200640 1 --2- 172.21.15.62:0/1605721664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f425fd10 0x55d8f4274af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f363fe00640 1 --2- 172.21.15.62:0/1605721664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f4265950 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f362c098ea0 tx=0x7f362c09c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0576bb59566bb38 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f3631600640 1 -- 172.21.15.62:0/1605721664 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0ce120 con 0x55d8f42609f0 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f3631600640 1 -- 172.21.15.62:0/1605721664 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f362c0a0e40 con 0x55d8f42609f0 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.335+0000 7f3631600640 1 -- 172.21.15.62:0/1605721664 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c09bf80 con 0x55d8f42609f0 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.336+0000 7f364a98c740 1 -- 172.21.15.62:0/1605721664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 msgr2=0x55d8f4265950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.336+0000 7f364a98c740 1 --2- 172.21.15.62:0/1605721664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f4265950 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f362c098ea0 tx=0x7f362c09c520 comp rx=0 tx=0).stop 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.336+0000 7f364a98c740 1 -- 172.21.15.62:0/1605721664 shutdown_connections 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.336+0000 7f364a98c740 1 --2- 172.21.15.62:0/1605721664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f4269080 0x55d8f4259720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.336+0000 7f364a98c740 1 --2- 172.21.15.62:0/1605721664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f42609f0 0x55d8f4265950 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.336+0000 7f364a98c740 1 --2- 172.21.15.62:0/1605721664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f425fd10 0x55d8f4274af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.336+0000 7f364a98c740 1 -- 172.21.15.62:0/1605721664 >> 172.21.15.62:0/1605721664 conn(0x55d8f4226a50 msgr2=0x55d8f4259010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.336+0000 7f364a98c740 1 -- 172.21.15.62:0/1605721664 shutdown_connections 2024-04-02T14:46:58.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.336+0000 7f364a98c740 1 -- 172.21.15.62:0/1605721664 wait complete. 2024-04-02T14:46:58.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.337+0000 7f364a98c740 1 -- start start 2024-04-02T14:46:58.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.337+0000 7f364a98c740 1 Processor -- start 2024-04-02T14:46:58.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.337+0000 7f364a98c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 0x55d8f4269e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.337+0000 7f364a98c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f42609f0 0x55d8f426a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.337+0000 7f364a98c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f4269080 0x55d8f426a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.337+0000 7f364a98c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36401b53e0 con 0x55d8f42609f0 2024-04-02T14:46:58.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.337+0000 7f364a98c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f36401b4cb0 con 0x55d8f4269080 2024-04-02T14:46:58.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.337+0000 7f364a98c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36401b1e60 con 0x55d8f425fd10 2024-04-02T14:46:58.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.337+0000 7f364a98c740 10 client.0 _ll_register_callbacks cb 0x7f362c009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3645200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 0x55d8f4269e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3645200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 0x55d8f4269e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:36396/0 (socket says 172.21.15.62:36396) 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3645200640 1 -- 172.21.15.62:0/3803310034 learned_addr learned my addr 172.21.15.62:0/3803310034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f363fe00640 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f42609f0 0x55d8f426a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3645c00640 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f4269080 0x55d8f426a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3645200640 1 -- 172.21.15.62:0/3803310034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f4269080 msgr2=0x55d8f426a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3645200640 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f4269080 0x55d8f426a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3645200640 1 -- 172.21.15.62:0/3803310034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f42609f0 msgr2=0x55d8f426a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3645200640 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f42609f0 0x55d8f426a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3645200640 1 -- 172.21.15.62:0/3803310034 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f426f5d0 con 0x55d8f425fd10 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3645c00640 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f4269080 0x55d8f426a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3645200640 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 0x55d8f4269e40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3634009c00 tx=0x7f363400f8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87ebaf3f64d07dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.338+0000 7f3626a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:46:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.339+0000 7f3626a00640 1 -- 172.21.15.62:0/3803310034 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363400f950 con 0x55d8f425fd10 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.339+0000 7f3626a00640 1 -- 172.21.15.62:0/3803310034 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f363400faf0 con 0x55d8f425fd10 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.339+0000 7f364a98c740 1 -- 172.21.15.62:0/3803310034 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f362c0a0e40 con 0x55d8f425fd10 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.339+0000 7f3626a00640 1 -- 172.21.15.62:0/3803310034 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f363403b050 con 0x55d8f425fd10 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.339+0000 7f364a98c740 10 client.26347 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.339+0000 7f364a98c740 1 -- 172.21.15.62:0/3803310034 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d8f42dfeb0 con 0x55d8f425fd10 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.339+0000 7f3626a00640 1 -- 172.21.15.62:0/3803310034 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f362c0a0e40 con 0x55d8f425fd10 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.340+0000 7f3626a00640 1 -- 172.21.15.62:0/3803310034 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f3634012710 con 0x55d8f425fd10 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.340+0000 7f364a98c740 10 client.26347 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.340+0000 7f364a98c740 10 client.26347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.340+0000 7f364a98c740 20 client.26347 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.340+0000 7f364a98c740 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55d8f42e0300 0x55d8f42e2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.340+0000 7f364a98c740 4 client.26347 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.340+0000 7f364a98c740 1 -- 172.21.15.62:0/3803310034 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8f41d1c30 con 0x55d8f42e0300 2024-04-02T14:46:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.340+0000 7f363fe00640 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55d8f42e0300 0x55d8f42e2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f363fe00640 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55d8f42e0300 0x55d8f42e2720 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e18a8b2e9748c06d server_cookie=23650f26f22521 in_seq=0 out_seq=0 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f3626a00640 10 client.26347 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f3626a00640 1 -- 172.21.15.62:0/3803310034 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8f41d1c30 con 0x55d8f42e0300 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f3626a00640 10 client.26347 handle_command_reply: tid=0 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 1 client.26347 shutdown 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 1 -- 172.21.15.62:0/3803310034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 msgr2=0x55d8f4269e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 0x55d8f4269e40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3634009c00 tx=0x7f363400f8e0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 1 -- 172.21.15.62:0/3803310034 shutdown_connections 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55d8f42e0300 0x55d8f42e2720 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d8f4269080 0x55d8f426a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8f42609f0 0x55d8f426a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 1 --2- 172.21.15.62:0/3803310034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f425fd10 0x55d8f4269e40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f3626a00640 0 client.26347 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:46:58.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 1 -- 172.21.15.62:0/3803310034 >> 172.21.15.62:0/3803310034 conn(0x55d8f4226a50 msgr2=0x55d8f42598a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 1 -- 172.21.15.62:0/3803310034 shutdown_connections 2024-04-02T14:46:58.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 1 -- 172.21.15.62:0/3803310034 wait complete. 2024-04-02T14:46:58.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.341+0000 7f364a98c740 20 client.26347 trim_cache size 0 max 16384 2024-04-02T14:46:58.340 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 -- 172.21.15.62:0/1788461413 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f36280a3da0 msgr2=0x7f36280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f36280a3da0 0x7f36280a6260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f36380099a0 tx=0x7f3638009500 comp rx=0 tx=0).stop 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 -- 172.21.15.62:0/1788461413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364010d3e0 msgr2=0x7f36401a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364010d3e0 0x7f36401a4c40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e9f0 tx=0x7f362c00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 -- 172.21.15.62:0/1788461413 shutdown_connections 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f36280a3da0 0x7f36280a6260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f364019e8b0 0x7f364019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364010d3e0 0x7f36401a4c40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 --2- 172.21.15.62:0/1788461413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3640108310 0x7f36401a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 -- 172.21.15.62:0/1788461413 >> 172.21.15.62:0/1788461413 conn(0x7f3640076040 msgr2=0x7f3640076bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 -- 172.21.15.62:0/1788461413 shutdown_connections 2024-04-02T14:46:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.342+0000 7f3647c00640 1 -- 172.21.15.62:0/1788461413 wait complete. 2024-04-02T14:46:58.353 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:46:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.563+0000 7f447e400640 1 -- 172.21.15.62:0/3657698095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4478102ab0 msgr2=0x7f4478102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.563+0000 7f447e400640 1 --2- 172.21.15.62:0/3657698095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4478102ab0 0x7f4478102e90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f446c009900 tx=0x7f446c02bcc0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.564+0000 7f447e400640 1 -- 172.21.15.62:0/3657698095 shutdown_connections 2024-04-02T14:46:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.564+0000 7f447e400640 1 --2- 172.21.15.62:0/3657698095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447810bed0 0x7f447810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.564+0000 7f447e400640 1 --2- 172.21.15.62:0/3657698095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44781033d0 0x7f447810b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.564+0000 7f447e400640 1 --2- 172.21.15.62:0/3657698095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4478102ab0 0x7f4478102e90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.564+0000 7f447e400640 1 -- 172.21.15.62:0/3657698095 >> 172.21.15.62:0/3657698095 conn(0x7f44780fc310 msgr2=0x7f44780fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.565+0000 7f447e400640 1 -- 172.21.15.62:0/3657698095 shutdown_connections 2024-04-02T14:46:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.565+0000 7f447e400640 1 -- 172.21.15.62:0/3657698095 wait complete. 2024-04-02T14:46:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.566+0000 7f447e400640 1 Processor -- start 2024-04-02T14:46:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.566+0000 7f447e400640 1 -- start start 2024-04-02T14:46:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.567+0000 7f447e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4478102ab0 0x7f4478198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.567+0000 7f447e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44781033d0 0x7f44781987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.567+0000 7f447e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f447810bed0 0x7f447819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.567+0000 7f447e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4478111df0 con 0x7f447810bed0 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.567+0000 7f447e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4478111c70 con 0x7f44781033d0 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.567+0000 7f447e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4478111f70 con 0x7f4478102ab0 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.567+0000 7f4477e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44781033d0 0x7f44781987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.567+0000 7f4477e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44781033d0 0x7f44781987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55732/0 (socket says 172.21.15.62:55732) 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.567+0000 7f4477e00640 1 -- 172.21.15.62:0/2642595505 learned_addr learned my addr 172.21.15.62:0/2642595505 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.567+0000 7f447da00640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f447810bed0 0x7f447819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.568+0000 7f447d000640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4478102ab0 0x7f4478198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.568+0000 7f4477e00640 1 -- 172.21.15.62:0/2642595505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4478102ab0 msgr2=0x7f4478198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.568+0000 7f4477e00640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4478102ab0 0x7f4478198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.568+0000 7f4477e00640 1 -- 172.21.15.62:0/2642595505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f447810bed0 msgr2=0x7f447819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.568+0000 7f4477e00640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f447810bed0 0x7f447819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.568+0000 7f4477e00640 1 -- 172.21.15.62:0/2642595505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447819d090 con 0x7f44781033d0 2024-04-02T14:46:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.568+0000 7f447da00640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f447810bed0 0x7f447819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.568+0000 7f4477e00640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44781033d0 0x7f44781987d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f446400d770 tx=0x7f446400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.569+0000 7f4475600640 1 -- 172.21.15.62:0/2642595505 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464017070 con 0x7f44781033d0 2024-04-02T14:46:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.569+0000 7f4475600640 1 -- 172.21.15.62:0/2642595505 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f446400de80 con 0x7f44781033d0 2024-04-02T14:46:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.569+0000 7f4475600640 1 -- 172.21.15.62:0/2642595505 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446400f620 con 0x7f44781033d0 2024-04-02T14:46:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.569+0000 7f447e400640 1 -- 172.21.15.62:0/2642595505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447819d380 con 0x7f44781033d0 2024-04-02T14:46:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.569+0000 7f447e400640 1 -- 172.21.15.62:0/2642595505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447819d880 con 0x7f44781033d0 2024-04-02T14:46:58.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.570+0000 7f71af200640 1 -- 172.21.15.62:0/3560027838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8112940 msgr2=0x7f71a8114d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.570+0000 7f71af200640 1 --2- 172.21.15.62:0/3560027838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8112940 0x7f71a8114d30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009c10 tx=0x7f71a002bef0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.570+0000 7f71af200640 1 -- 172.21.15.62:0/3560027838 shutdown_connections 2024-04-02T14:46:58.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.570+0000 7f71af200640 1 --2- 172.21.15.62:0/3560027838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8112940 0x7f71a8114d30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.570+0000 7f71af200640 1 --2- 172.21.15.62:0/3560027838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a8076950 0x7f71a8112400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.570+0000 7f71af200640 1 --2- 172.21.15.62:0/3560027838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8076030 0x7f71a8076410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.570+0000 7f71af200640 1 -- 172.21.15.62:0/3560027838 >> 172.21.15.62:0/3560027838 conn(0x7f71a80ff770 msgr2=0x7f71a8101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.570+0000 7f71af200640 1 -- 172.21.15.62:0/3560027838 shutdown_connections 2024-04-02T14:46:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.571+0000 7f71af200640 1 -- 172.21.15.62:0/3560027838 wait complete. 2024-04-02T14:46:58.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.572+0000 7f71af200640 1 Processor -- start 2024-04-02T14:46:58.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.572+0000 7f71af200640 1 -- start start 2024-04-02T14:46:58.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.572+0000 7f71af200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a8076030 0x7f71a81a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71af200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8076950 0x7f71a81a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71af200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8112940 0x7f71a819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71af200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f71a81185c0 con 0x7f71a8076030 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71af200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8118440 con 0x7f71a8112940 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71af200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8118740 con 0x7f71a8076950 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a8076030 0x7f71a81a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a8076030 0x7f71a81a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42290/0 (socket says 172.21.15.62:42290) 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71a7e00640 1 -- 172.21.15.62:0/4245152221 learned_addr learned my addr 172.21.15.62:0/4245152221 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71ad200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8112940 0x7f71a819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71a7400640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8076950 0x7f71a81a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71ad200640 1 -- 172.21.15.62:0/4245152221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8076950 msgr2=0x7f71a81a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71ad200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8076950 0x7f71a81a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.573+0000 7f71ad200640 1 -- 172.21.15.62:0/4245152221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a8076030 msgr2=0x7f71a81a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.574+0000 7f71ad200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a8076030 0x7f71a81a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.574+0000 7f71ad200640 1 -- 172.21.15.62:0/4245152221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a819f060 con 0x7f71a8112940 2024-04-02T14:46:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.574+0000 7f71a7400640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8076950 0x7f71a81a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.574+0000 7f71ad200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8112940 0x7f71a819e8e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f71a002c500 tx=0x7f71a0033720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.574+0000 7f71a4c00640 1 -- 172.21.15.62:0/4245152221 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0043020 con 0x7f71a8112940 2024-04-02T14:46:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.574+0000 7f71a4c00640 1 -- 172.21.15.62:0/4245152221 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f71a0004040 con 0x7f71a8112940 2024-04-02T14:46:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.574+0000 7f71a4c00640 1 -- 172.21.15.62:0/4245152221 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a003e670 con 0x7f71a8112940 2024-04-02T14:46:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.574+0000 7f71af200640 1 -- 172.21.15.62:0/4245152221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a819f2f0 con 0x7f71a8112940 2024-04-02T14:46:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.574+0000 7f71af200640 1 -- 172.21.15.62:0/4245152221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a81ab5a0 con 0x7f71a8112940 2024-04-02T14:46:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.575+0000 7f448280d740 1 Processor -- start 2024-04-02T14:46:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.575+0000 7f4475600640 1 -- 172.21.15.62:0/2642595505 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4464009c30 con 0x7f44781033d0 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.575+0000 7f448280d740 1 -- start start 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.576+0000 7f448280d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b464c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.576+0000 7f448280d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.576+0000 7f448280d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.576+0000 7f448280d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b9d6aaad70 con 0x55b9d6b460e0 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.576+0000 7f448280d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b9d6a532a0 con 0x55b9d6b4f5e0 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.576+0000 7f448280d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b9d6aa6d20 con 0x55b9d6b46a00 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.576+0000 7f447da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.576+0000 7f447da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b464c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42304/0 (socket says 172.21.15.62:42304) 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.576+0000 7f447da00640 1 -- 172.21.15.62:0/3480403133 learned_addr learned my addr 172.21.15.62:0/3480403133 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.576+0000 7f4477e00640 1 --2- 172.21.15.62:0/3480403133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.577+0000 7f447da00640 1 -- 172.21.15.62:0/3480403133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b46a00 msgr2=0x55b9d6b4f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.577+0000 7f447da00640 1 --2- 172.21.15.62:0/3480403133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.577+0000 7f447da00640 1 -- 172.21.15.62:0/3480403133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 msgr2=0x55b9d6b519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.577+0000 7f447da00640 1 --2- 172.21.15.62:0/3480403133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.577+0000 7f447da00640 1 -- 172.21.15.62:0/3480403133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d6a67840 con 0x55b9d6b460e0 2024-04-02T14:46:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.577+0000 7f4477e00640 1 --2- 172.21.15.62:0/3480403133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.577+0000 7f4475600640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44600a3d90 0x7f44600a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.578+0000 7f447da00640 1 --2- 172.21.15.62:0/3480403133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b464c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4470009fd0 tx=0x7f4470010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748f8d6126d940d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.578+0000 7f4475600640 1 -- 172.21.15.62:0/2642595505 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f44640d04f0 con 0x7f44781033d0 2024-04-02T14:46:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.578+0000 7f447d000640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44600a3d90 0x7f44600a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.578+0000 7f446a000640 1 -- 172.21.15.62:0/3480403133 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4470011700 con 0x55b9d6b460e0 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.578+0000 7f446a000640 1 -- 172.21.15.62:0/3480403133 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4470011da0 con 0x55b9d6b460e0 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.578+0000 7f446a000640 1 -- 172.21.15.62:0/3480403133 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447000e3c0 con 0x55b9d6b460e0 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f447d000640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44600a3d90 0x7f44600a6250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55b9d6b45bb0 tx=0x7f446c005d50 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f448280d740 1 -- 172.21.15.62:0/3480403133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 msgr2=0x55b9d6b464c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f448280d740 1 --2- 172.21.15.62:0/3480403133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b464c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4470009fd0 tx=0x7f4470010860 comp rx=0 tx=0).stop 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f448280d740 1 -- 172.21.15.62:0/3480403133 shutdown_connections 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f448280d740 1 --2- 172.21.15.62:0/3480403133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f448280d740 1 --2- 172.21.15.62:0/3480403133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f448280d740 1 --2- 172.21.15.62:0/3480403133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b464c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f448280d740 1 -- 172.21.15.62:0/3480403133 >> 172.21.15.62:0/3480403133 conn(0x55b9d6b40810 msgr2=0x55b9d6b40bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f448280d740 1 -- 172.21.15.62:0/3480403133 shutdown_connections 2024-04-02T14:46:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f448280d740 1 -- 172.21.15.62:0/3480403133 wait complete. 2024-04-02T14:46:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.578+0000 7f71a4c00640 1 -- 172.21.15.62:0/4245152221 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f71a0046020 con 0x7f71a8112940 2024-04-02T14:46:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f71b1fdf740 1 Processor -- start 2024-04-02T14:46:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.579+0000 7f71b1fdf740 1 -- start start 2024-04-02T14:46:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd8163740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd8164190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd817e090 0x555cd8180480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555cd80a4400 con 0x555cd81649f0 2024-04-02T14:46:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x555cd8088f10 con 0x555cd8163d10 2024-04-02T14:46:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555cd80c86f0 con 0x555cd817e090 2024-04-02T14:46:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f448280d740 1 -- start start 2024-04-02T14:46:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71a4c00640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f718c0a3de0 0x7f718c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71a7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd8163740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71a7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd8163740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42312/0 (socket says 172.21.15.62:42312) 2024-04-02T14:46:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71a7400640 1 -- 172.21.15.62:0/1149853515 learned_addr learned my addr 172.21.15.62:0/1149853515 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f448280d740 1 Processor -- start 2024-04-02T14:46:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71a4c00640 1 -- 172.21.15.62:0/4245152221 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f71a00f2cb0 con 0x7f71a8112940 2024-04-02T14:46:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.580+0000 7f71a7400640 1 --2- 172.21.15.62:0/1149853515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd817e090 0x555cd8180480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f71ad200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f718c0a3de0 0x7f718c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f71a7400640 1 -- 172.21.15.62:0/1149853515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd817e090 msgr2=0x555cd8180480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f71a7400640 1 --2- 172.21.15.62:0/1149853515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd817e090 0x555cd8180480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f71a7400640 1 -- 172.21.15.62:0/1149853515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 msgr2=0x555cd8164190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T14:46:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f71a7400640 1 --2- 172.21.15.62:0/1149853515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd8164190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f71a7400640 1 -- 172.21.15.62:0/1149853515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd7ddc3f0 con 0x555cd81649f0 2024-04-02T14:46:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f71a7400640 1 --2- 172.21.15.62:0/1149853515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd8163740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7194012510 tx=0x7f7194009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4b19fa9c046603 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f71ad200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f718c0a3de0 0x7f718c0a62a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71a0006010 tx=0x7f71a0009500 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f448280d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b460e0 0x55b9d6b4c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f448280d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f448280d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b4d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f448280d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b9d6b633e0 con 0x55b9d6b4f5e0 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f448280d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b9d6a66660 con 0x55b9d6b46a00 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f448280d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b9d6b63560 con 0x55b9d6b460e0 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f448280d740 10 client.0 _ll_register_callbacks cb 0x55b9d6a66050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f4477e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f4477e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55772/0 (socket says 172.21.15.62:55772) 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f4477e00640 1 -- 172.21.15.62:0/1410138250 learned_addr learned my addr 172.21.15.62:0/1410138250 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.581+0000 7f447d000640 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b460e0 0x55b9d6b4c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f4477e00640 1 -- 172.21.15.62:0/1410138250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b460e0 msgr2=0x55b9d6b4c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f4477e00640 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b460e0 0x55b9d6b4c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f4477e00640 1 -- 172.21.15.62:0/1410138250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b4f5e0 msgr2=0x55b9d6b4d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f4477e00640 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b4d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f4477e00640 1 -- 172.21.15.62:0/1410138250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d6a75e80 con 0x55b9d6b46a00 2024-04-02T14:46:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f719d600640 1 -- 172.21.15.62:0/1149853515 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194005020 con 0x555cd81649f0 2024-04-02T14:46:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f719d600640 1 -- 172.21.15.62:0/1149853515 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f71940047a0 con 0x555cd81649f0 2024-04-02T14:46:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f719d600640 1 -- 172.21.15.62:0/1149853515 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194010750 con 0x555cd81649f0 2024-04-02T14:46:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f4477e00640 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4cb90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f44640cdc90 tx=0x7f446409c250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f402b051e904ba22 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f445ea00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f445ea00640 1 -- 172.21.15.62:0/1410138250 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464020070 con 0x55b9d6b46a00 2024-04-02T14:46:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f448280d740 1 -- 172.21.15.62:0/1410138250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4470011da0 con 0x55b9d6b46a00 2024-04-02T14:46:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f445ea00640 1 -- 172.21.15.62:0/1410138250 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f446409b8d0 con 0x55b9d6b46a00 2024-04-02T14:46:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f445ea00640 1 -- 172.21.15.62:0/1410138250 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f44640ce0b0 con 0x55b9d6b46a00 2024-04-02T14:46:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f448280d740 10 client.26414 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.582+0000 7f448280d740 1 -- 172.21.15.62:0/1410138250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b9d67bf210 con 0x55b9d6b46a00 2024-04-02T14:46:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f445ea00640 1 -- 172.21.15.62:0/1410138250 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4470011da0 con 0x55b9d6b46a00 2024-04-02T14:46:58.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1149853515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 msgr2=0x555cd8163740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/1149853515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd8163740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7194012510 tx=0x7f7194009790 comp rx=0 tx=0).stop 2024-04-02T14:46:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f445ea00640 1 -- 172.21.15.62:0/1410138250 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f44640d0020 con 0x55b9d6b46a00 2024-04-02T14:46:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f448280d740 10 client.26414 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f448280d740 10 client.26414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f448280d740 20 client.26414 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1149853515 shutdown_connections 2024-04-02T14:46:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/1149853515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd817e090 0x555cd8180480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/1149853515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd8164190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/1149853515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd8163740 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1149853515 >> 172.21.15.62:0/1149853515 conn(0x555cd812aa50 msgr2=0x555cd815e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1149853515 shutdown_connections 2024-04-02T14:46:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f448280d740 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b9d6b70a80 0x55b9d6b72e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f448280d740 4 client.26414 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f448280d740 1 -- 172.21.15.62:0/1410138250 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9d6a5acd0 con 0x55b9d6b70a80 2024-04-02T14:46:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.584+0000 7f447d000640 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b9d6b70a80 0x55b9d6b72e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1149853515 wait complete. 2024-04-02T14:46:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.583+0000 7f71b1fdf740 1 -- start start 2024-04-02T14:46:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.584+0000 7f71b1fdf740 1 Processor -- start 2024-04-02T14:46:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.584+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd817b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.584+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd81649f0 0x555cd817b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.584+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd817e090 0x555cd817bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.584+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555cd80c46a0 con 0x555cd817e090 2024-04-02T14:46:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.584+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x555cd8070980 con 0x555cd8163d10 2024-04-02T14:46:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.584+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555cd8083cb0 con 0x555cd81649f0 2024-04-02T14:46:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.584+0000 7f71b1fdf740 10 client.0 _ll_register_callbacks cb 0x555cd80836a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.584+0000 7f71a7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd817b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f71a7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd817b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55776/0 (socket says 172.21.15.62:55776) 2024-04-02T14:46:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f71a7400640 1 -- 172.21.15.62:0/4081593026 learned_addr learned my addr 172.21.15.62:0/4081593026 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.584+0000 7f447d000640 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b9d6b70a80 0x55b9d6b72e90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c330e3e473e380 server_cookie=ce04955c65c4e60d in_seq=0 out_seq=0 2024-04-02T14:46:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f445ea00640 10 client.26414 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f71a7400640 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd817e090 0x555cd817bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f71a7e00640 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd81649f0 0x555cd817b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f71a7400640 1 -- 172.21.15.62:0/4081593026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd81649f0 msgr2=0x555cd817b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f71a7400640 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd81649f0 0x555cd817b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f71a7400640 1 -- 172.21.15.62:0/4081593026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd817e090 msgr2=0x555cd817bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f71a7400640 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd817e090 0x555cd817bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f71a7400640 1 -- 172.21.15.62:0/4081593026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd7ea67d0 con 0x555cd8163d10 2024-04-02T14:46:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f71a7e00640 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd81649f0 0x555cd817b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f445ea00640 1 -- 172.21.15.62:0/1410138250 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9d6a5acd0 con 0x55b9d6b70a80 2024-04-02T14:46:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f445ea00640 10 client.26414 handle_command_reply: tid=0 2024-04-02T14:46:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f71a7400640 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd817b140 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7194004770 tx=0x7f71940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b483137640dc0a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.585+0000 7f7192000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 1 client.26414 shutdown 2024-04-02T14:46:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f7192000640 1 -- 172.21.15.62:0/4081593026 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71940052b0 con 0x555cd8163d10 2024-04-02T14:46:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 1 -- 172.21.15.62:0/1410138250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 msgr2=0x55b9d6b4cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4cb90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f44640cdc90 tx=0x7f446409c250 comp rx=0 tx=0).stop 2024-04-02T14:46:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f7192000640 1 -- 172.21.15.62:0/4081593026 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f7194005450 con 0x555cd8163d10 2024-04-02T14:46:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f71b1fdf740 1 -- 172.21.15.62:0/4081593026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71940047a0 con 0x555cd8163d10 2024-04-02T14:46:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 1 -- 172.21.15.62:0/1410138250 shutdown_connections 2024-04-02T14:46:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b9d6b70a80 0x55b9d6b72e90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b4d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4cb90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 1 --2- 172.21.15.62:0/1410138250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b460e0 0x55b9d6b4c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 1 -- 172.21.15.62:0/1410138250 >> 172.21.15.62:0/1410138250 conn(0x55b9d6b40810 msgr2=0x55b9d6b42560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f445ea00640 0 client.26414 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f7192000640 1 -- 172.21.15.62:0/4081593026 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7194034050 con 0x555cd8163d10 2024-04-02T14:46:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 1 -- 172.21.15.62:0/1410138250 shutdown_connections 2024-04-02T14:46:58.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f71b1fdf740 10 client.26417 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:58.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f71b1fdf740 1 -- 172.21.15.62:0/4081593026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555cd818cff0 con 0x555cd8163d10 2024-04-02T14:46:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 1 -- 172.21.15.62:0/1410138250 wait complete. 2024-04-02T14:46:58.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f7192000640 1 -- 172.21.15.62:0/4081593026 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f71940047a0 con 0x555cd8163d10 2024-04-02T14:46:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.586+0000 7f448280d740 20 client.26414 trim_cache size 0 max 16384 2024-04-02T14:46:58.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.587+0000 7f7192000640 1 -- 172.21.15.62:0/4081593026 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f71940039f0 con 0x555cd8163d10 2024-04-02T14:46:58.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.587+0000 7f71b1fdf740 10 client.26417 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:58.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.587+0000 7f71b1fdf740 10 client.26417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:46:58.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.587+0000 7f71b1fdf740 20 client.26417 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.587+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x555cd818e340 0x555cd8190730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.587+0000 7f71b1fdf740 4 client.26417 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.587+0000 7f71b1fdf740 1 -- 172.21.15.62:0/4081593026 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cd8088040 con 0x555cd818e340 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.587+0000 7f71a7e00640 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x555cd818e340 0x555cd8190730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.588+0000 7f71a7e00640 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x555cd818e340 0x555cd8190730 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c7cc1661309c731 server_cookie=84d26bd61207572 in_seq=0 out_seq=0 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.588+0000 7f7192000640 10 client.26417 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.589+0000 7f7192000640 1 -- 172.21.15.62:0/4081593026 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cd8088040 con 0x555cd818e340 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.589+0000 7f7192000640 10 client.26417 handle_command_reply: tid=0 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.589+0000 7f71b1fdf740 1 client.26417 shutdown 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f71b1fdf740 1 -- 172.21.15.62:0/4081593026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 msgr2=0x555cd817b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd817b140 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7194004770 tx=0x7f71940046c0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f71b1fdf740 1 -- 172.21.15.62:0/4081593026 shutdown_connections 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x555cd818e340 0x555cd8190730 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd817e090 0x555cd817bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd81649f0 0x555cd817b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/4081593026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd817b140 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f71b1fdf740 1 -- 172.21.15.62:0/4081593026 >> 172.21.15.62:0/4081593026 conn(0x555cd812aa50 msgr2=0x555cd815d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f7192000640 0 client.26417 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:46:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f71b1fdf740 1 -- 172.21.15.62:0/4081593026 shutdown_connections 2024-04-02T14:46:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f71b1fdf740 1 -- 172.21.15.62:0/4081593026 wait complete. 2024-04-02T14:46:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.590+0000 7f71b1fdf740 20 client.26417 trim_cache size 0 max 16384 2024-04-02T14:46:58.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.605+0000 7f71b1fdf740 1 Processor -- start 2024-04-02T14:46:58.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.605+0000 7f71b1fdf740 1 -- start start 2024-04-02T14:46:58.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.605+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd8189b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.605+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd817b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.605+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd817e090 0x555cd8195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.605+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f71a810db40 con 0x555cd81649f0 2024-04-02T14:46:58.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.605+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f71a81ad2b0 con 0x555cd8163d10 2024-04-02T14:46:58.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.605+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f71a81abcf0 con 0x555cd817e090 2024-04-02T14:46:58.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd817b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd817b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42344/0 (socket says 172.21.15.62:42344) 2024-04-02T14:46:58.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f71a7e00640 1 -- 172.21.15.62:0/3234524318 learned_addr learned my addr 172.21.15.62:0/3234524318 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f71a7400640 1 --2- 172.21.15.62:0/3234524318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd8189b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f71a7400640 1 --2- 172.21.15.62:0/3234524318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd817e090 0x555cd8195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f448280d740 1 Processor -- start 2024-04-02T14:46:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f448280d740 1 -- start start 2024-04-02T14:46:58.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f71a7e00640 1 -- 172.21.15.62:0/3234524318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd817e090 msgr2=0x555cd8195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f71a7e00640 1 --2- 172.21.15.62:0/3234524318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd817e090 0x555cd8195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f71a7e00640 1 -- 172.21.15.62:0/3234524318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 msgr2=0x555cd8189b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f71a7e00640 1 --2- 172.21.15.62:0/3234524318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd8189b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f71a7e00640 1 -- 172.21.15.62:0/3234524318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd8162f70 con 0x555cd81649f0 2024-04-02T14:46:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.606+0000 7f448280d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b6c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f448280d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f448280d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b787a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f448280d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4478073b80 con 0x55b9d6b460e0 2024-04-02T14:46:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f448280d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f44781a4930 con 0x55b9d6b46a00 2024-04-02T14:46:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f448280d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44781a3370 con 0x55b9d6b4f5e0 2024-04-02T14:46:58.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f71a7e00640 1 --2- 172.21.15.62:0/3234524318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd817b9e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7198009810 tx=0x7f7198010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a27688319e907f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f447d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b6c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f4477e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f447d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b6c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42356/0 (socket says 172.21.15.62:42356) 2024-04-02T14:46:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f447d000640 1 -- 172.21.15.62:0/471543758 learned_addr learned my addr 172.21.15.62:0/471543758 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f447da00640 1 --2- 172.21.15.62:0/471543758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f4477e00640 1 -- 172.21.15.62:0/471543758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b4f5e0 msgr2=0x55b9d6b787a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f4477e00640 1 --2- 172.21.15.62:0/471543758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b787a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f4477e00640 1 -- 172.21.15.62:0/471543758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 msgr2=0x55b9d6b6c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f4477e00640 1 --2- 172.21.15.62:0/471543758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b6c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f4477e00640 1 -- 172.21.15.62:0/471543758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d688b620 con 0x55b9d6b46a00 2024-04-02T14:46:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f719cc00640 1 -- 172.21.15.62:0/3234524318 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719801d470 con 0x555cd81649f0 2024-04-02T14:46:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f719cc00640 1 -- 172.21.15.62:0/3234524318 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7198005520 con 0x555cd81649f0 2024-04-02T14:46:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.607+0000 7f719cc00640 1 -- 172.21.15.62:0/3234524318 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719801c3d0 con 0x555cd81649f0 2024-04-02T14:46:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.608+0000 7f4477e00640 1 --2- 172.21.15.62:0/471543758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4cef0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f44640988e0 tx=0x7f446409c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55beef1efd8f5a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.608+0000 7f4469600640 1 -- 172.21.15.62:0/471543758 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464014030 con 0x55b9d6b46a00 2024-04-02T14:46:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.608+0000 7f4469600640 1 -- 172.21.15.62:0/471543758 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f446409c480 con 0x55b9d6b46a00 2024-04-02T14:46:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.608+0000 7f4469600640 1 -- 172.21.15.62:0/471543758 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44640ce0c0 con 0x55b9d6b46a00 2024-04-02T14:46:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.608+0000 7f71b1fdf740 1 -- 172.21.15.62:0/3234524318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 msgr2=0x555cd817b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.608+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/3234524318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd817b9e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7198009810 tx=0x7f7198010800 comp rx=0 tx=0).stop 2024-04-02T14:46:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.608+0000 7f71b1fdf740 1 -- 172.21.15.62:0/3234524318 shutdown_connections 2024-04-02T14:46:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.608+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/3234524318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd817e090 0x555cd8195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.608+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/3234524318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd81649f0 0x555cd817b9e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.608+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/3234524318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd8163d10 0x555cd8189b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.608+0000 7f71b1fdf740 1 -- 172.21.15.62:0/3234524318 >> 172.21.15.62:0/3234524318 conn(0x555cd812aa50 msgr2=0x555cd815e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f71b1fdf740 1 -- 172.21.15.62:0/3234524318 shutdown_connections 2024-04-02T14:46:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f71b1fdf740 1 -- 172.21.15.62:0/3234524318 wait complete. 2024-04-02T14:46:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f448280d740 1 -- 172.21.15.62:0/471543758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 msgr2=0x55b9d6b4cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f448280d740 1 --2- 172.21.15.62:0/471543758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4cef0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f44640988e0 tx=0x7f446409c380 comp rx=0 tx=0).stop 2024-04-02T14:46:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f448280d740 1 -- 172.21.15.62:0/471543758 shutdown_connections 2024-04-02T14:46:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f448280d740 1 --2- 172.21.15.62:0/471543758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b787a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f448280d740 1 --2- 172.21.15.62:0/471543758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b46a00 0x55b9d6b4cef0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f448280d740 1 --2- 172.21.15.62:0/471543758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b6c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f448280d740 1 -- 172.21.15.62:0/471543758 >> 172.21.15.62:0/471543758 conn(0x55b9d6b40810 msgr2=0x55b9d6b3f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f448280d740 1 -- 172.21.15.62:0/471543758 shutdown_connections 2024-04-02T14:46:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f448280d740 1 -- 172.21.15.62:0/471543758 wait complete. 2024-04-02T14:46:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f71b1fdf740 1 -- start start 2024-04-02T14:46:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f71b1fdf740 1 Processor -- start 2024-04-02T14:46:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.609+0000 7f448280d740 1 -- start start 2024-04-02T14:46:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f448280d740 1 Processor -- start 2024-04-02T14:46:58.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd8163d10 0x555cd8186330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd81649f0 0x555cd817ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71b1fdf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd817e090 0x555cd817efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f71a81acb20 con 0x555cd817e090 2024-04-02T14:46:58.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f71a81ac3f0 con 0x555cd81649f0 2024-04-02T14:46:58.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71b1fdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f71a810bb20 con 0x555cd8163d10 2024-04-02T14:46:58.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71b1fdf740 10 client.0 _ll_register_callbacks cb 0x7f7194004da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:58.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd81649f0 0x555cd817ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd81649f0 0x555cd817ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55804/0 (socket says 172.21.15.62:55804) 2024-04-02T14:46:58.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71a7e00640 1 -- 172.21.15.62:0/1966382842 learned_addr learned my addr 172.21.15.62:0/1966382842 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71a7400640 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd8163d10 0x555cd8186330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f448280d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71a7400640 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd817e090 0x555cd817efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f448280d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b46a00 0x55b9d6b68d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f448280d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b63f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f448280d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f44781a41a0 con 0x55b9d6b460e0 2024-04-02T14:46:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f448280d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f44781a3a70 con 0x55b9d6b4f5e0 2024-04-02T14:46:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f448280d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44781995e0 con 0x55b9d6b46a00 2024-04-02T14:46:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f448280d740 10 client.0 _ll_register_callbacks cb 0x7f4464098060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:58.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71a7e00640 1 -- 172.21.15.62:0/1966382842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd8163d10 msgr2=0x555cd8186330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71a7e00640 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd8163d10 0x555cd8186330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71a7e00640 1 -- 172.21.15.62:0/1966382842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd817e090 msgr2=0x555cd817efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71a7e00640 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd817e090 0x555cd817efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.610+0000 7f71a7e00640 1 -- 172.21.15.62:0/1966382842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd817f710 con 0x555cd81649f0 2024-04-02T14:46:58.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f71a7400640 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd817e090 0x555cd817efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f4477e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b46a00 0x55b9d6b68d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f4477e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b46a00 0x55b9d6b68d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:36520/0 (socket says 172.21.15.62:36520) 2024-04-02T14:46:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f4477e00640 1 -- 172.21.15.62:0/842240676 learned_addr learned my addr 172.21.15.62:0/842240676 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f447da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b63f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f447d000640 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f71a7e00640 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd81649f0 0x555cd817ea90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7198005e00 tx=0x7f7198005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25b969b348afd0bc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f7192000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f4477e00640 1 -- 172.21.15.62:0/842240676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 msgr2=0x55b9d6b63f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f4477e00640 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b63f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f4477e00640 1 -- 172.21.15.62:0/842240676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 msgr2=0x55b9d6b687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f4477e00640 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f4477e00640 1 -- 172.21.15.62:0/842240676 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d6b64740 con 0x55b9d6b46a00 2024-04-02T14:46:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f447da00640 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b63f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:46:58.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f7192000640 1 -- 172.21.15.62:0/1966382842 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719802b020 con 0x555cd81649f0 2024-04-02T14:46:58.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1966382842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7198005520 con 0x555cd81649f0 2024-04-02T14:46:58.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f7192000640 1 -- 172.21.15.62:0/1966382842 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f7198005020 con 0x555cd81649f0 2024-04-02T14:46:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f4477e00640 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b46a00 0x55b9d6b68d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4464000c00 tx=0x7f4464098dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e73d2c112545153 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.611+0000 7f445ea00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:46:58.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f7192000640 1 -- 172.21.15.62:0/1966382842 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7198005220 con 0x555cd81649f0 2024-04-02T14:46:58.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f71b1fdf740 10 client.26426 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:58.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1966382842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555cd81f4760 con 0x555cd81649f0 2024-04-02T14:46:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f445ea00640 1 -- 172.21.15.62:0/842240676 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464014070 con 0x55b9d6b46a00 2024-04-02T14:46:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f448280d740 1 -- 172.21.15.62:0/842240676 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f446409c480 con 0x55b9d6b46a00 2024-04-02T14:46:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f445ea00640 1 -- 172.21.15.62:0/842240676 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f44640a19c0 con 0x55b9d6b46a00 2024-04-02T14:46:58.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f7192000640 1 -- 172.21.15.62:0/1966382842 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7198005520 con 0x555cd81649f0 2024-04-02T14:46:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f445ea00640 1 -- 172.21.15.62:0/842240676 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f44640b4070 con 0x55b9d6b46a00 2024-04-02T14:46:58.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f7192000640 1 -- 172.21.15.62:0/1966382842 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f7198027900 con 0x555cd81649f0 2024-04-02T14:46:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f448280d740 10 client.26380 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f448280d740 1 -- 172.21.15.62:0/842240676 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b9d6bd6cf0 con 0x55b9d6b46a00 2024-04-02T14:46:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f71b1fdf740 10 client.26426 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f71b1fdf740 10 client.26426 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:46:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f71b1fdf740 20 client.26426 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x555cd81f4dd0 0x555cd81f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f71b1fdf740 4 client.26426 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:46:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1966382842 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cd80d5c30 con 0x555cd81f4dd0 2024-04-02T14:46:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.612+0000 7f445ea00640 1 -- 172.21.15.62:0/842240676 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f446409c480 con 0x55b9d6b46a00 2024-04-02T14:46:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.613+0000 7f445ea00640 1 -- 172.21.15.62:0/842240676 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f44640a1bc0 con 0x55b9d6b46a00 2024-04-02T14:46:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.613+0000 7f448280d740 10 client.26380 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.613+0000 7f448280d740 10 client.26380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.613+0000 7f448280d740 20 client.26380 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.613+0000 7f448280d740 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b9d6bd7220 0x55b9d6bd9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.613+0000 7f448280d740 4 client.26380 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:46:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.613+0000 7f448280d740 1 -- 172.21.15.62:0/842240676 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9d6ab82b0 con 0x55b9d6bd7220 2024-04-02T14:46:58.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.613+0000 7f71a7400640 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x555cd81f4dd0 0x555cd81f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.613+0000 7f447d000640 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b9d6bd7220 0x55b9d6bd9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f447d000640 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b9d6bd7220 0x55b9d6bd9640 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60445a60ce38000 server_cookie=6a9db26d83e551c2 in_seq=0 out_seq=0 2024-04-02T14:46:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f445ea00640 10 client.26380 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f71a7400640 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x555cd81f4dd0 0x555cd81f71f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2dc81b2321bb7360 server_cookie=5785459cff26d6c1 in_seq=0 out_seq=0 2024-04-02T14:46:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f7192000640 10 client.26426 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:46:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f7192000640 1 -- 172.21.15.62:0/1966382842 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cd80d5c30 con 0x555cd81f4dd0 2024-04-02T14:46:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f7192000640 10 client.26426 handle_command_reply: tid=0 2024-04-02T14:46:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f71b1fdf740 1 client.26426 shutdown 2024-04-02T14:46:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1966382842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd81649f0 msgr2=0x555cd817ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd81649f0 0x555cd817ea90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7198005e00 tx=0x7f7198005760 comp rx=0 tx=0).stop 2024-04-02T14:46:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1966382842 shutdown_connections 2024-04-02T14:46:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x555cd81f4dd0 0x555cd81f71f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cd817e090 0x555cd817efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555cd81649f0 0x555cd817ea90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f71b1fdf740 1 --2- 172.21.15.62:0/1966382842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cd8163d10 0x555cd8186330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1966382842 >> 172.21.15.62:0/1966382842 conn(0x555cd812aa50 msgr2=0x555cd815d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.614+0000 7f7192000640 0 client.26426 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:46:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.615+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1966382842 shutdown_connections 2024-04-02T14:46:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.615+0000 7f71b1fdf740 1 -- 172.21.15.62:0/1966382842 wait complete. 2024-04-02T14:46:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.615+0000 7f71b1fdf740 20 client.26426 trim_cache size 0 max 16384 2024-04-02T14:46:58.624 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:46:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.615+0000 7f71af200640 1 -- 172.21.15.62:0/4245152221 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f718c0a3de0 msgr2=0x7f718c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.615+0000 7f71af200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f718c0a3de0 0x7f718c0a62a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71a0006010 tx=0x7f71a0009500 comp rx=0 tx=0).stop 2024-04-02T14:46:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.615+0000 7f71af200640 1 -- 172.21.15.62:0/4245152221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8112940 msgr2=0x7f71a819e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.615+0000 7f71af200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8112940 0x7f71a819e8e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f71a002c500 tx=0x7f71a0033720 comp rx=0 tx=0).stop 2024-04-02T14:46:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.616+0000 7f71af200640 1 -- 172.21.15.62:0/4245152221 shutdown_connections 2024-04-02T14:46:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.616+0000 7f71af200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f718c0a3de0 0x7f718c0a62a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.616+0000 7f71af200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8112940 0x7f71a819e8e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.616+0000 7f71af200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8076950 0x7f71a81a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.616+0000 7f71af200640 1 --2- 172.21.15.62:0/4245152221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a8076030 0x7f71a81a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.616+0000 7f71af200640 1 -- 172.21.15.62:0/4245152221 >> 172.21.15.62:0/4245152221 conn(0x7f71a80ff770 msgr2=0x7f71a8114430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.616+0000 7f71af200640 1 -- 172.21.15.62:0/4245152221 shutdown_connections 2024-04-02T14:46:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:58.616+0000 7f71af200640 1 -- 172.21.15.62:0/4245152221 wait complete. 2024-04-02T14:46:58.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:58 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2832065442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:46:58.636 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:46:58.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:58 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2832065442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:46:58.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:58 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2832065442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:46:59.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.026+0000 7f0b31000640 1 -- 172.21.15.62:0/3477934077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2c076950 msgr2=0x7f0b2c112400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:59.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.026+0000 7f0b31000640 1 --2- 172.21.15.62:0/3477934077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2c076950 0x7f0b2c112400 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009930 tx=0x7f0b1c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:46:59.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.027+0000 7f0b31000640 1 -- 172.21.15.62:0/3477934077 shutdown_connections 2024-04-02T14:46:59.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.027+0000 7f0b31000640 1 --2- 172.21.15.62:0/3477934077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2c112940 0x7f0b2c114d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.027+0000 7f0b31000640 1 --2- 172.21.15.62:0/3477934077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2c076950 0x7f0b2c112400 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.028+0000 7f0b31000640 1 --2- 172.21.15.62:0/3477934077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b2c076030 0x7f0b2c076410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.028+0000 7f0b31000640 1 -- 172.21.15.62:0/3477934077 >> 172.21.15.62:0/3477934077 conn(0x7f0b2c0ff7b0 msgr2=0x7f0b2c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.028+0000 7f0b31000640 1 -- 172.21.15.62:0/3477934077 shutdown_connections 2024-04-02T14:46:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.028+0000 7f0b31000640 1 -- 172.21.15.62:0/3477934077 wait complete. 2024-04-02T14:46:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.029+0000 7f0b31000640 1 Processor -- start 2024-04-02T14:46:59.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.031+0000 7f0b31000640 1 -- start start 2024-04-02T14:46:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.031+0000 7f0b31000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2c076030 0x7f0b2c1a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.031+0000 7f0b31000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2c076950 0x7f0b2c1a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.031+0000 7f0b31000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b2c112940 0x7f0b2c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.031+0000 7f0b31000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1185f0 con 0x7f0b2c076030 2024-04-02T14:46:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.031+0000 7f0b31000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c118470 con 0x7f0b2c112940 2024-04-02T14:46:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.031+0000 7f0b31000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c118770 con 0x7f0b2c076950 2024-04-02T14:46:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.032+0000 7f0b29400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2c076950 0x7f0b2c1a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.032+0000 7f0b29400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2c076950 0x7f0b2c1a4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:36532/0 (socket says 172.21.15.62:36532) 2024-04-02T14:46:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.032+0000 7f0b29400640 1 -- 172.21.15.62:0/2772484214 learned_addr learned my addr 172.21.15.62:0/2772484214 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.032+0000 7f0b29e00640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2c076030 0x7f0b2c1a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.032+0000 7f0b2a800640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b2c112940 0x7f0b2c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.032+0000 7f0b29400640 1 -- 172.21.15.62:0/2772484214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b2c112940 msgr2=0x7f0b2c19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.032+0000 7f0b29400640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b2c112940 0x7f0b2c19e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.032+0000 7f0b29400640 1 -- 172.21.15.62:0/2772484214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2c076030 msgr2=0x7f0b2c1a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.032+0000 7f0b29400640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2c076030 0x7f0b2c1a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.032+0000 7f0b29400640 1 -- 172.21.15.62:0/2772484214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c19f150 con 0x7f0b2c076950 2024-04-02T14:46:59.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.033+0000 7f0b29400640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2c076950 0x7f0b2c1a4d60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000c00 tx=0x7f0b1c02c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:59.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.033+0000 7f0b22a00640 1 -- 172.21.15.62:0/2772484214 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c043020 con 0x7f0b2c076950 2024-04-02T14:46:59.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.033+0000 7f0b22a00640 1 -- 172.21.15.62:0/2772484214 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0b1c0042d0 con 0x7f0b2c076950 2024-04-02T14:46:59.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.034+0000 7f0b31000640 1 -- 172.21.15.62:0/2772484214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c19f3e0 con 0x7f0b2c076950 2024-04-02T14:46:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.034+0000 7f0b22a00640 1 -- 172.21.15.62:0/2772484214 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c003b50 con 0x7f0b2c076950 2024-04-02T14:46:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.034+0000 7f0b31000640 1 -- 172.21.15.62:0/2772484214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c1ab6e0 con 0x7f0b2c076950 2024-04-02T14:46:59.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.042+0000 7f0b22a00640 1 -- 172.21.15.62:0/2772484214 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0b1c046020 con 0x7f0b2c076950 2024-04-02T14:46:59.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.045+0000 7f0b33c36740 1 Processor -- start 2024-04-02T14:46:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.048+0000 7f0b33c36740 1 -- start start 2024-04-02T14:46:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.048+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 0x556033e9b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.049+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 0x556033e9c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.049+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033ea5080 0x556033ea7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.049+0000 7f0b33c36740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556033ddc400 con 0x556033ea5080 2024-04-02T14:46:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.049+0000 7f0b33c36740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556033dc0f10 con 0x556033e9c9f0 2024-04-02T14:46:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.049+0000 7f0b33c36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556033e006f0 con 0x556033e9bd10 2024-04-02T14:46:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.050+0000 7f0b2a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 0x556033e9b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.050+0000 7f0b2a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 0x556033e9b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55836/0 (socket says 172.21.15.62:55836) 2024-04-02T14:46:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.050+0000 7f0b2a800640 1 -- 172.21.15.62:0/1740229597 learned_addr learned my addr 172.21.15.62:0/1740229597 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.050+0000 7f0b29400640 1 --2- 172.21.15.62:0/1740229597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033ea5080 0x556033ea7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.050+0000 7f0b29e00640 1 --2- 172.21.15.62:0/1740229597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 0x556033e9c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.050+0000 7f0b22a00640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0b100a3ea0 0x7f0b100a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.051+0000 7f0b22a00640 1 -- 172.21.15.62:0/2772484214 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f0b1c0f2870 con 0x7f0b2c076950 2024-04-02T14:46:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.052+0000 7f0b29e00640 1 -- 172.21.15.62:0/1740229597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 msgr2=0x556033e9b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.052+0000 7f0b29e00640 1 --2- 172.21.15.62:0/1740229597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 0x556033e9b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.052+0000 7f0b29e00640 1 -- 172.21.15.62:0/1740229597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033ea5080 msgr2=0x556033ea7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.052+0000 7f0b29e00640 1 --2- 172.21.15.62:0/1740229597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033ea5080 0x556033ea7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.052+0000 7f0b29e00640 1 -- 172.21.15.62:0/1740229597 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556033b143f0 con 0x556033e9bd10 2024-04-02T14:46:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.052+0000 7f0b2a800640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0b100a3ea0 0x7f0b100a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.053+0000 7f0b29e00640 1 --2- 172.21.15.62:0/1740229597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 0x556033e9c190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009970 tx=0x7f0b18009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46adef956fe210d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.054+0000 7f0b17400640 1 -- 172.21.15.62:0/1740229597 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18017070 con 0x556033e9bd10 2024-04-02T14:46:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.054+0000 7f0b17400640 1 -- 172.21.15.62:0/1740229597 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0b18013470 con 0x556033e9bd10 2024-04-02T14:46:59.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.054+0000 7f0b17400640 1 -- 172.21.15.62:0/1740229597 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18017070 con 0x556033e9bd10 2024-04-02T14:46:59.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.056+0000 7f0b2a800640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0b100a3ea0 0x7f0b100a6360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0b240135a0 tx=0x7f0b24004ab0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:59.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.056+0000 7f0b33c36740 1 -- 172.21.15.62:0/1740229597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 msgr2=0x556033e9c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:46:59.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.056+0000 7f0b33c36740 1 --2- 172.21.15.62:0/1740229597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 0x556033e9c190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009970 tx=0x7f0b18009e30 comp rx=0 tx=0).stop 2024-04-02T14:46:59.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.057+0000 7f0b33c36740 1 -- 172.21.15.62:0/1740229597 shutdown_connections 2024-04-02T14:46:59.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.057+0000 7f0b33c36740 1 --2- 172.21.15.62:0/1740229597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033ea5080 0x556033ea7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.057+0000 7f0b33c36740 1 --2- 172.21.15.62:0/1740229597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 0x556033e9c190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.057+0000 7f0b33c36740 1 --2- 172.21.15.62:0/1740229597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 0x556033e9b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.057+0000 7f0b33c36740 1 -- 172.21.15.62:0/1740229597 >> 172.21.15.62:0/1740229597 conn(0x556033e62a50 msgr2=0x556033e96760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:46:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.057+0000 7f0b33c36740 1 -- 172.21.15.62:0/1740229597 shutdown_connections 2024-04-02T14:46:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.057+0000 7f0b33c36740 1 -- 172.21.15.62:0/1740229597 wait complete. 2024-04-02T14:46:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.058+0000 7f0b33c36740 1 -- start start 2024-04-02T14:46:59.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.058+0000 7f0b33c36740 1 Processor -- start 2024-04-02T14:46:59.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.060+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 0x556033ea2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.060+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 0x556033ea2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.060+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033ea5080 0x556033ea2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.060+0000 7f0b33c36740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556033dfc6a0 con 0x556033ea5080 2024-04-02T14:46:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.060+0000 7f0b33c36740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556033da8980 con 0x556033e9c9f0 2024-04-02T14:46:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.060+0000 7f0b33c36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556033dbbcb0 con 0x556033e9bd10 2024-04-02T14:46:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.060+0000 7f0b33c36740 10 client.0 _ll_register_callbacks cb 0x556033dbb6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:46:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.061+0000 7f0b29400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 0x556033ea2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.061+0000 7f0b29e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 0x556033ea2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.061+0000 7f0b29400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 0x556033ea2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55848/0 (socket says 172.21.15.62:55848) 2024-04-02T14:46:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.061+0000 7f0b29400640 1 -- 172.21.15.62:0/1768038715 learned_addr learned my addr 172.21.15.62:0/1768038715 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:46:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.061+0000 7f0b2a800640 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033ea5080 0x556033ea2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.063+0000 7f0b29400640 1 -- 172.21.15.62:0/1768038715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 msgr2=0x556033ea2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.063+0000 7f0b29400640 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 0x556033ea2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.063+0000 7f0b29400640 1 -- 172.21.15.62:0/1768038715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033ea5080 msgr2=0x556033ea2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:46:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.063+0000 7f0b29400640 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033ea5080 0x556033ea2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:46:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.063+0000 7f0b29400640 1 -- 172.21.15.62:0/1768038715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556033bde7d0 con 0x556033e9c9f0 2024-04-02T14:46:59.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.064+0000 7f0b29e00640 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 0x556033ea2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:46:59.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.064+0000 7f0b29400640 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 0x556033ea2590 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0f0180 tx=0x7f0b1c0be4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1467bc22fb7c54a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:46:59.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.064+0000 7f0b0be00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:46:59.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.064+0000 7f0b0be00640 1 -- 172.21.15.62:0/1768038715 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c0bf070 con 0x556033e9c9f0 2024-04-02T14:46:59.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.064+0000 7f0b33c36740 1 -- 172.21.15.62:0/1768038715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b18013470 con 0x556033e9c9f0 2024-04-02T14:46:59.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.064+0000 7f0b0be00640 1 -- 172.21.15.62:0/1768038715 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f0b1c005470 con 0x556033e9c9f0 2024-04-02T14:46:59.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.065+0000 7f0b0be00640 1 -- 172.21.15.62:0/1768038715 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0b1c005670 con 0x556033e9c9f0 2024-04-02T14:46:59.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.065+0000 7f0b33c36740 10 client.26438 fetch_fsmap learned FSMap version 36 2024-04-02T14:46:59.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.065+0000 7f0b33c36740 1 -- 172.21.15.62:0/1768038715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556033eb3d20 con 0x556033e9c9f0 2024-04-02T14:46:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.066+0000 7f0b0be00640 1 -- 172.21.15.62:0/1768038715 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0b18013470 con 0x556033e9c9f0 2024-04-02T14:46:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.068+0000 7f0b0be00640 1 -- 172.21.15.62:0/1768038715 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f0b1c0bec60 con 0x556033e9c9f0 2024-04-02T14:46:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.068+0000 7f0b33c36740 10 client.26438 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:46:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.068+0000 7f0b33c36740 10 client.26438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:46:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.068+0000 7f0b33c36740 20 client.26438 populate_metadata read hostname 'smithi062' 2024-04-02T14:46:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.068+0000 7f0b33c36740 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x556033eb51b0 0x556033eb7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:46:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.068+0000 7f0b33c36740 4 client.26438 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:46:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.068+0000 7f0b33c36740 1 -- 172.21.15.62:0/1768038715 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556033dc0040 con 0x556033eb51b0 2024-04-02T14:46:59.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.069+0000 7f0b29e00640 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x556033eb51b0 0x556033eb7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:46:59.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.069+0000 7f0b0be00640 10 client.26438 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:46:59.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:46:59.069+0000 7f0b29e00640 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x556033eb51b0 0x556033eb7610 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a580e1794aa4ceee server_cookie=3434db51a382d2e8 in_seq=0 out_seq=0 2024-04-02T14:46:59.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:46:59 smithi204 ceph-mon[35948]: pgmap v92: 97 pgs: 97 active+clean; 139 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 992 KiB/s wr, 177 op/s 2024-04-02T14:46:59.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:46:59 smithi062 ceph-mon[32116]: pgmap v92: 97 pgs: 97 active+clean; 139 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 992 KiB/s wr, 177 op/s 2024-04-02T14:46:59.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:46:59 smithi113 ceph-mon[36135]: pgmap v92: 97 pgs: 97 active+clean; 139 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 992 KiB/s wr, 177 op/s 2024-04-02T14:47:01.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:01 smithi204 ceph-mon[35948]: pgmap v93: 97 pgs: 97 active+clean; 139 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 989 KiB/s wr, 130 op/s 2024-04-02T14:47:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.784+0000 7f445ea00640 1 -- 172.21.15.62:0/842240676 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9d6ab82b0 con 0x55b9d6bd7220 2024-04-02T14:47:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.784+0000 7f445ea00640 10 client.26380 handle_command_reply: tid=0 2024-04-02T14:47:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.784+0000 7f448280d740 1 client.26380 shutdown 2024-04-02T14:47:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.784+0000 7f0b0be00640 1 -- 172.21.15.62:0/1768038715 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556033dc0040 con 0x556033eb51b0 2024-04-02T14:47:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.784+0000 7f0b0be00640 10 client.26438 handle_command_reply: tid=0 2024-04-02T14:47:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.784+0000 7f448280d740 1 -- 172.21.15.62:0/842240676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b46a00 msgr2=0x55b9d6b68d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.784+0000 7f448280d740 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b46a00 0x55b9d6b68d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4464000c00 tx=0x7f4464098dc0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.784+0000 7f0b33c36740 1 client.26438 shutdown 2024-04-02T14:47:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f448280d740 1 -- 172.21.15.62:0/842240676 shutdown_connections 2024-04-02T14:47:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f448280d740 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55b9d6bd7220 0x55b9d6bd9640 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f448280d740 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b9d6b4f5e0 0x55b9d6b63f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f448280d740 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9d6b46a00 0x55b9d6b68d10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f448280d740 1 --2- 172.21.15.62:0/842240676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9d6b460e0 0x55b9d6b687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f448280d740 1 -- 172.21.15.62:0/842240676 >> 172.21.15.62:0/842240676 conn(0x55b9d6b40810 msgr2=0x55b9d6b41810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f445ea00640 0 client.26380 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f0b33c36740 1 -- 172.21.15.62:0/1768038715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 msgr2=0x556033ea2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f0b33c36740 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 0x556033ea2590 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0f0180 tx=0x7f0b1c0be4b0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f0b33c36740 1 -- 172.21.15.62:0/1768038715 shutdown_connections 2024-04-02T14:47:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f0b33c36740 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x556033eb51b0 0x556033eb7610 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f0b33c36740 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033ea5080 0x556033ea2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f0b33c36740 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9c9f0 0x556033ea2590 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f0b33c36740 1 --2- 172.21.15.62:0/1768038715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033e9bd10 0x556033ea2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.785+0000 7f0b33c36740 1 -- 172.21.15.62:0/1768038715 >> 172.21.15.62:0/1768038715 conn(0x556033e62a50 msgr2=0x556033e984b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.786+0000 7f0b0be00640 0 client.26438 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.786+0000 7f448280d740 1 -- 172.21.15.62:0/842240676 shutdown_connections 2024-04-02T14:47:01.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.786+0000 7f0b33c36740 1 -- 172.21.15.62:0/1768038715 shutdown_connections 2024-04-02T14:47:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.786+0000 7f448280d740 1 -- 172.21.15.62:0/842240676 wait complete. 2024-04-02T14:47:01.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.786+0000 7f0b33c36740 1 -- 172.21.15.62:0/1768038715 wait complete. 2024-04-02T14:47:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.786+0000 7f448280d740 20 client.26380 trim_cache size 0 max 16384 2024-04-02T14:47:01.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.786+0000 7f0b33c36740 20 client.26438 trim_cache size 0 max 16384 2024-04-02T14:47:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:47:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:47:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:47:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:47:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.787+0000 7f447e400640 1 -- 172.21.15.62:0/2642595505 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44600a3d90 msgr2=0x7f44600a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.787+0000 7f447e400640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44600a3d90 0x7f44600a6250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55b9d6b45bb0 tx=0x7f446c005d50 comp rx=0 tx=0).stop 2024-04-02T14:47:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.787+0000 7f447e400640 1 -- 172.21.15.62:0/2642595505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44781033d0 msgr2=0x7f44781987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.787+0000 7f447e400640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44781033d0 0x7f44781987d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f446400d770 tx=0x7f446400dc30 comp rx=0 tx=0).stop 2024-04-02T14:47:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.788+0000 7f447e400640 1 -- 172.21.15.62:0/2642595505 shutdown_connections 2024-04-02T14:47:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.788+0000 7f447e400640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f44600a3d90 0x7f44600a6250 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.788+0000 7f447e400640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f447810bed0 0x7f447819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.788+0000 7f447e400640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44781033d0 0x7f44781987d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.788+0000 7f447e400640 1 --2- 172.21.15.62:0/2642595505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4478102ab0 0x7f4478198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.788+0000 7f447e400640 1 -- 172.21.15.62:0/2642595505 >> 172.21.15.62:0/2642595505 conn(0x7f44780fc310 msgr2=0x7f44780fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.788+0000 7f447e400640 1 -- 172.21.15.62:0/2642595505 shutdown_connections 2024-04-02T14:47:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:01.788+0000 7f447e400640 1 -- 172.21.15.62:0/2642595505 wait complete. 2024-04-02T14:47:01.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.807+0000 7f0b33c36740 1 Processor -- start 2024-04-02T14:47:01.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.807+0000 7f0b33c36740 1 -- start start 2024-04-02T14:47:01.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.808+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:01.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.808+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033e9c9f0 0x556033eb0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:01.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.808+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033ea5080 0x556033ebcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:01.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.808+0000 7f0b33c36740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1b3da0 con 0x556033e9c9f0 2024-04-02T14:47:01.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.808+0000 7f0b33c36740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1b5a90 con 0x556033e9bd10 2024-04-02T14:47:01.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.808+0000 7f0b33c36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1b44d0 con 0x556033ea5080 2024-04-02T14:47:01.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.808+0000 7f0b29e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.808+0000 7f0b29e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55854/0 (socket says 172.21.15.62:55854) 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.808+0000 7f0b29e00640 1 -- 172.21.15.62:0/2459431857 learned_addr learned my addr 172.21.15.62:0/2459431857 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.808+0000 7f0b29400640 1 --2- 172.21.15.62:0/2459431857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033e9c9f0 0x556033eb0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.808+0000 7f0b2a800640 1 --2- 172.21.15.62:0/2459431857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033ea5080 0x556033ebcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.809+0000 7f0b29e00640 1 -- 172.21.15.62:0/2459431857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033ea5080 msgr2=0x556033ebcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.809+0000 7f0b29e00640 1 --2- 172.21.15.62:0/2459431857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033ea5080 0x556033ebcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.809+0000 7f0b29e00640 1 -- 172.21.15.62:0/2459431857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033e9c9f0 msgr2=0x556033eb0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.809+0000 7f0b29e00640 1 --2- 172.21.15.62:0/2459431857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033e9c9f0 0x556033eb0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.809+0000 7f0b29e00640 1 -- 172.21.15.62:0/2459431857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556033e9ad40 con 0x556033e9bd10 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.809+0000 7f0b29e00640 1 --2- 172.21.15.62:0/2459431857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea4780 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0b18013fd0 tx=0x7f0b18015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5dfa9e36698fbf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.809+0000 7f0b16a00640 1 -- 172.21.15.62:0/2459431857 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18017040 con 0x556033e9bd10 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.809+0000 7f0b16a00640 1 -- 172.21.15.62:0/2459431857 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0b18013470 con 0x556033e9bd10 2024-04-02T14:47:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.809+0000 7f0b16a00640 1 -- 172.21.15.62:0/2459431857 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1801c690 con 0x556033e9bd10 2024-04-02T14:47:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.810+0000 7f0b33c36740 1 -- 172.21.15.62:0/2459431857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 msgr2=0x556033ea4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.810+0000 7f0b33c36740 1 --2- 172.21.15.62:0/2459431857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea4780 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0b18013fd0 tx=0x7f0b18015040 comp rx=0 tx=0).stop 2024-04-02T14:47:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.810+0000 7f0b33c36740 1 -- 172.21.15.62:0/2459431857 shutdown_connections 2024-04-02T14:47:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.810+0000 7f0b33c36740 1 --2- 172.21.15.62:0/2459431857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033ea5080 0x556033ebcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.810+0000 7f0b33c36740 1 --2- 172.21.15.62:0/2459431857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033e9c9f0 0x556033eb0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.810+0000 7f0b33c36740 1 --2- 172.21.15.62:0/2459431857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea4780 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.810+0000 7f0b33c36740 1 -- 172.21.15.62:0/2459431857 >> 172.21.15.62:0/2459431857 conn(0x556033e62a50 msgr2=0x556033e96290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.810+0000 7f0b33c36740 1 -- 172.21.15.62:0/2459431857 shutdown_connections 2024-04-02T14:47:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.810+0000 7f0b33c36740 1 -- 172.21.15.62:0/2459431857 wait complete. 2024-04-02T14:47:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.811+0000 7f0b33c36740 1 -- start start 2024-04-02T14:47:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.811+0000 7f0b33c36740 1 Processor -- start 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.811+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.811+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033e9c9f0 0x556033ea5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.811+0000 7f0b33c36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033ea5080 0x556033eac030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.811+0000 7f0b33c36740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1b5300 con 0x556033e9c9f0 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.811+0000 7f0b33c36740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1b4bd0 con 0x556033e9bd10 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.811+0000 7f0b33c36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1b1d80 con 0x556033ea5080 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.811+0000 7f0b33c36740 10 client.0 _ll_register_callbacks cb 0x7f0b1c009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b29e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b29e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55862/0 (socket says 172.21.15.62:55862) 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b29e00640 1 -- 172.21.15.62:0/2607052119 learned_addr learned my addr 172.21.15.62:0/2607052119 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b29400640 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033e9c9f0 0x556033ea5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b2a800640 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033ea5080 0x556033eac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b29e00640 1 -- 172.21.15.62:0/2607052119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033ea5080 msgr2=0x556033eac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b29e00640 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033ea5080 0x556033eac030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b29e00640 1 -- 172.21.15.62:0/2607052119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033e9c9f0 msgr2=0x556033ea5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b29e00640 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033e9c9f0 0x556033ea5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b29e00640 1 -- 172.21.15.62:0/2607052119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556033eac570 con 0x556033e9bd10 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b29400640 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033e9c9f0 0x556033ea5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b29e00640 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea5a00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009970 tx=0x7f0b18005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7127419f4bef52eb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.812+0000 7f0b0be00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b0be00640 1 -- 172.21.15.62:0/2607052119 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18038060 con 0x556033e9bd10 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b0be00640 1 -- 172.21.15.62:0/2607052119 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f0b18013820 con 0x556033e9bd10 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b33c36740 1 -- 172.21.15.62:0/2607052119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b18013470 con 0x556033e9bd10 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b0be00640 1 -- 172.21.15.62:0/2607052119 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0b18017040 con 0x556033e9bd10 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b33c36740 10 client.26444 fetch_fsmap learned FSMap version 36 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b33c36740 1 -- 172.21.15.62:0/2607052119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556033ead7a0 con 0x556033e9bd10 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b0be00640 1 -- 172.21.15.62:0/2607052119 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0b18013470 con 0x556033e9bd10 2024-04-02T14:47:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b0be00640 1 -- 172.21.15.62:0/2607052119 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f0b18006610 con 0x556033e9bd10 2024-04-02T14:47:01.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b33c36740 10 client.26444 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:47:01.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b33c36740 10 client.26444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:47:01.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b33c36740 20 client.26444 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:01.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.813+0000 7f0b33c36740 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x556033f1b8d0 0x556033f1dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:01.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.814+0000 7f0b33c36740 4 client.26444 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:47:01.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.814+0000 7f0b33c36740 1 -- 172.21.15.62:0/2607052119 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556033e0dc30 con 0x556033f1b8d0 2024-04-02T14:47:01.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.814+0000 7f0b29400640 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x556033f1b8d0 0x556033f1dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:01.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.814+0000 7f0b29400640 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x556033f1b8d0 0x556033f1dcf0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee791242635b1579 server_cookie=722ec42eefea7ff4 in_seq=0 out_seq=0 2024-04-02T14:47:01.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.814+0000 7f0b0be00640 10 client.26444 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:01.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.815+0000 7f0b0be00640 1 -- 172.21.15.62:0/2607052119 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556033e0dc30 con 0x556033f1b8d0 2024-04-02T14:47:01.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.815+0000 7f0b0be00640 10 client.26444 handle_command_reply: tid=0 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.815+0000 7f0b33c36740 1 client.26444 shutdown 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b33c36740 1 -- 172.21.15.62:0/2607052119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 msgr2=0x556033ea5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b33c36740 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea5a00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009970 tx=0x7f0b18005bf0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b33c36740 1 -- 172.21.15.62:0/2607052119 shutdown_connections 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b33c36740 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x556033f1b8d0 0x556033f1dcf0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b33c36740 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556033ea5080 0x556033eac030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b33c36740 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556033e9c9f0 0x556033ea5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b33c36740 1 --2- 172.21.15.62:0/2607052119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556033e9bd10 0x556033ea5a00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b33c36740 1 -- 172.21.15.62:0/2607052119 >> 172.21.15.62:0/2607052119 conn(0x556033e62a50 msgr2=0x556033e95010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b0be00640 0 client.26444 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b33c36740 1 -- 172.21.15.62:0/2607052119 shutdown_connections 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b33c36740 1 -- 172.21.15.62:0/2607052119 wait complete. 2024-04-02T14:47:01.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.816+0000 7f0b33c36740 20 client.26444 trim_cache size 0 max 16384 2024-04-02T14:47:01.814 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 -- 172.21.15.62:0/2772484214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0b100a3ea0 msgr2=0x7f0b100a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0b100a3ea0 0x7f0b100a6360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0b240135a0 tx=0x7f0b24004ab0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 -- 172.21.15.62:0/2772484214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2c076950 msgr2=0x7f0b2c1a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2c076950 0x7f0b2c1a4d60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000c00 tx=0x7f0b1c02c920 comp rx=0 tx=0).stop 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 -- 172.21.15.62:0/2772484214 shutdown_connections 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0b100a3ea0 0x7f0b100a6360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b2c112940 0x7f0b2c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2c076950 0x7f0b2c1a4d60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 --2- 172.21.15.62:0/2772484214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2c076030 0x7f0b2c1a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 -- 172.21.15.62:0/2772484214 >> 172.21.15.62:0/2772484214 conn(0x7f0b2c0ff7b0 msgr2=0x7f0b2c101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 -- 172.21.15.62:0/2772484214 shutdown_connections 2024-04-02T14:47:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:01.817+0000 7f0b31000640 1 -- 172.21.15.62:0/2772484214 wait complete. 2024-04-02T14:47:01.826 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:01 smithi062 ceph-mon[32116]: pgmap v93: 97 pgs: 97 active+clean; 139 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 989 KiB/s wr, 130 op/s 2024-04-02T14:47:01.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:01 smithi113 ceph-mon[36135]: pgmap v93: 97 pgs: 97 active+clean; 139 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 989 KiB/s wr, 130 op/s 2024-04-02T14:47:02.103 INFO:tasks.workunit.client.0.smithi062.stderr:+ sync 2024-04-02T14:47:02.829 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:47:02.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:47:02.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:47:02.973 INFO:tasks.workunit.client.0.smithi062.stderr:+ tar c linux-4.0.5 2024-04-02T14:47:03.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.083+0000 7f9ab7200640 1 -- 172.21.15.62:0/4082160343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab8073ed0 msgr2=0x7f9ab80742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.083+0000 7f9ab7200640 1 --2- 172.21.15.62:0/4082160343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab8073ed0 0x7f9ab80742b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009900 tx=0x7f9aa802bcd0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.083+0000 7f9ab7200640 1 -- 172.21.15.62:0/4082160343 shutdown_connections 2024-04-02T14:47:03.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.083+0000 7f9ab7200640 1 --2- 172.21.15.62:0/4082160343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ab810e210 0x7f9ab8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.083+0000 7f9ab7200640 1 --2- 172.21.15.62:0/4082160343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80747f0 0x7f9ab810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.083+0000 7f9ab7200640 1 --2- 172.21.15.62:0/4082160343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab8073ed0 0x7f9ab80742b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.083+0000 7f9ab7200640 1 -- 172.21.15.62:0/4082160343 >> 172.21.15.62:0/4082160343 conn(0x7f9ab80fd4a0 msgr2=0x7f9ab80ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.083+0000 7f9ab7200640 1 -- 172.21.15.62:0/4082160343 shutdown_connections 2024-04-02T14:47:03.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.083+0000 7f9ab7200640 1 -- 172.21.15.62:0/4082160343 wait complete. 2024-04-02T14:47:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.084+0000 7f9ab7200640 1 Processor -- start 2024-04-02T14:47:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.084+0000 7f9ab7200640 1 -- start start 2024-04-02T14:47:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.085+0000 7f9ab7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab8073ed0 0x7f9ab819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.085+0000 7f9ab7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab80747f0 0x7f9ab819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.085+0000 7f9ab7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ab810e210 0x7f9ab819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.085+0000 7f9ab7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab8114120 con 0x7f9ab80747f0 2024-04-02T14:47:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.085+0000 7f9ab7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab8113fa0 con 0x7f9ab8073ed0 2024-04-02T14:47:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.085+0000 7f9ab7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81142a0 con 0x7f9ab810e210 2024-04-02T14:47:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.085+0000 7f9ab5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab8073ed0 0x7f9ab819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.085+0000 7f9ab5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab8073ed0 0x7f9ab819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:39728/0 (socket says 172.21.15.62:39728) 2024-04-02T14:47:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.085+0000 7f9ab5e00640 1 -- 172.21.15.62:0/2054628179 learned_addr learned my addr 172.21.15.62:0/2054628179 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:03.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.085+0000 7f9ab6800640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ab810e210 0x7f9ab819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.085+0000 7f9ab5400640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab80747f0 0x7f9ab819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.086+0000 7f9ab5e00640 1 -- 172.21.15.62:0/2054628179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ab810e210 msgr2=0x7f9ab819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.086+0000 7f9ab5e00640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ab810e210 0x7f9ab819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.086+0000 7f9ab5e00640 1 -- 172.21.15.62:0/2054628179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab80747f0 msgr2=0x7f9ab819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.086+0000 7f9ab5e00640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab80747f0 0x7f9ab819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.086+0000 7f9ab5e00640 1 -- 172.21.15.62:0/2054628179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab819f580 con 0x7f9ab8073ed0 2024-04-02T14:47:03.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.086+0000 7f9ab5e00640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab8073ed0 0x7f9ab819a5e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8000c00 tx=0x7f9aa8004a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.086+0000 7f9aaea00640 1 -- 172.21.15.62:0/2054628179 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8043020 con 0x7f9ab8073ed0 2024-04-02T14:47:03.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.086+0000 7f9aaea00640 1 -- 172.21.15.62:0/2054628179 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9aa8004310 con 0x7f9ab8073ed0 2024-04-02T14:47:03.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.086+0000 7f9aaea00640 1 -- 172.21.15.62:0/2054628179 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8003bc0 con 0x7f9ab8073ed0 2024-04-02T14:47:03.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.086+0000 7f9ab7200640 1 -- 172.21.15.62:0/2054628179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab819f810 con 0x7f9ab8073ed0 2024-04-02T14:47:03.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.086+0000 7f9ab7200640 1 -- 172.21.15.62:0/2054628179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab81a7100 con 0x7f9ab8073ed0 2024-04-02T14:47:03.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9abfba6740 1 Processor -- start 2024-04-02T14:47:03.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9aaea00640 1 -- 172.21.15.62:0/2054628179 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9aa8046020 con 0x7f9ab8073ed0 2024-04-02T14:47:03.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9abfba6740 1 -- start start 2024-04-02T14:47:03.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfc1f0 0x5642f2cfaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfb510 0x5642f2cfb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d06c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9abfba6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5642f2c5f310 con 0x5642f2d04880 2024-04-02T14:47:03.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9abfba6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5642f2c07690 con 0x5642f2cfc1f0 2024-04-02T14:47:03.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9abfba6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5642f2c5b2c0 con 0x5642f2cfb510 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9ab6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfc1f0 0x5642f2cfaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9ab6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfc1f0 0x5642f2cfaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:39742/0 (socket says 172.21.15.62:39742) 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.090+0000 7f9ab6800640 1 -- 172.21.15.62:0/4287963096 learned_addr learned my addr 172.21.15.62:0/4287963096 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9ab5400640 1 --2- 172.21.15.62:0/4287963096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d06c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9ab5e00640 1 --2- 172.21.15.62:0/4287963096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfb510 0x5642f2cfb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9ab6800640 1 -- 172.21.15.62:0/4287963096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfb510 msgr2=0x5642f2cfb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9ab6800640 1 --2- 172.21.15.62:0/4287963096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfb510 0x5642f2cfb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9ab6800640 1 -- 172.21.15.62:0/4287963096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 msgr2=0x5642f2d06c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9ab6800640 1 --2- 172.21.15.62:0/4287963096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d06c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9ab6800640 1 -- 172.21.15.62:0/4287963096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642f2a3b7e0 con 0x5642f2cfc1f0 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9ab5e00640 1 --2- 172.21.15.62:0/4287963096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfb510 0x5642f2cfb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9ab5400640 1 --2- 172.21.15.62:0/4287963096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d06c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9ab6800640 1 --2- 172.21.15.62:0/4287963096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfc1f0 0x5642f2cfaf40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009f10 tx=0x7f9ab0010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3797f70ed59237c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9aaea00640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9a9c0a3d20 0x7f9a9c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.091+0000 7f9aa3400640 1 -- 172.21.15.62:0/4287963096 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0016070 con 0x5642f2cfc1f0 2024-04-02T14:47:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.092+0000 7f9aa3400640 1 -- 172.21.15.62:0/4287963096 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9ab001bb60 con 0x5642f2cfc1f0 2024-04-02T14:47:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.092+0000 7f9aa3400640 1 -- 172.21.15.62:0/4287963096 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0012d60 con 0x5642f2cfc1f0 2024-04-02T14:47:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.092+0000 7f9aaea00640 1 -- 172.21.15.62:0/2054628179 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f9aa80f2650 con 0x7f9ab8073ed0 2024-04-02T14:47:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.092+0000 7f9ab5e00640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9a9c0a3d20 0x7f9a9c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.092+0000 7f9abfba6740 1 -- 172.21.15.62:0/4287963096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfc1f0 msgr2=0x5642f2cfaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.092+0000 7f9abfba6740 1 --2- 172.21.15.62:0/4287963096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfc1f0 0x5642f2cfaf40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009f10 tx=0x7f9ab0010740 comp rx=0 tx=0).stop 2024-04-02T14:47:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.092+0000 7f9abfba6740 1 -- 172.21.15.62:0/4287963096 shutdown_connections 2024-04-02T14:47:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.092+0000 7f9abfba6740 1 --2- 172.21.15.62:0/4287963096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d06c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.092+0000 7f9abfba6740 1 --2- 172.21.15.62:0/4287963096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfb510 0x5642f2cfb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.092+0000 7f9abfba6740 1 --2- 172.21.15.62:0/4287963096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfc1f0 0x5642f2cfaf40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.092+0000 7f9abfba6740 1 -- 172.21.15.62:0/4287963096 >> 172.21.15.62:0/4287963096 conn(0x5642f2c8f270 msgr2=0x5642f2cf5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.093+0000 7f9ab5e00640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9a9c0a3d20 0x7f9a9c0a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80effb0 tx=0x7f9aa8009260 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.093+0000 7f9abfba6740 1 -- 172.21.15.62:0/4287963096 shutdown_connections 2024-04-02T14:47:03.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.093+0000 7f9abfba6740 1 -- 172.21.15.62:0/4287963096 wait complete. 2024-04-02T14:47:03.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.093+0000 7f9abfba6740 1 -- start start 2024-04-02T14:47:03.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.093+0000 7f9abfba6740 1 Processor -- start 2024-04-02T14:47:03.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 0x5642f2d018b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d01df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d02330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9abfba6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5642f2d076f0 con 0x5642f2d04880 2024-04-02T14:47:03.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9abfba6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5642f2c1aa60 con 0x5642f2cfb510 2024-04-02T14:47:03.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9abfba6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5642f2d07870 con 0x5642f2cfc1f0 2024-04-02T14:47:03.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9abfba6740 10 client.0 _ll_register_callbacks cb 0x5642f2c20030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:03.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9ab5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d01df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9ab5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d01df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:56006/0 (socket says 172.21.15.62:56006) 2024-04-02T14:47:03.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9ab5400640 1 -- 172.21.15.62:0/1484063476 learned_addr learned my addr 172.21.15.62:0/1484063476 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9ab6800640 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 0x5642f2d018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.094+0000 7f9ab6800640 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d02330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.095+0000 7f9ab5400640 1 -- 172.21.15.62:0/1484063476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 msgr2=0x5642f2d018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.095+0000 7f9ab5400640 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 0x5642f2d018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.095+0000 7f9ab5400640 1 -- 172.21.15.62:0/1484063476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 msgr2=0x5642f2d02330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.095+0000 7f9ab5400640 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d02330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.095+0000 7f9ab5400640 1 -- 172.21.15.62:0/1484063476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642f2a9bf70 con 0x5642f2cfc1f0 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.095+0000 7f9ab6800640 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d02330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.095+0000 7f9ab5400640 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d01df0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4012700 tx=0x7f9aa4012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9712aed63b1b2be server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.095+0000 7f9a97e00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.095+0000 7f9a97e00640 1 -- 172.21.15.62:0/1484063476 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400f9a0 con 0x5642f2cfc1f0 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.095+0000 7f9a97e00640 1 -- 172.21.15.62:0/1484063476 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f9aa400fb40 con 0x5642f2cfc1f0 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.096+0000 7f9abfba6740 1 -- 172.21.15.62:0/1484063476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab001bb60 con 0x5642f2cfc1f0 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.096+0000 7f9a97e00640 1 -- 172.21.15.62:0/1484063476 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9aa400fd40 con 0x5642f2cfc1f0 2024-04-02T14:47:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.096+0000 7f9abfba6740 10 client.26410 fetch_fsmap learned FSMap version 36 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.096+0000 7f9abfba6740 1 -- 172.21.15.62:0/1484063476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5642f2933170 con 0x5642f2cfc1f0 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.096+0000 7f9a97e00640 1 -- 172.21.15.62:0/1484063476 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9ab001bb60 con 0x5642f2cfc1f0 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.096+0000 7f9a97e00640 1 -- 172.21.15.62:0/1484063476 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f9aa401c4b0 con 0x5642f2cfc1f0 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.097+0000 7f9abfba6740 10 client.26410 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.097+0000 7f9abfba6740 10 client.26410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.097+0000 7f9abfba6740 20 client.26410 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.097+0000 7f9abfba6740 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5642f2d14e00 0x5642f2d17220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.097+0000 7f9abfba6740 4 client.26410 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.097+0000 7f9abfba6740 1 -- 172.21.15.62:0/1484063476 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642f2c6c850 con 0x5642f2d14e00 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.097+0000 7f9ab6800640 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5642f2d14e00 0x5642f2d17220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.098+0000 7f9ab6800640 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5642f2d14e00 0x5642f2d17220 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0247d2b96c8902 server_cookie=5a4db574de52b60c in_seq=0 out_seq=0 2024-04-02T14:47:03.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.098+0000 7f9a97e00640 10 client.26410 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:03.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9a97e00640 1 -- 172.21.15.62:0/1484063476 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642f2c6c850 con 0x5642f2d14e00 2024-04-02T14:47:03.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9a97e00640 10 client.26410 handle_command_reply: tid=0 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9abfba6740 1 client.26410 shutdown 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9abfba6740 1 -- 172.21.15.62:0/1484063476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 msgr2=0x5642f2d01df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9abfba6740 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d01df0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4012700 tx=0x7f9aa4012bc0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9abfba6740 1 -- 172.21.15.62:0/1484063476 shutdown_connections 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9abfba6740 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5642f2d14e00 0x5642f2d17220 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9abfba6740 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d02330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9abfba6740 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d01df0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9abfba6740 1 --2- 172.21.15.62:0/1484063476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 0x5642f2d018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9a97e00640 0 client.26410 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.099+0000 7f9abfba6740 1 -- 172.21.15.62:0/1484063476 >> 172.21.15.62:0/1484063476 conn(0x5642f2c8f270 msgr2=0x5642f2cf7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.100+0000 7f9abfba6740 1 -- 172.21.15.62:0/1484063476 shutdown_connections 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.100+0000 7f9abfba6740 1 -- 172.21.15.62:0/1484063476 wait complete. 2024-04-02T14:47:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.100+0000 7f9abfba6740 20 client.26410 trim_cache size 0 max 16384 2024-04-02T14:47:03.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.121+0000 7f9abfba6740 1 Processor -- start 2024-04-02T14:47:03.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.121+0000 7f9abfba6740 1 -- start start 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2cfb510 0x5642f2d105f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d02150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2d04880 0x5642f2d1caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9abfba6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81af8b0 con 0x5642f2cfb510 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9abfba6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81b15a0 con 0x5642f2d04880 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9abfba6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81affe0 con 0x5642f2cfc1f0 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9ab5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d02150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9ab5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d02150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:56022/0 (socket says 172.21.15.62:56022) 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9ab5400640 1 -- 172.21.15.62:0/3498685559 learned_addr learned my addr 172.21.15.62:0/3498685559 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9ab6800640 1 --2- 172.21.15.62:0/3498685559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2cfb510 0x5642f2d105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9ab5400640 1 -- 172.21.15.62:0/3498685559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2d04880 msgr2=0x5642f2d1caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.122+0000 7f9ab6800640 1 --2- 172.21.15.62:0/3498685559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2d04880 0x5642f2d1caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.123+0000 7f9ab5400640 1 --2- 172.21.15.62:0/3498685559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2d04880 0x5642f2d1caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.123+0000 7f9ab5400640 1 -- 172.21.15.62:0/3498685559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2cfb510 msgr2=0x5642f2d105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.123+0000 7f9ab5400640 1 --2- 172.21.15.62:0/3498685559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2cfb510 0x5642f2d105f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.123+0000 7f9ab5400640 1 -- 172.21.15.62:0/3498685559 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642f2cf0190 con 0x5642f2cfc1f0 2024-04-02T14:47:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.123+0000 7f9ab6800640 1 --2- 172.21.15.62:0/3498685559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2d04880 0x5642f2d1caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:47:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.123+0000 7f9ab5400640 1 --2- 172.21.15.62:0/3498685559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d02150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4000c00 tx=0x7f9aa400fc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7cbbd8682e3f9cf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.123+0000 7f9aa2a00640 1 -- 172.21.15.62:0/3498685559 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4029020 con 0x5642f2cfc1f0 2024-04-02T14:47:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.123+0000 7f9aa2a00640 1 -- 172.21.15.62:0/3498685559 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9aa40055b0 con 0x5642f2cfc1f0 2024-04-02T14:47:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.123+0000 7f9aa2a00640 1 -- 172.21.15.62:0/3498685559 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40035f0 con 0x5642f2cfc1f0 2024-04-02T14:47:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.124+0000 7f9abfba6740 1 -- 172.21.15.62:0/3498685559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 msgr2=0x5642f2d02150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.124+0000 7f9abfba6740 1 --2- 172.21.15.62:0/3498685559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d02150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4000c00 tx=0x7f9aa400fc60 comp rx=0 tx=0).stop 2024-04-02T14:47:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.124+0000 7f9abfba6740 1 -- 172.21.15.62:0/3498685559 shutdown_connections 2024-04-02T14:47:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.124+0000 7f9abfba6740 1 --2- 172.21.15.62:0/3498685559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2d04880 0x5642f2d1caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.124+0000 7f9abfba6740 1 --2- 172.21.15.62:0/3498685559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d02150 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.124+0000 7f9abfba6740 1 --2- 172.21.15.62:0/3498685559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2cfb510 0x5642f2d105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.124+0000 7f9abfba6740 1 -- 172.21.15.62:0/3498685559 >> 172.21.15.62:0/3498685559 conn(0x5642f2c8f270 msgr2=0x5642f2cf4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.124+0000 7f9abfba6740 1 -- 172.21.15.62:0/3498685559 shutdown_connections 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.124+0000 7f9abfba6740 1 -- 172.21.15.62:0/3498685559 wait complete. 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.125+0000 7f9abfba6740 1 -- start start 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.125+0000 7f9abfba6740 1 Processor -- start 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.125+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 0x5642f2d0d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.125+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d07ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.125+0000 7f9abfba6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d08200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.125+0000 7f9abfba6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81b0e10 con 0x5642f2d04880 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.125+0000 7f9abfba6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81b06e0 con 0x5642f2cfb510 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.125+0000 7f9abfba6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81ad890 con 0x5642f2cfc1f0 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9abfba6740 10 client.0 _ll_register_callbacks cb 0x7f9aa40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9ab6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 0x5642f2d0d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9ab6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d08200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9ab6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 0x5642f2d0d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:39774/0 (socket says 172.21.15.62:39774) 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9ab6800640 1 -- 172.21.15.62:0/3770221926 learned_addr learned my addr 172.21.15.62:0/3770221926 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9ab5400640 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d07ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9ab6800640 1 -- 172.21.15.62:0/3770221926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 msgr2=0x5642f2d07ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9ab6800640 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d07ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9ab6800640 1 -- 172.21.15.62:0/3770221926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 msgr2=0x5642f2d08200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9ab6800640 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d08200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9ab6800640 1 -- 172.21.15.62:0/3770221926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642f2d10b30 con 0x5642f2cfb510 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.126+0000 7f9ab5400640 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d07ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.127+0000 7f9ab6800640 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 0x5642f2d0d010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009cd0 tx=0x7f9ab0014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e719e7411107b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.127+0000 7f9a97e00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.127+0000 7f9a97e00640 1 -- 172.21.15.62:0/3770221926 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0016040 con 0x5642f2cfb510 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.127+0000 7f9a97e00640 1 -- 172.21.15.62:0/3770221926 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f9ab002c480 con 0x5642f2cfb510 2024-04-02T14:47:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.127+0000 7f9abfba6740 1 -- 172.21.15.62:0/3770221926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa40055b0 con 0x5642f2cfb510 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.127+0000 7f9a97e00640 1 -- 172.21.15.62:0/3770221926 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9ab003d050 con 0x5642f2cfb510 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.127+0000 7f9abfba6740 10 client.26459 fetch_fsmap learned FSMap version 36 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.127+0000 7f9abfba6740 1 -- 172.21.15.62:0/3770221926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5642f2cf8260 con 0x5642f2cfb510 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.127+0000 7f9a97e00640 1 -- 172.21.15.62:0/3770221926 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9aa40055b0 con 0x5642f2cfb510 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.128+0000 7f9a97e00640 1 -- 172.21.15.62:0/3770221926 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f9ab000f040 con 0x5642f2cfb510 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.128+0000 7f9abfba6740 10 client.26459 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.128+0000 7f9abfba6740 10 client.26459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.128+0000 7f9abfba6740 20 client.26459 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.128+0000 7f9abfba6740 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5642f2d7c730 0x5642f2d7eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.128+0000 7f9abfba6740 4 client.26459 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.128+0000 7f9abfba6740 1 -- 172.21.15.62:0/3770221926 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5642f2c66ba0 con 0x5642f2d7c730 2024-04-02T14:47:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.128+0000 7f9ab5400640 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5642f2d7c730 0x5642f2d7eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.128+0000 7f9ab5400640 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5642f2d7c730 0x5642f2d7eb50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b59924795bf17f16 server_cookie=1e17a302cb400863 in_seq=0 out_seq=0 2024-04-02T14:47:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.128+0000 7f9a97e00640 10 client.26459 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9a97e00640 1 -- 172.21.15.62:0/3770221926 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5642f2c66ba0 con 0x5642f2d7c730 2024-04-02T14:47:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9a97e00640 10 client.26459 handle_command_reply: tid=0 2024-04-02T14:47:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 1 client.26459 shutdown 2024-04-02T14:47:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 1 -- 172.21.15.62:0/3770221926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 msgr2=0x5642f2d0d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 0x5642f2d0d010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009cd0 tx=0x7f9ab0014040 comp rx=0 tx=0).stop 2024-04-02T14:47:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 1 -- 172.21.15.62:0/3770221926 shutdown_connections 2024-04-02T14:47:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5642f2d7c730 0x5642f2d7eb50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f2d04880 0x5642f2d08200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642f2cfc1f0 0x5642f2d07ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9a97e00640 0 client.26459 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:03.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 1 --2- 172.21.15.62:0/3770221926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642f2cfb510 0x5642f2d0d010 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 1 -- 172.21.15.62:0/3770221926 >> 172.21.15.62:0/3770221926 conn(0x5642f2c8f270 msgr2=0x5642f2cf7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 1 -- 172.21.15.62:0/3770221926 shutdown_connections 2024-04-02T14:47:03.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 1 -- 172.21.15.62:0/3770221926 wait complete. 2024-04-02T14:47:03.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.131+0000 7f9abfba6740 20 client.26459 trim_cache size 0 max 16384 2024-04-02T14:47:03.128 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:47:03.128 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-02T14:47:03.128 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "6b5439f8-c217-4095-8802-7744a420c9bc", 2024-04-02T14:47:03.128 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-02T14:47:03.129 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:47:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.132+0000 7f9ab7200640 1 -- 172.21.15.62:0/2054628179 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9a9c0a3d20 msgr2=0x7f9a9c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.132+0000 7f9ab7200640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9a9c0a3d20 0x7f9a9c0a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80effb0 tx=0x7f9aa8009260 comp rx=0 tx=0).stop 2024-04-02T14:47:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.132+0000 7f9ab7200640 1 -- 172.21.15.62:0/2054628179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab8073ed0 msgr2=0x7f9ab819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.132+0000 7f9ab7200640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab8073ed0 0x7f9ab819a5e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8000c00 tx=0x7f9aa8004a00 comp rx=0 tx=0).stop 2024-04-02T14:47:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.132+0000 7f9ab7200640 1 -- 172.21.15.62:0/2054628179 shutdown_connections 2024-04-02T14:47:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.132+0000 7f9ab7200640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f9a9c0a3d20 0x7f9a9c0a61e0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80effb0 tx=0x7f9aa8009260 comp rx=0 tx=0).stop 2024-04-02T14:47:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.132+0000 7f9ab7200640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ab810e210 0x7f9ab819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.132+0000 7f9ab7200640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab80747f0 0x7f9ab819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.132+0000 7f9ab7200640 1 --2- 172.21.15.62:0/2054628179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab8073ed0 0x7f9ab819a5e0 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8000c00 tx=0x7f9aa8004a00 comp rx=0 tx=0).stop 2024-04-02T14:47:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.132+0000 7f9ab7200640 1 -- 172.21.15.62:0/2054628179 >> 172.21.15.62:0/2054628179 conn(0x7f9ab80fd4a0 msgr2=0x7f9ab80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.133+0000 7f9ab7200640 1 -- 172.21.15.62:0/2054628179 shutdown_connections 2024-04-02T14:47:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.133+0000 7f9ab7200640 1 -- 172.21.15.62:0/2054628179 wait complete. 2024-04-02T14:47:03.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:47:03.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:47:03.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.381+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1343781490 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cbc11b2e0 msgr2=0x7f3cbc11d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.381+0000 7f3cc2c00640 1 --2- 172.21.15.62:0/1343781490 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cbc11b2e0 0x7f3cbc11d6d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009900 tx=0x7f3cac02bc30 comp rx=0 tx=0).stop 2024-04-02T14:47:03.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.381+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1343781490 shutdown_connections 2024-04-02T14:47:03.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.381+0000 7f3cc2c00640 1 --2- 172.21.15.62:0/1343781490 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cbc11b2e0 0x7f3cbc11d6d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.381+0000 7f3cc2c00640 1 --2- 172.21.15.62:0/1343781490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cbc10e350 0x7f3cbc10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.381+0000 7f3cc2c00640 1 --2- 172.21.15.62:0/1343781490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc110320 0x7f3cbc10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.381+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1343781490 >> 172.21.15.62:0/1343781490 conn(0x7f3cbc06cbe0 msgr2=0x7f3cbc06cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.382+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1343781490 shutdown_connections 2024-04-02T14:47:03.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.382+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1343781490 wait complete. 2024-04-02T14:47:03.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.383+0000 7f3cc2c00640 1 Processor -- start 2024-04-02T14:47:03.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.383+0000 7f3cc2c00640 1 -- start start 2024-04-02T14:47:03.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.383+0000 7f3cc2c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cbc10e350 0x7f3cbc1a7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.383+0000 7f3cc2c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cbc110320 0x7f3cbc1a7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.383+0000 7f3cc2c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc11b2e0 0x7f3cbc1abad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.383+0000 7f3cc2c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc120d50 con 0x7f3cbc110320 2024-04-02T14:47:03.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.383+0000 7f3cc2c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc120bd0 con 0x7f3cbc10e350 2024-04-02T14:47:03.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.383+0000 7f3cc2c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc120ed0 con 0x7f3cbc11b2e0 2024-04-02T14:47:03.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.384+0000 7f3cbbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cbc10e350 0x7f3cbc1a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.384+0000 7f3cbbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cbc10e350 0x7f3cbc1a7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:39786/0 (socket says 172.21.15.62:39786) 2024-04-02T14:47:03.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.384+0000 7f3cbbe00640 1 -- 172.21.15.62:0/1955764742 learned_addr learned my addr 172.21.15.62:0/1955764742 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:03.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.384+0000 7f3cbb400640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cbc110320 0x7f3cbc1a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.384+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc11b2e0 0x7f3cbc1abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.384+0000 7f3cbbe00640 1 -- 172.21.15.62:0/1955764742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc11b2e0 msgr2=0x7f3cbc1abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.384+0000 7f3cbbe00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc11b2e0 0x7f3cbc1abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.384+0000 7f3cbbe00640 1 -- 172.21.15.62:0/1955764742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cbc110320 msgr2=0x7f3cbc1a7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.384+0000 7f3cbbe00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cbc110320 0x7f3cbc1a7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.384+0000 7f3cbbe00640 1 -- 172.21.15.62:0/1955764742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cbc1ac200 con 0x7f3cbc10e350 2024-04-02T14:47:03.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.384+0000 7f3cbb400640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cbc110320 0x7f3cbc1a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:03.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.385+0000 7f3cbbe00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cbc10e350 0x7f3cbc1a7210 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009d50 tx=0x7f3cb400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.385+0000 7f3cb8c00640 1 -- 172.21.15.62:0/1955764742 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4003b70 con 0x7f3cbc10e350 2024-04-02T14:47:03.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.385+0000 7f3cb8c00640 1 -- 172.21.15.62:0/1955764742 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3cb4003d10 con 0x7f3cbc10e350 2024-04-02T14:47:03.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.385+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1955764742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cbc1ac4f0 con 0x7f3cbc10e350 2024-04-02T14:47:03.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.385+0000 7f3cb8c00640 1 -- 172.21.15.62:0/1955764742 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4010a90 con 0x7f3cbc10e350 2024-04-02T14:47:03.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.385+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1955764742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cbc1b3d80 con 0x7f3cbc10e350 2024-04-02T14:47:03.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.389+0000 7f3cb8c00640 1 -- 172.21.15.62:0/1955764742 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3cb4004000 con 0x7f3cbc10e350 2024-04-02T14:47:03.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.389+0000 7f3cc584c740 1 Processor -- start 2024-04-02T14:47:03.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.389+0000 7f3cc584c740 1 -- start start 2024-04-02T14:47:03.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 0x558c3ebeab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 0x558c3ebf3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc584c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558c3eb4e770 con 0x558c3ebf3cb0 2024-04-02T14:47:03.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc584c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558c3eaf6700 con 0x558c3ebeb0d0 2024-04-02T14:47:03.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc584c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558c3eb4a720 con 0x558c3ebea7b0 2024-04-02T14:47:03.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 0x558c3ebeab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cbbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 0x558c3ebf3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 0x558c3ebeab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:56044/0 (socket says 172.21.15.62:56044) 2024-04-02T14:47:03.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc0c00640 1 -- 172.21.15.62:0/3379216509 learned_addr learned my addr 172.21.15.62:0/3379216509 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:03.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cbb400640 1 --2- 172.21.15.62:0/3379216509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc0c00640 1 -- 172.21.15.62:0/3379216509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 msgr2=0x558c3ebf3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/3379216509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 0x558c3ebf3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc0c00640 1 -- 172.21.15.62:0/3379216509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 msgr2=0x558c3ebf60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/3379216509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.391+0000 7f3cb8c00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3ca00a3da0 0x7f3ca00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.390+0000 7f3cc0c00640 1 -- 172.21.15.62:0/3379216509 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c3e92c7d0 con 0x558c3ebea7b0 2024-04-02T14:47:03.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.391+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/3379216509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 0x558c3ebeab90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009230 tx=0x7f3cac009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c131d674d928f1d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.391+0000 7f3cb8c00640 1 -- 172.21.15.62:0/1955764742 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f3cb40d0d60 con 0x7f3cbc10e350 2024-04-02T14:47:03.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.391+0000 7f3cbbe00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3ca00a3da0 0x7f3ca00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.391+0000 7f3cb1200640 1 -- 172.21.15.62:0/3379216509 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac02c710 con 0x558c3ebea7b0 2024-04-02T14:47:03.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.391+0000 7f3cb1200640 1 -- 172.21.15.62:0/3379216509 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3cac004040 con 0x558c3ebea7b0 2024-04-02T14:47:03.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.391+0000 7f3cb1200640 1 -- 172.21.15.62:0/3379216509 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac0046c0 con 0x558c3ebea7b0 2024-04-02T14:47:03.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.392+0000 7f3cc584c740 1 -- 172.21.15.62:0/3379216509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 msgr2=0x558c3ebeab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.392+0000 7f3cc584c740 1 --2- 172.21.15.62:0/3379216509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 0x558c3ebeab90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009230 tx=0x7f3cac009560 comp rx=0 tx=0).stop 2024-04-02T14:47:03.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.392+0000 7f3cbbe00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3ca00a3da0 0x7f3ca00a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x558c3ebea280 tx=0x7f3cb40d4810 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.392+0000 7f3cc584c740 1 -- 172.21.15.62:0/3379216509 shutdown_connections 2024-04-02T14:47:03.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.392+0000 7f3cc584c740 1 --2- 172.21.15.62:0/3379216509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.392+0000 7f3cc584c740 1 --2- 172.21.15.62:0/3379216509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 0x558c3ebf3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.392+0000 7f3cc584c740 1 --2- 172.21.15.62:0/3379216509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 0x558c3ebeab90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.392+0000 7f3cc584c740 1 -- 172.21.15.62:0/3379216509 >> 172.21.15.62:0/3379216509 conn(0x558c3ebe4ee0 msgr2=0x558c3ebe52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.392+0000 7f3cc584c740 1 -- 172.21.15.62:0/3379216509 shutdown_connections 2024-04-02T14:47:03.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.393+0000 7f3cc584c740 1 -- 172.21.15.62:0/3379216509 wait complete. 2024-04-02T14:47:03.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.393+0000 7f3cc584c740 1 -- start start 2024-04-02T14:47:03.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.393+0000 7f3cc584c740 1 Processor -- start 2024-04-02T14:47:03.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 0x558c3ebf0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebeb0d0 0x558c3ebf1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cc584c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558c3ebf6a40 con 0x558c3ebeb0d0 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cc584c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558c3eb09ac0 con 0x558c3ebea7b0 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cc584c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558c3ebf6bc0 con 0x558c3ebf3cb0 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cc584c740 10 client.0 _ll_register_callbacks cb 0x558c3eb0c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cc0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 0x558c3ebf0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cbb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebeb0d0 0x558c3ebf1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cc0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 0x558c3ebf0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:39810/0 (socket says 172.21.15.62:39810) 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cc0c00640 1 -- 172.21.15.62:0/3207625851 learned_addr learned my addr 172.21.15.62:0/3207625851 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.394+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.395+0000 7f3cbb400640 1 -- 172.21.15.62:0/3207625851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebf3cb0 msgr2=0x558c3ebf1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.395+0000 7f3cbb400640 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.395+0000 7f3cbb400640 1 -- 172.21.15.62:0/3207625851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 msgr2=0x558c3ebf0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.395+0000 7f3cbb400640 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 0x558c3ebf0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.395+0000 7f3cbb400640 1 -- 172.21.15.62:0/3207625851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c3e8623f0 con 0x558c3ebeb0d0 2024-04-02T14:47:03.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.395+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 0x558c3ebf0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:03.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.395+0000 7f3cbb400640 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebeb0d0 0x558c3ebf1240 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4012cf0 tx=0x7f3ca400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cfd5ea408aba6cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.396+0000 7f3ca9600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.396+0000 7f3ca9600640 1 -- 172.21.15.62:0/3207625851 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4012110 con 0x558c3ebeb0d0 2024-04-02T14:47:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.396+0000 7f3ca9600640 1 -- 172.21.15.62:0/3207625851 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f3ca4004050 con 0x558c3ebeb0d0 2024-04-02T14:47:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.396+0000 7f3cc584c740 1 -- 172.21.15.62:0/3207625851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cac004040 con 0x558c3ebeb0d0 2024-04-02T14:47:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.396+0000 7f3ca9600640 1 -- 172.21.15.62:0/3207625851 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3ca40122b0 con 0x558c3ebeb0d0 2024-04-02T14:47:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.396+0000 7f3cc584c740 10 client.16338 fetch_fsmap learned FSMap version 36 2024-04-02T14:47:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.396+0000 7f3cc584c740 1 -- 172.21.15.62:0/3207625851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558c3e8220f0 con 0x558c3ebeb0d0 2024-04-02T14:47:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.397+0000 7f3ca9600640 1 -- 172.21.15.62:0/3207625851 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3cac004040 con 0x558c3ebeb0d0 2024-04-02T14:47:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.397+0000 7f3ca9600640 1 -- 172.21.15.62:0/3207625851 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f3ca4026580 con 0x558c3ebeb0d0 2024-04-02T14:47:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.397+0000 7f3cc584c740 10 client.16338 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:47:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.397+0000 7f3cc584c740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:47:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.397+0000 7f3cc584c740 20 client.16338 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.397+0000 7f3cc584c740 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558c3ec03cc0 0x558c3ec060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.397+0000 7f3cc584c740 4 client.16338 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:47:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.397+0000 7f3cc584c740 1 -- 172.21.15.62:0/3207625851 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c3eaef630 con 0x558c3ec03cc0 2024-04-02T14:47:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.398+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558c3ec03cc0 0x558c3ec060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.398+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558c3ec03cc0 0x558c3ec060b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c203153da8c5f3c6 server_cookie=d1ccf0a184639ae0 in_seq=0 out_seq=0 2024-04-02T14:47:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.398+0000 7f3ca9600640 10 client.16338 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.399+0000 7f3ca9600640 1 -- 172.21.15.62:0/3207625851 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c3eaef630 con 0x558c3ec03cc0 2024-04-02T14:47:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.399+0000 7f3ca9600640 10 client.16338 handle_command_reply: tid=0 2024-04-02T14:47:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.399+0000 7f3cc584c740 1 client.16338 shutdown 2024-04-02T14:47:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3cc584c740 1 -- 172.21.15.62:0/3207625851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebeb0d0 msgr2=0x558c3ebf1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3cc584c740 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebeb0d0 0x558c3ebf1240 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4012cf0 tx=0x7f3ca400f790 comp rx=0 tx=0).stop 2024-04-02T14:47:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3cc584c740 1 -- 172.21.15.62:0/3207625851 shutdown_connections 2024-04-02T14:47:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3cc584c740 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558c3ec03cc0 0x558c3ec060b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3cc584c740 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3cc584c740 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebeb0d0 0x558c3ebf1240 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3cc584c740 1 --2- 172.21.15.62:0/3207625851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 0x558c3ebf0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3cc584c740 1 -- 172.21.15.62:0/3207625851 >> 172.21.15.62:0/3207625851 conn(0x558c3ebe4ee0 msgr2=0x558c3ebe6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3ca9600640 0 client.16338 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3cc584c740 1 -- 172.21.15.62:0/3207625851 shutdown_connections 2024-04-02T14:47:03.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3cc584c740 1 -- 172.21.15.62:0/3207625851 wait complete. 2024-04-02T14:47:03.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.400+0000 7f3cc584c740 20 client.16338 trim_cache size 0 max 16384 2024-04-02T14:47:03.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.425+0000 7f3cc584c740 1 Processor -- start 2024-04-02T14:47:03.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.425+0000 7f3cc584c740 1 -- start start 2024-04-02T14:47:03.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.425+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 0x558c3ebf3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.425+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebeb0d0 0x558c3ebff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.425+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.425+0000 7f3cc584c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1bc610 con 0x558c3ebf3cb0 2024-04-02T14:47:03.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.425+0000 7f3cc584c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1be300 con 0x558c3ebea7b0 2024-04-02T14:47:03.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.425+0000 7f3cc584c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1bcd40 con 0x558c3ebeb0d0 2024-04-02T14:47:03.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.426+0000 7f3cbb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebeb0d0 0x558c3ebff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.426+0000 7f3cbb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebeb0d0 0x558c3ebff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:56070/0 (socket says 172.21.15.62:56070) 2024-04-02T14:47:03.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.426+0000 7f3cbb400640 1 -- 172.21.15.62:0/1783242528 learned_addr learned my addr 172.21.15.62:0/1783242528 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:03.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.426+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/1783242528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 0x558c3ebf3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.426+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/1783242528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.426+0000 7f3cbb400640 1 -- 172.21.15.62:0/1783242528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 msgr2=0x558c3ebf3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.426+0000 7f3cbb400640 1 --2- 172.21.15.62:0/1783242528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 0x558c3ebf3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.426+0000 7f3cbb400640 1 -- 172.21.15.62:0/1783242528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 msgr2=0x558c3ebf13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.426+0000 7f3cbb400640 1 --2- 172.21.15.62:0/1783242528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.426+0000 7f3cbb400640 1 -- 172.21.15.62:0/1783242528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c3ebe5ab0 con 0x558c3ebeb0d0 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.427+0000 7f3cbb400640 1 --2- 172.21.15.62:0/1783242528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebeb0d0 0x558c3ebff960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4000c00 tx=0x7f3ca4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21aabd111a1affbf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.427+0000 7f3cabe00640 1 -- 172.21.15.62:0/1783242528 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca40128b0 con 0x558c3ebeb0d0 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.427+0000 7f3cabe00640 1 -- 172.21.15.62:0/1783242528 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3ca402b070 con 0x558c3ebeb0d0 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.427+0000 7f3cabe00640 1 -- 172.21.15.62:0/1783242528 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4003850 con 0x558c3ebeb0d0 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.428+0000 7f3cc584c740 1 -- 172.21.15.62:0/1783242528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebeb0d0 msgr2=0x558c3ebff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.428+0000 7f3cc584c740 1 --2- 172.21.15.62:0/1783242528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebeb0d0 0x558c3ebff960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4000c00 tx=0x7f3ca4009c00 comp rx=0 tx=0).stop 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.428+0000 7f3cc584c740 1 -- 172.21.15.62:0/1783242528 shutdown_connections 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.428+0000 7f3cc584c740 1 --2- 172.21.15.62:0/1783242528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebf13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.428+0000 7f3cc584c740 1 --2- 172.21.15.62:0/1783242528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebeb0d0 0x558c3ebff960 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.428+0000 7f3cc584c740 1 --2- 172.21.15.62:0/1783242528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebea7b0 0x558c3ebf3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.428+0000 7f3cc584c740 1 -- 172.21.15.62:0/1783242528 >> 172.21.15.62:0/1783242528 conn(0x558c3ebe4ee0 msgr2=0x558c3ebf0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.428+0000 7f3cc584c740 1 -- 172.21.15.62:0/1783242528 shutdown_connections 2024-04-02T14:47:03.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.428+0000 7f3cc584c740 1 -- 172.21.15.62:0/1783242528 wait complete. 2024-04-02T14:47:03.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.429+0000 7f3cc584c740 1 -- start start 2024-04-02T14:47:03.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.429+0000 7f3cc584c740 1 Processor -- start 2024-04-02T14:47:03.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 0x558c3ebfc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 0x558c3ebed850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cc584c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebedf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cc584c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1bdb70 con 0x558c3ebf3cb0 2024-04-02T14:47:03.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cc584c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1bd440 con 0x558c3ebeb0d0 2024-04-02T14:47:03.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cc584c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1ba5f0 con 0x558c3ebea7b0 2024-04-02T14:47:03.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cc584c740 10 client.0 _ll_register_callbacks cb 0x7f3ca4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cbb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 0x558c3ebed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cbb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 0x558c3ebed850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:39824/0 (socket says 172.21.15.62:39824) 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cbb400640 1 -- 172.21.15.62:0/1405548666 learned_addr learned my addr 172.21.15.62:0/1405548666 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 0x558c3ebfc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebedf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cbb400640 1 -- 172.21.15.62:0/1405548666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 msgr2=0x558c3ebfc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cbb400640 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 0x558c3ebfc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cbb400640 1 -- 172.21.15.62:0/1405548666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 msgr2=0x558c3ebedf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cbb400640 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebedf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cbb400640 1 -- 172.21.15.62:0/1405548666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c3ec69230 con 0x558c3ebeb0d0 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.430+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebedf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.431+0000 7f3cbb400640 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 0x558c3ebed850 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4012cf0 tx=0x7f3ca4012920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e03030f176fd30c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.431+0000 7f3ca9600640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:47:03.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.431+0000 7f3ca9600640 1 -- 172.21.15.62:0/1405548666 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4034070 con 0x558c3ebeb0d0 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.431+0000 7f3ca9600640 1 -- 172.21.15.62:0/1405548666 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f3ca4005250 con 0x558c3ebeb0d0 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.431+0000 7f3cc584c740 1 -- 172.21.15.62:0/1405548666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca402b070 con 0x558c3ebeb0d0 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.431+0000 7f3ca9600640 1 -- 172.21.15.62:0/1405548666 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3ca403e050 con 0x558c3ebeb0d0 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.431+0000 7f3cc584c740 10 client.26477 fetch_fsmap learned FSMap version 36 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.431+0000 7f3cc584c740 1 -- 172.21.15.62:0/1405548666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558c3ec6a3b0 con 0x558c3ebeb0d0 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.432+0000 7f3ca9600640 1 -- 172.21.15.62:0/1405548666 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3ca402b070 con 0x558c3ebeb0d0 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.432+0000 7f3ca9600640 1 -- 172.21.15.62:0/1405548666 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f3ca4030580 con 0x558c3ebeb0d0 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.432+0000 7f3cc584c740 10 client.26477 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.432+0000 7f3cc584c740 10 client.26477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.432+0000 7f3cc584c740 20 client.26477 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.432+0000 7f3cc584c740 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558c3ec6ab70 0x558c3ec6cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.432+0000 7f3cc584c740 4 client.26477 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:47:03.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.432+0000 7f3cc584c740 1 -- 172.21.15.62:0/1405548666 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c3eb5bcb0 con 0x558c3ec6ab70 2024-04-02T14:47:03.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.432+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558c3ec6ab70 0x558c3ec6cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:03.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.433+0000 7f3cc0c00640 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558c3ec6ab70 0x558c3ec6cf90 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a28d4d574b6e9413 server_cookie=9df41dea93e2547f in_seq=0 out_seq=0 2024-04-02T14:47:03.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.433+0000 7f3ca9600640 10 client.26477 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:03.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.434+0000 7f3ca9600640 1 -- 172.21.15.62:0/1405548666 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x558c3eb5bcb0 con 0x558c3ec6ab70 2024-04-02T14:47:03.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.434+0000 7f3ca9600640 10 client.26477 handle_command_reply: tid=0 2024-04-02T14:47:03.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.434+0000 7f3cc584c740 1 client.26477 shutdown 2024-04-02T14:47:03.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.434+0000 7f3cc584c740 1 -- 172.21.15.62:0/1405548666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 msgr2=0x558c3ebed850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.434+0000 7f3cc584c740 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 0x558c3ebed850 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4012cf0 tx=0x7f3ca4012920 comp rx=0 tx=0).stop 2024-04-02T14:47:03.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.435+0000 7f3cc584c740 1 -- 172.21.15.62:0/1405548666 shutdown_connections 2024-04-02T14:47:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.435+0000 7f3cc584c740 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x558c3ec6ab70 0x558c3ec6cf90 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.435+0000 7f3cc584c740 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3ebf3cb0 0x558c3ebedf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.435+0000 7f3cc584c740 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558c3ebeb0d0 0x558c3ebed850 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.435+0000 7f3cc584c740 1 --2- 172.21.15.62:0/1405548666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c3ebea7b0 0x558c3ebfc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.435+0000 7f3ca9600640 0 client.26477 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.435+0000 7f3cc584c740 1 -- 172.21.15.62:0/1405548666 >> 172.21.15.62:0/1405548666 conn(0x558c3ebe4ee0 msgr2=0x558c3ebf1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.435+0000 7f3cc584c740 1 -- 172.21.15.62:0/1405548666 shutdown_connections 2024-04-02T14:47:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.435+0000 7f3cc584c740 1 -- 172.21.15.62:0/1405548666 wait complete. 2024-04-02T14:47:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.435+0000 7f3cc584c740 20 client.26477 trim_cache size 0 max 16384 2024-04-02T14:47:03.431 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:47:03.432 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "scrub active (852 inodes in the stack)", 2024-04-02T14:47:03.432 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-02T14:47:03.432 INFO:teuthology.orchestra.run.smithi062.stdout: "6b5439f8-c217-4095-8802-7744a420c9bc": { 2024-04-02T14:47:03.432 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-04-02T14:47:03.432 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "6b5439f8-c217-4095-8802-7744a420c9bc", 2024-04-02T14:47:03.432 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-04-02T14:47:03.432 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:47:03.432 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:47:03.432 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:47:03.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1955764742 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3ca00a3da0 msgr2=0x7f3ca00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3ca00a3da0 0x7f3ca00a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x558c3ebea280 tx=0x7f3cb40d4810 comp rx=0 tx=0).stop 2024-04-02T14:47:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1955764742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cbc10e350 msgr2=0x7f3cbc1a7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cbc10e350 0x7f3cbc1a7210 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009d50 tx=0x7f3cb400c6a0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1955764742 shutdown_connections 2024-04-02T14:47:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3ca00a3da0 0x7f3ca00a6260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc11b2e0 0x7f3cbc1abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cbc110320 0x7f3cbc1a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 --2- 172.21.15.62:0/1955764742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cbc10e350 0x7f3cbc1a7210 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1955764742 >> 172.21.15.62:0/1955764742 conn(0x7f3cbc06cbe0 msgr2=0x7f3cbc0725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1955764742 shutdown_connections 2024-04-02T14:47:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:03.436+0000 7f3cc2c00640 1 -- 172.21.15.62:0/1955764742 wait complete. 2024-04-02T14:47:03.451 INFO:tasks.cephfs.filesystem:scrub status for tag:6b5439f8-c217-4095-8802-7744a420c9bc - {'path': '/', 'tag': '6b5439f8-c217-4095-8802-7744a420c9bc', 'options': 'recursive,force'} 2024-04-02T14:47:03.510 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-mon[36135]: pgmap v94: 97 pgs: 97 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 1.09k op/s 2024-04-02T14:47:03.510 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-mon[36135]: scrub queued for path: / 2024-04-02T14:47:03.510 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:03.510 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:03.511 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: 2024-04-02T14:47:03.258+0000 7f25d5509700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002bb4 [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt auth{0=1} v6 ap=1 s=4267 n(v0 rc2024-04-02T14:46:55.871169+0000 b4267 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@4} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25310 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5633844f7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002bb4:[<0x10000002bb3/ti-gpmc.txt v2>,<0x10000001e8a/bus-devices v2173>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002bb4:[<0x10000002bb3/ti-gpmc.txt v6>,<0x10000001e8a/bus-devices v2173>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.511 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: 2024-04-02T14:47:03.481+0000 7f25d5509700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002a27 [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c auth v12 ap=1 s=13293 n(v0 rc2024-04-02T14:46:54.305194+0000 b13293 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5633844f4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002a27:[<0x10000002a26/txtimestamp.c v2>,<0x100000029e1/timestamping v148>,<0x10000001e8a/networking v2221>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002a27:[<0x10000002a26/txtimestamp.c v12>,<0x100000029e1/timestamping v322>,<0x10000001e8a/networking v2613>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.511 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: 2024-04-02T14:47:03.481+0000 7f25d5509700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002a2a [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/timestamping.c auth v18 ap=1 s=15146 n(v0 rc2024-04-02T14:46:54.312194+0000 b15146 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5633844f6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002a2a:[<0x10000002a26/timestamping.c v8>,<0x100000029e1/timestamping v148>,<0x10000001e8a/networking v2221>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002a2a:[<0x10000002a26/timestamping.c v18>,<0x100000029e1/timestamping v322>,<0x10000001e8a/networking v2613>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.511 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: 2024-04-02T14:47:03.481+0000 7f25d5509700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002a2b [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/.gitignore auth v14 ap=1 s=41 n(v0 rc2024-04-02T14:46:54.314194+0000 b41 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5633844f6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002a2b:[<0x10000002a26/.gitignore v10>,<0x100000029e1/timestamping v148>,<0x10000001e8a/networking v2221>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002a2b:[<0x10000002a26/.gitignore v14>,<0x100000029e1/timestamping v322>,<0x10000001e8a/networking v2613>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.511 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: 2024-04-02T14:47:03.481+0000 7f25d5509700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002a28 [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/Makefile auth v22 ap=1 s=408 n(v0 rc2024-04-02T14:46:54.308194+0000 b408 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@4} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25310 | ptrwaiter=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56338402e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002a28:[<0x10000002a26/Makefile v4>,<0x100000029e1/timestamping v148>,<0x10000001e8a/networking v2221>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002a28:[<0x10000002a26/Makefile v22>,<0x100000029e1/timestamping v322>,<0x10000001e8a/networking v2613>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.511 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: 2024-04-02T14:47:03.482+0000 7f25d5509700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002a29 [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c auth v16 ap=1 s=3089 n(v0 rc2024-04-02T14:46:54.310194+0000 b3089 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5633844f5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002a29:[<0x10000002a26/hwtstamp_config.c v6>,<0x100000029e1/timestamping v148>,<0x10000001e8a/networking v2221>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002a29:[<0x10000002a26/hwtstamp_config.c v16>,<0x100000029e1/timestamping v322>,<0x10000001e8a/networking v2613>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.511 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: 2024-04-02T14:47:03.514+0000 7f25d5509700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002baf [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt auth v4 ap=1 s=4488 n(v0 rc2024-04-02T14:46:55.863169+0000 b4488 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563383c44c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002baf:[<0x10000002bae/pxa3xx-nand.txt v2>,<0x10000002bad/nand v4>,<0x10000001e8a/mtd v2167>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002baf:[<0x10000002bae/pxa3xx-nand.txt v4>,<0x10000002bad/nand v18>,<0x10000001e8a/mtd v2451>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.512 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-mon[32116]: pgmap v94: 97 pgs: 97 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 1.09k op/s 2024-04-02T14:47:03.512 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-mon[32116]: scrub queued for path: / 2024-04-02T14:47:03.512 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:03.512 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:03.513 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-g[71943]: 2024-04-02T14:47:03.257+0000 7f23f9800700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ba7 [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/compartmentalisation.txt auth v6 ap=1 s=1944 n(v0 rc2024-04-02T14:46:55.849169+0000 b1944 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x557022b08b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002ba7:[<0x10000002ba6/compartmentalisation.txt v2>,<0x10000001e8a/mn10300 v2151>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002ba7:[<0x10000002ba6/compartmentalisation.txt v6>,<0x10000001e8a/mn10300 v2331>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.513 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-g[71943]: 2024-04-02T14:47:03.259+0000 7f23f9800700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ba8 [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/ABI.txt auth v8 ap=1 s=4007 n(v0 rc2024-04-02T14:46:55.851169+0000 b4007 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x557022b09080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002ba8:[<0x10000002ba6/ABI.txt v4>,<0x10000001e8a/mn10300 v2151>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002ba8:[<0x10000002ba6/ABI.txt v8>,<0x10000001e8a/mn10300 v2331>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.647 INFO:tasks.workunit.client.0.smithi062.stderr:+ echo Ok 2024-04-02T14:47:03.647 INFO:tasks.workunit.client.0.smithi062.stdout:Ok 2024-04-02T14:47:03.649 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-02T14:47:03.649 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-02T14:47:03.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:03 smithi204 ceph-mon[35948]: pgmap v94: 97 pgs: 97 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 1.09k op/s 2024-04-02T14:47:03.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:03 smithi204 ceph-mon[35948]: scrub queued for path: / 2024-04-02T14:47:03.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:03 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:03.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:03 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:03.838 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: 2024-04-02T14:47:03.516+0000 7f12b6a00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002b98 [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/00-INDEX auth v14 ap=1 s=333 n(v0 rc2024-04-02T14:46:55.825170+0000 b333 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561dfd84ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002b98:[<0x10000002b97/00-INDEX v2>,<0x10000002b95/masters v16>,<0x10000001e8a/w1 v2145>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002b98:[<0x10000002b97/00-INDEX v14>,<0x10000002b95/masters v46>,<0x10000001e8a/w1 v2461>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.839 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: 2024-04-02T14:47:03.516+0000 7f12b6a00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002b99 [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2490 auth v20 ap=1 s=3442 n(v0 rc2024-04-02T14:46:55.827170+0000 b3442 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561dfd793b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002b99:[<0x10000002b97/ds2490 v4>,<0x10000002b95/masters v16>,<0x10000001e8a/w1 v2145>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002b99:[<0x10000002b97/ds2490 v20>,<0x10000002b95/masters v46>,<0x10000001e8a/w1 v2461>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.839 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: 2024-04-02T14:47:03.516+0000 7f12b6a00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002b9a [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/w1-gpio auth v22 ap=1 s=698 n(v0 rc2024-04-02T14:46:55.828170+0000 b698 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561dfd793600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002b9a:[<0x10000002b97/w1-gpio v6>,<0x10000002b95/masters v16>,<0x10000001e8a/w1 v2145>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002b9a:[<0x10000002b97/w1-gpio v22>,<0x10000002b95/masters v46>,<0x10000001e8a/w1 v2461>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.839 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: 2024-04-02T14:47:03.517+0000 7f12b6a00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002b9b [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/omap-hdq auth v16 ap=1 s=1875 n(v0 rc2024-04-02T14:46:55.830170+0000 b1875 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561dfd794100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002b9b:[<0x10000002b97/omap-hdq v8>,<0x10000002b95/masters v16>,<0x10000001e8a/w1 v2145>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002b9b:[<0x10000002b97/omap-hdq v16>,<0x10000002b95/masters v46>,<0x10000001e8a/w1 v2461>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.839 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: 2024-04-02T14:47:03.517+0000 7f12b6a00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002b9c [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2482 auth v18 ap=1 s=780 n(v0 rc2024-04-02T14:46:55.831170+0000 b780 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@3} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561dfd795180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002b9c:[<0x10000002b97/ds2482 v10>,<0x10000002b95/masters v16>,<0x10000001e8a/w1 v2145>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002b9c:[<0x10000002b97/ds2482 v18>,<0x10000002b95/masters v46>,<0x10000001e8a/w1 v2461>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.839 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:47:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: 2024-04-02T14:47:03.518+0000 7f12b6a00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002b9d [4,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/mxc-w1 auth v26 ap=1 s=474 n(v0 rc2024-04-02T14:46:55.833170+0000 b474 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@4} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25310 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561dfd795700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002b9d:[<0x10000002b97/mxc-w1 v12>,<0x10000002b95/masters v16>,<0x10000001e8a/w1 v2145>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002b9d:[<0x10000002b97/mxc-w1 v26>,<0x10000002b95/masters v46>,<0x10000001e8a/w1 v2461>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:03.864 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:47:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: 2024-04-02T14:47:03.575+0000 7f25d5509700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002de1 [5,head] /volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt auth v6 ap=1 s=120016 n(v0 rc2024-04-02T14:46:58.763123+0000 b120016 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25310=0-4194304@4} caps={25310=pAsxLsXsxFsxcrwb/pAsxXsxFxcwb@1},l=25310 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5633844a9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002de1:[<0x10000002de0/UserModeLinux-HOWTO.txt v2>,<0x10000002dc8/uml v36>,<0x10000001e8a/virtual v2183>,<0x10000001e89/Documentation v521>,<0x10000000201/linux-4.0.5 v166>,<0x10000000006/untar_tar v170>,<0x10000000005/tmp v148>,<0x100000001fb/client.0 v135>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v150>,<0x10000000001/sv_1 v167>,<0x10000000000/qa v156>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000002de1:[<0x10000002de0/UserModeLinux-HOWTO.txt v6>,<0x10000002dc8/uml v36>,<0x10000001e8a/virtual v2245>,<0x10000001e89/Documentation v1296>,<0x10000000201/linux-4.0.5 v179>,<0x10000000006/untar_tar v172>,<0x10000000005/tmp v142>,<0x100000001fb/client.0 v137>,<0x100000001fa/578900f3-6c0c-478c-90ae-75c0f7bd10b8 v153>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v159>,<0x1/volumes v145>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T14:47:04.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002ba7 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/compartmentalisation.txt) see mds.g log and `damage ls` output for details 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002bb4 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002ba8 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/ABI.txt) see mds.g log and `damage ls` output for details 2024-04-02T14:47:04.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002a27 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002a2a (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/timestamping.c) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002a2b (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/.gitignore) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002a28 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/Makefile) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002a29 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002baf (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002b98 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/00-INDEX) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002b99 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2490) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002b9a (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/w1-gpio) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002b9b (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/omap-hdq) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002b9c (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2482) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002b9d (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/mxc-w1) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: Scrub error on inode 0x10000002de1 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:04.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:04 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002ba7 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/compartmentalisation.txt) see mds.g log and `damage ls` output for details 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002bb4 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002ba8 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/ABI.txt) see mds.g log and `damage ls` output for details 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002a27 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002a2a (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/timestamping.c) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002a2b (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/.gitignore) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002a28 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/Makefile) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002a29 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002baf (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002b98 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/00-INDEX) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002b99 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2490) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002b9a (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/w1-gpio) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002b9b (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/omap-hdq) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002b9c (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2482) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002b9d (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/mxc-w1) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: Scrub error on inode 0x10000002de1 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:04.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:04 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002ba7 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/compartmentalisation.txt) see mds.g log and `damage ls` output for details 2024-04-02T14:47:04.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002bb4 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002ba8 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/ABI.txt) see mds.g log and `damage ls` output for details 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002a27 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002a2a (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/timestamping.c) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002a2b (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/.gitignore) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002a28 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/Makefile) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002a29 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.867 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002baf (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002b98 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/00-INDEX) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002b99 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2490) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002b9a (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/w1-gpio) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002b9b (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/omap-hdq) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002b9c (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2482) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002b9d (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/mxc-w1) see mds.a log and `damage ls` output for details 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: Scrub error on inode 0x10000002de1 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt) see mds.h log and `damage ls` output for details 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.868 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:04.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:04.869 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:04 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:05.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:05.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:05.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:05.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:05.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: active paths [/] 2024-04-02T14:47:05.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:05 smithi204 ceph-mon[35948]: pgmap v95: 97 pgs: 97 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 2.7 MiB/s wr, 985 op/s 2024-04-02T14:47:05.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:05.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:05.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:05.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:05.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:05.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: active paths [/] 2024-04-02T14:47:05.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:05 smithi062 ceph-mon[32116]: pgmap v95: 97 pgs: 97 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 2.7 MiB/s wr, 985 op/s 2024-04-02T14:47:05.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:05.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:05.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:05.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:05.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:05.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: active paths [/] 2024-04-02T14:47:05.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: scrub summary: idle+waiting paths [/] 2024-04-02T14:47:05.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:05 smithi113 ceph-mon[36135]: pgmap v95: 97 pgs: 97 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 2.7 MiB/s wr, 985 op/s 2024-04-02T14:47:07.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:07 smithi204 ceph-mon[35948]: pgmap v96: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 1.33k op/s 2024-04-02T14:47:07.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[32112]: 2024-04-02T14:47:07.437+0000 7f7747c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T14:47:07.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:07 smithi062 ceph-mon[32116]: pgmap v96: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 1.33k op/s 2024-04-02T14:47:07.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:07 smithi113 ceph-mon[36135]: pgmap v96: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 1.33k op/s 2024-04-02T14:47:08.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:08 smithi204 ceph-mon[35948]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T14:47:08.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:08 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] up:active 2024-04-02T14:47:08.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:08 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:47:08.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:08 smithi062 ceph-mon[32116]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T14:47:08.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:08 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] up:active 2024-04-02T14:47:08.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:08 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:47:08.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:08 smithi113 ceph-mon[36135]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T14:47:08.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:08 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] up:active 2024-04-02T14:47:08.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:08 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:47:09.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:09 smithi204 ceph-mon[35948]: pgmap v97: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 1.40k op/s 2024-04-02T14:47:09.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:09 smithi062 ceph-mon[32116]: pgmap v97: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 1.40k op/s 2024-04-02T14:47:09.866 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:09 smithi113 ceph-mon[36135]: pgmap v97: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 1.40k op/s 2024-04-02T14:47:10.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:10 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] up:active 2024-04-02T14:47:10.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:10 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:47:10.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:10 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:47:10.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:10 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] up:active 2024-04-02T14:47:10.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:10 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:47:10.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:10 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:47:10.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:10 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] up:active 2024-04-02T14:47:10.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:10 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:47:10.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:10 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:47:11.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:11 smithi204 ceph-mon[35948]: pgmap v98: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 1.40k op/s 2024-04-02T14:47:11.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:11 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:47:11.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:11 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:47:11.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:11 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:47:11.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:11 smithi204 ceph-mon[35948]: scrub summary: idle 2024-04-02T14:47:11.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:11 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] up:active 2024-04-02T14:47:11.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:11 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:47:11.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:11 smithi062 ceph-mon[32116]: pgmap v98: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 1.40k op/s 2024-04-02T14:47:11.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:11 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:47:11.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:11 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:47:11.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:11 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:47:11.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:11 smithi062 ceph-mon[32116]: scrub summary: idle 2024-04-02T14:47:11.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:11 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] up:active 2024-04-02T14:47:11.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:11 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:47:11.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:11 smithi113 ceph-mon[36135]: pgmap v98: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 1.40k op/s 2024-04-02T14:47:11.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:11 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:47:11.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:11 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:47:11.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:11 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:47:11.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:11 smithi113 ceph-mon[36135]: scrub summary: idle 2024-04-02T14:47:11.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:11 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] up:active 2024-04-02T14:47:11.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:11 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:47:12.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:12 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:47:12.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:12 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:47:12.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:12 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:47:13.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:13 smithi204 ceph-mon[35948]: pgmap v99: 97 pgs: 97 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 1.53k op/s 2024-04-02T14:47:13.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:13 smithi062 ceph-mon[32116]: pgmap v99: 97 pgs: 97 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 1.53k op/s 2024-04-02T14:47:13.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:13 smithi113 ceph-mon[36135]: pgmap v99: 97 pgs: 97 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 1.53k op/s 2024-04-02T14:47:14.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:14 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:47:14.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:14 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:47:14.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:14 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:47:14.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:14 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:47:14.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:14 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:47:14.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:14 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:47:14.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:14 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:47:14.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:14 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:47:14.865 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:14 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:47:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.943+0000 7f23ece00640 1 -- 172.21.15.62:0/2778422650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8107ac0 msgr2=0x7f23e8107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.943+0000 7f23ece00640 1 --2- 172.21.15.62:0/2778422650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8107ac0 0x7f23e8107ea0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009900 tx=0x7f23d802bc30 comp rx=0 tx=0).stop 2024-04-02T14:47:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.943+0000 7f23ece00640 1 -- 172.21.15.62:0/2778422650 shutdown_connections 2024-04-02T14:47:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.943+0000 7f23ece00640 1 --2- 172.21.15.62:0/2778422650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23e8102470 0x7f23e810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.943+0000 7f23ece00640 1 --2- 172.21.15.62:0/2778422650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23e8101ab0 0x7f23e8101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.943+0000 7f23ece00640 1 --2- 172.21.15.62:0/2778422650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8107ac0 0x7f23e8107ea0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.943+0000 7f23ece00640 1 -- 172.21.15.62:0/2778422650 >> 172.21.15.62:0/2778422650 conn(0x7f23e80fd5c0 msgr2=0x7f23e80ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.943+0000 7f23ece00640 1 -- 172.21.15.62:0/2778422650 shutdown_connections 2024-04-02T14:47:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.943+0000 7f23ece00640 1 -- 172.21.15.62:0/2778422650 wait complete. 2024-04-02T14:47:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.944+0000 7f23ece00640 1 Processor -- start 2024-04-02T14:47:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.944+0000 7f23ece00640 1 -- start start 2024-04-02T14:47:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23ece00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23e8101ab0 0x7f23e8073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23ece00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8102470 0x7f23e8074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23ece00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23e8107ac0 0x7f23e8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23ece00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8111ff0 con 0x7f23e8107ac0 2024-04-02T14:47:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23ece00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8111e70 con 0x7f23e8102470 2024-04-02T14:47:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23ece00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8112170 con 0x7f23e8101ab0 2024-04-02T14:47:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23e5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23e8101ab0 0x7f23e8073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23dfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8102470 0x7f23e8074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23e5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23e8101ab0 0x7f23e8073e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39712/0 (socket says 172.21.15.62:39712) 2024-04-02T14:47:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23e5e00640 1 -- 172.21.15.62:0/147647084 learned_addr learned my addr 172.21.15.62:0/147647084 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23e6800640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23e8107ac0 0x7f23e8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23e5e00640 1 -- 172.21.15.62:0/147647084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8102470 msgr2=0x7f23e8074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23e5e00640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8102470 0x7f23e8074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23e5e00640 1 -- 172.21.15.62:0/147647084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23e8107ac0 msgr2=0x7f23e8074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23e5e00640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23e8107ac0 0x7f23e8074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.945+0000 7f23e5e00640 1 -- 172.21.15.62:0/147647084 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e81a6890 con 0x7f23e8101ab0 2024-04-02T14:47:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.946+0000 7f23e5e00640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23e8101ab0 0x7f23e8073e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005ab0 tx=0x7f23d802dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.946+0000 7f23dea00640 1 -- 172.21.15.62:0/147647084 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8043020 con 0x7f23e8101ab0 2024-04-02T14:47:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.946+0000 7f23dea00640 1 -- 172.21.15.62:0/147647084 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23d8004e50 con 0x7f23e8101ab0 2024-04-02T14:47:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.946+0000 7f23ece00640 1 -- 172.21.15.62:0/147647084 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e81a6b20 con 0x7f23e8101ab0 2024-04-02T14:47:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.946+0000 7f23dea00640 1 -- 172.21.15.62:0/147647084 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8003850 con 0x7f23e8101ab0 2024-04-02T14:47:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.946+0000 7f23ece00640 1 -- 172.21.15.62:0/147647084 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e81a70c0 con 0x7f23e8101ab0 2024-04-02T14:47:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.949+0000 7f23dea00640 1 -- 172.21.15.62:0/147647084 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f23d8046020 con 0x7f23e8101ab0 2024-04-02T14:47:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.950+0000 7f23efba6740 1 Processor -- start 2024-04-02T14:47:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.950+0000 7f23efba6740 1 -- start start 2024-04-02T14:47:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.950+0000 7f23efba6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 0x5556eca8e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.950+0000 7f23efba6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8e7d0 0x5556eca96e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23efba6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca973b0 0x5556eca997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23efba6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5556ec9f2d70 con 0x5556eca8e7d0 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23efba6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5556ec99b2a0 con 0x5556eca973b0 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23efba6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5556ec9eed20 con 0x5556eca8deb0 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8e7d0 0x5556eca96e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23e6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 0x5556eca8e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23dfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca973b0 0x5556eca997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23e6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 0x5556eca8e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:39728/0 (socket says 172.21.15.62:39728) 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23dfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca973b0 0x5556eca997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:44706/0 (socket says 172.21.15.62:44706) 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23e6800640 1 -- 172.21.15.62:0/3006383870 learned_addr learned my addr 172.21.15.62:0/3006383870 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23e6800640 1 -- 172.21.15.62:0/3006383870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca973b0 msgr2=0x5556eca997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23e6800640 1 --2- 172.21.15.62:0/3006383870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca973b0 0x5556eca997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23e6800640 1 -- 172.21.15.62:0/3006383870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8e7d0 msgr2=0x5556eca96e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23e6800640 1 --2- 172.21.15.62:0/3006383870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8e7d0 0x5556eca96e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23dea00640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23cc0a3e70 0x7f23cc0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.951+0000 7f23e6800640 1 -- 172.21.15.62:0/3006383870 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ec9af840 con 0x5556eca8deb0 2024-04-02T14:47:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.952+0000 7f23dea00640 1 -- 172.21.15.62:0/147647084 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f23d80f25c0 con 0x7f23e8101ab0 2024-04-02T14:47:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.952+0000 7f23e6800640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23cc0a3e70 0x7f23cc0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.952+0000 7f23e6800640 1 --2- 172.21.15.62:0/3006383870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 0x5556eca8e290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009f60 tx=0x7f23e0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e40679fd3a586fe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.952+0000 7f23d7400640 1 -- 172.21.15.62:0/3006383870 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0004250 con 0x5556eca8deb0 2024-04-02T14:47:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.952+0000 7f23d7400640 1 -- 172.21.15.62:0/3006383870 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23e0003e50 con 0x5556eca8deb0 2024-04-02T14:47:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.952+0000 7f23d7400640 1 -- 172.21.15.62:0/3006383870 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0019610 con 0x5556eca8deb0 2024-04-02T14:47:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.953+0000 7f23e6800640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23cc0a3e70 0x7f23cc0a6330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f23e0020000 tx=0x7f23e0019c60 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.953+0000 7f23efba6740 1 -- 172.21.15.62:0/3006383870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 msgr2=0x5556eca8e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.953+0000 7f23efba6740 1 --2- 172.21.15.62:0/3006383870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 0x5556eca8e290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009f60 tx=0x7f23e0009f90 comp rx=0 tx=0).stop 2024-04-02T14:47:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.953+0000 7f23efba6740 1 -- 172.21.15.62:0/3006383870 shutdown_connections 2024-04-02T14:47:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.953+0000 7f23efba6740 1 --2- 172.21.15.62:0/3006383870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca973b0 0x5556eca997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.953+0000 7f23efba6740 1 --2- 172.21.15.62:0/3006383870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8e7d0 0x5556eca96e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.953+0000 7f23efba6740 1 --2- 172.21.15.62:0/3006383870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 0x5556eca8e290 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.953+0000 7f23efba6740 1 -- 172.21.15.62:0/3006383870 >> 172.21.15.62:0/3006383870 conn(0x5556eca885e0 msgr2=0x5556eca889c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.953+0000 7f23efba6740 1 -- 172.21.15.62:0/3006383870 shutdown_connections 2024-04-02T14:47:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.953+0000 7f23efba6740 1 -- 172.21.15.62:0/3006383870 wait complete. 2024-04-02T14:47:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.954+0000 7f23efba6740 1 -- start start 2024-04-02T14:47:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.954+0000 7f23efba6740 1 Processor -- start 2024-04-02T14:47:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23efba6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8deb0 0x5556eca94430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23efba6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca94970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23efba6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca973b0 0x5556eca94eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23efba6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5556eca9a1a0 con 0x5556eca8deb0 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23efba6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5556ec9ae660 con 0x5556eca8e7d0 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23efba6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5556eca9a320 con 0x5556eca973b0 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23efba6740 10 client.0 _ll_register_callbacks cb 0x5556ec9ae050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23dfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca94970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23dfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca94970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:44712/0 (socket says 172.21.15.62:44712) 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8deb0 0x5556eca94430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23dfe00640 1 -- 172.21.15.62:0/2772477465 learned_addr learned my addr 172.21.15.62:0/2772477465 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23e6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca973b0 0x5556eca94eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23dfe00640 1 -- 172.21.15.62:0/2772477465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca973b0 msgr2=0x5556eca94eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23dfe00640 1 --2- 172.21.15.62:0/2772477465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca973b0 0x5556eca94eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23dfe00640 1 -- 172.21.15.62:0/2772477465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8deb0 msgr2=0x5556eca94430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23dfe00640 1 --2- 172.21.15.62:0/2772477465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8deb0 0x5556eca94430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.955+0000 7f23dfe00640 1 -- 172.21.15.62:0/2772477465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ec9bde80 con 0x5556eca8e7d0 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.956+0000 7f23dfe00640 1 --2- 172.21.15.62:0/2772477465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca94970 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f23d0012750 tx=0x7f23d0012c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c38d72dde8bf15 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.956+0000 7f23cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.956+0000 7f23cbe00640 1 -- 172.21.15.62:0/2772477465 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0012e60 con 0x5556eca8e7d0 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.956+0000 7f23efba6740 1 -- 172.21.15.62:0/2772477465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e0003e50 con 0x5556eca8e7d0 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.956+0000 7f23cbe00640 1 -- 172.21.15.62:0/2772477465 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f23d000f8a0 con 0x5556eca8e7d0 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.956+0000 7f23cbe00640 1 -- 172.21.15.62:0/2772477465 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f23d000faa0 con 0x5556eca8e7d0 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.956+0000 7f23efba6740 10 client.26489 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.956+0000 7f23efba6740 1 -- 172.21.15.62:0/2772477465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5556ec707210 con 0x5556eca8e7d0 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.957+0000 7f23cbe00640 1 -- 172.21.15.62:0/2772477465 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23e0003e50 con 0x5556eca8e7d0 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.957+0000 7f23cbe00640 1 -- 172.21.15.62:0/2772477465 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f23d001c420 con 0x5556eca8e7d0 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.957+0000 7f23efba6740 10 client.26489 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.957+0000 7f23efba6740 10 client.26489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.957+0000 7f23efba6740 20 client.26489 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.957+0000 7f23efba6740 1 --2- 172.21.15.62:0/2772477465 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5556ecaa78f0 0x5556ecaa9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.957+0000 7f23efba6740 4 client.26489 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:47:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.957+0000 7f23efba6740 1 -- 172.21.15.62:0/2772477465 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556ec9a2cd0 con 0x5556ecaa78f0 2024-04-02T14:47:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.957+0000 7f23e5e00640 1 --2- 172.21.15.62:0/2772477465 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5556ecaa78f0 0x5556ecaa9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.958+0000 7f23e5e00640 1 --2- 172.21.15.62:0/2772477465 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5556ecaa78f0 0x5556ecaa9db0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e36bad5bd6ff1b1 server_cookie=34e0dbeb5989ce1d in_seq=0 out_seq=0 2024-04-02T14:47:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.958+0000 7f23cbe00640 10 client.26489 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23cbe00640 1 -- 172.21.15.62:0/2772477465 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556ec9a2cd0 con 0x5556ecaa78f0 2024-04-02T14:47:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23cbe00640 10 client.26489 handle_command_reply: tid=0 2024-04-02T14:47:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 1 client.26489 shutdown 2024-04-02T14:47:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 1 -- 172.21.15.62:0/2772477465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 msgr2=0x5556eca94970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 1 --2- 172.21.15.62:0/2772477465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca94970 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f23d0012750 tx=0x7f23d0012c10 comp rx=0 tx=0).stop 2024-04-02T14:47:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 1 -- 172.21.15.62:0/2772477465 shutdown_connections 2024-04-02T14:47:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 1 --2- 172.21.15.62:0/2772477465 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5556ecaa78f0 0x5556ecaa9db0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 1 --2- 172.21.15.62:0/2772477465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca973b0 0x5556eca94eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 1 --2- 172.21.15.62:0/2772477465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca94970 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 1 --2- 172.21.15.62:0/2772477465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8deb0 0x5556eca94430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 1 -- 172.21.15.62:0/2772477465 >> 172.21.15.62:0/2772477465 conn(0x5556eca885e0 msgr2=0x5556eca8a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23cbe00640 0 client.26489 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 1 -- 172.21.15.62:0/2772477465 shutdown_connections 2024-04-02T14:47:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 1 -- 172.21.15.62:0/2772477465 wait complete. 2024-04-02T14:47:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.959+0000 7f23efba6740 20 client.26489 trim_cache size 0 max 16384 2024-04-02T14:47:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.977+0000 7f23efba6740 1 Processor -- start 2024-04-02T14:47:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23efba6740 1 -- start start 2024-04-02T14:47:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23efba6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 0x5556eca96b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23efba6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556ecaa3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23efba6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca973b0 0x5556eca93590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23efba6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81af830 con 0x5556eca973b0 2024-04-02T14:47:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23efba6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b1520 con 0x5556eca8e7d0 2024-04-02T14:47:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23efba6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81aff60 con 0x5556eca8deb0 2024-04-02T14:47:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23e6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca973b0 0x5556eca93590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23dfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556ecaa3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23e6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca973b0 0x5556eca93590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37836/0 (socket says 172.21.15.62:37836) 2024-04-02T14:47:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23e6800640 1 -- 172.21.15.62:0/4193988314 learned_addr learned my addr 172.21.15.62:0/4193988314 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.978+0000 7f23e5e00640 1 --2- 172.21.15.62:0/4193988314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 0x5556eca96b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.979+0000 7f23dfe00640 1 -- 172.21.15.62:0/4193988314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 msgr2=0x5556eca96b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.979+0000 7f23dfe00640 1 --2- 172.21.15.62:0/4193988314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 0x5556eca96b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.979+0000 7f23dfe00640 1 -- 172.21.15.62:0/4193988314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca973b0 msgr2=0x5556eca93590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.979+0000 7f23dfe00640 1 --2- 172.21.15.62:0/4193988314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca973b0 0x5556eca93590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.979+0000 7f23dfe00640 1 -- 172.21.15.62:0/4193988314 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556eca8e600 con 0x5556eca8e7d0 2024-04-02T14:47:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.979+0000 7f23dfe00640 1 --2- 172.21.15.62:0/4193988314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556ecaa3060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f23d001c910 tx=0x7f23d000fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54875caa338d6617 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.979+0000 7f23d6a00640 1 -- 172.21.15.62:0/4193988314 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0029030 con 0x5556eca8e7d0 2024-04-02T14:47:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.979+0000 7f23d6a00640 1 -- 172.21.15.62:0/4193988314 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23d0005570 con 0x5556eca8e7d0 2024-04-02T14:47:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.979+0000 7f23d6a00640 1 -- 172.21.15.62:0/4193988314 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0012e60 con 0x5556eca8e7d0 2024-04-02T14:47:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.980+0000 7f23efba6740 1 -- 172.21.15.62:0/4193988314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 msgr2=0x5556ecaa3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.980+0000 7f23efba6740 1 --2- 172.21.15.62:0/4193988314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556ecaa3060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f23d001c910 tx=0x7f23d000fa30 comp rx=0 tx=0).stop 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.980+0000 7f23efba6740 1 -- 172.21.15.62:0/4193988314 shutdown_connections 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.980+0000 7f23efba6740 1 --2- 172.21.15.62:0/4193988314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca973b0 0x5556eca93590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.980+0000 7f23efba6740 1 --2- 172.21.15.62:0/4193988314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556ecaa3060 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.980+0000 7f23efba6740 1 --2- 172.21.15.62:0/4193988314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca8deb0 0x5556eca96b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.980+0000 7f23efba6740 1 -- 172.21.15.62:0/4193988314 >> 172.21.15.62:0/4193988314 conn(0x5556eca885e0 msgr2=0x5556eca86db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.980+0000 7f23efba6740 1 -- 172.21.15.62:0/4193988314 shutdown_connections 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.980+0000 7f23efba6740 1 -- 172.21.15.62:0/4193988314 wait complete. 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.980+0000 7f23efba6740 1 -- start start 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.980+0000 7f23efba6740 1 Processor -- start 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23efba6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8deb0 0x5556eca9f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23efba6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca90c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23efba6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca973b0 0x5556eca911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23efba6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b0d90 con 0x5556eca8deb0 2024-04-02T14:47:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23efba6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b0660 con 0x5556eca8e7d0 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23efba6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81ad810 con 0x5556eca973b0 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23efba6740 10 client.0 _ll_register_callbacks cb 0x7f23d0009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23dfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca90c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8deb0 0x5556eca9f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23dfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca90c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:44730/0 (socket says 172.21.15.62:44730) 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23dfe00640 1 -- 172.21.15.62:0/2560850690 learned_addr learned my addr 172.21.15.62:0/2560850690 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.981+0000 7f23e6800640 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca973b0 0x5556eca911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23dfe00640 1 -- 172.21.15.62:0/2560850690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca973b0 msgr2=0x5556eca911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23dfe00640 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca973b0 0x5556eca911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23dfe00640 1 -- 172.21.15.62:0/2560850690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8deb0 msgr2=0x5556eca9f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23dfe00640 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8deb0 0x5556eca9f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23dfe00640 1 -- 172.21.15.62:0/2560850690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ec7d3620 con 0x5556eca8e7d0 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23e5e00640 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8deb0 0x5556eca9f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23dfe00640 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca90c80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f23d000fdc0 tx=0x7f23d0009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb386ed55f0b0da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-02T14:47:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23cbe00640 1 -- 172.21.15.62:0/2560850690 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0031020 con 0x5556eca8e7d0 2024-04-02T14:47:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23cbe00640 1 -- 172.21.15.62:0/2560850690 <== mon.2 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f23d001ca80 con 0x5556eca8e7d0 2024-04-02T14:47:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23efba6740 1 -- 172.21.15.62:0/2560850690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23d0005570 con 0x5556eca8e7d0 2024-04-02T14:47:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.982+0000 7f23cbe00640 1 -- 172.21.15.62:0/2560850690 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f23d003c050 con 0x5556eca8e7d0 2024-04-02T14:47:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.983+0000 7f23efba6740 10 client.26495 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.983+0000 7f23efba6740 1 -- 172.21.15.62:0/2560850690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5556ecb0dac0 con 0x5556eca8e7d0 2024-04-02T14:47:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.983+0000 7f23cbe00640 1 -- 172.21.15.62:0/2560850690 <== mon.2 v2:172.21.15.113:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23d0005570 con 0x5556eca8e7d0 2024-04-02T14:47:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.983+0000 7f23cbe00640 1 -- 172.21.15.62:0/2560850690 <== mon.2 v2:172.21.15.113:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f23d0003b20 con 0x5556eca8e7d0 2024-04-02T14:47:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.983+0000 7f23efba6740 10 client.26495 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.983+0000 7f23efba6740 10 client.26495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T14:47:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.983+0000 7f23efba6740 20 client.26495 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.983+0000 7f23efba6740 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5556ecb0e1e0 0x5556ecb10600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.983+0000 7f23efba6740 4 client.26495 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T14:47:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.983+0000 7f23efba6740 1 -- 172.21.15.62:0/2560850690 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556eca002b0 con 0x5556ecb0e1e0 2024-04-02T14:47:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.983+0000 7f23e5e00640 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5556ecb0e1e0 0x5556ecb10600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.984+0000 7f23e5e00640 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5556ecb0e1e0 0x5556ecb10600 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4df6ad93c5bd94 server_cookie=b4615746f3cf55bd in_seq=0 out_seq=0 2024-04-02T14:47:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:14.984+0000 7f23cbe00640 10 client.26495 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.440+0000 7f23cbe00640 1 -- 172.21.15.62:0/2560850690 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5556eca002b0 con 0x5556ecb0e1e0 2024-04-02T14:47:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.440+0000 7f23cbe00640 10 client.26495 handle_command_reply: tid=0 2024-04-02T14:47:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.440+0000 7f23efba6740 1 client.26495 shutdown 2024-04-02T14:47:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.441+0000 7f23efba6740 1 -- 172.21.15.62:0/2560850690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 msgr2=0x5556eca90c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.441+0000 7f23efba6740 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca90c80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f23d000fdc0 tx=0x7f23d0009370 comp rx=0 tx=0).stop 2024-04-02T14:47:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.441+0000 7f23efba6740 1 -- 172.21.15.62:0/2560850690 shutdown_connections 2024-04-02T14:47:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.441+0000 7f23efba6740 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x5556ecb0e1e0 0x5556ecb10600 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.441+0000 7f23efba6740 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556eca973b0 0x5556eca911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.441+0000 7f23efba6740 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556eca8e7d0 0x5556eca90c80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.441+0000 7f23efba6740 1 --2- 172.21.15.62:0/2560850690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556eca8deb0 0x5556eca9f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.441+0000 7f23efba6740 1 -- 172.21.15.62:0/2560850690 >> 172.21.15.62:0/2560850690 conn(0x5556eca885e0 msgr2=0x5556eca8a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.441+0000 7f23cbe00640 0 client.26495 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.441+0000 7f23efba6740 1 -- 172.21.15.62:0/2560850690 shutdown_connections 2024-04-02T14:47:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.441+0000 7f23efba6740 1 -- 172.21.15.62:0/2560850690 wait complete. 2024-04-02T14:47:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-04-02T14:47:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-04-02T14:47:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-04-02T14:47:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-04-02T14:47:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.442+0000 7f23efba6740 20 client.26495 trim_cache size 0 max 16384 2024-04-02T14:47:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.444+0000 7f23ece00640 1 -- 172.21.15.62:0/147647084 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23cc0a3e70 msgr2=0x7f23cc0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.444+0000 7f23ece00640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23cc0a3e70 0x7f23cc0a6330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f23e0020000 tx=0x7f23e0019c60 comp rx=0 tx=0).stop 2024-04-02T14:47:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.444+0000 7f23ece00640 1 -- 172.21.15.62:0/147647084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23e8101ab0 msgr2=0x7f23e8073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.444+0000 7f23ece00640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23e8101ab0 0x7f23e8073e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005ab0 tx=0x7f23d802dc00 comp rx=0 tx=0).stop 2024-04-02T14:47:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.444+0000 7f23ece00640 1 -- 172.21.15.62:0/147647084 shutdown_connections 2024-04-02T14:47:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.444+0000 7f23ece00640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f23cc0a3e70 0x7f23cc0a6330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.444+0000 7f23ece00640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23e8107ac0 0x7f23e8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.444+0000 7f23ece00640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8102470 0x7f23e8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.444+0000 7f23ece00640 1 --2- 172.21.15.62:0/147647084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23e8101ab0 0x7f23e8073e00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.444+0000 7f23ece00640 1 -- 172.21.15.62:0/147647084 >> 172.21.15.62:0/147647084 conn(0x7f23e80fd5c0 msgr2=0x7f23e80fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.445+0000 7f23ece00640 1 -- 172.21.15.62:0/147647084 shutdown_connections 2024-04-02T14:47:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-04-02T14:47:15.445+0000 7f23ece00640 1 -- 172.21.15.62:0/147647084 wait complete. 2024-04-02T14:47:16.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:15 smithi204 ceph-mon[35948]: pgmap v100: 97 pgs: 97 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 569 op/s 2024-04-02T14:47:16.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:15 smithi062 ceph-mon[32116]: pgmap v100: 97 pgs: 97 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 569 op/s 2024-04-02T14:47:16.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:15 smithi113 ceph-mon[36135]: pgmap v100: 97 pgs: 97 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 569 op/s 2024-04-02T14:47:17.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:16 smithi204 ceph-mon[35948]: Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T14:47:17.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:16 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[32112]: 2024-04-02T14:47:16.603+0000 7f7747c00700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T14:47:17.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:16 smithi062 ceph-mon[32116]: Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T14:47:17.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:16 smithi113 ceph-mon[36135]: Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T14:47:18.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:17 smithi204 ceph-mon[35948]: pgmap v101: 97 pgs: 97 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 852 op/s 2024-04-02T14:47:18.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:17 smithi062 ceph-mon[32116]: pgmap v101: 97 pgs: 97 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 852 op/s 2024-04-02T14:47:18.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:17 smithi113 ceph-mon[36135]: pgmap v101: 97 pgs: 97 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 852 op/s 2024-04-02T14:47:19.216 INFO:tasks.workunit:Stopping ['fs/norstats'] on client.0... 2024-04-02T14:47:19.217 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-02T14:47:20.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:19 smithi204 ceph-mon[35948]: pgmap v102: 97 pgs: 97 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 553 op/s 2024-04-02T14:47:20.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:19 smithi062 ceph-mon[32116]: pgmap v102: 97 pgs: 97 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 553 op/s 2024-04-02T14:47:20.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:19 smithi113 ceph-mon[36135]: pgmap v102: 97 pgs: 97 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 553 op/s 2024-04-02T14:47:20.358 DEBUG:teuthology.parallel:result is None 2024-04-02T14:47:20.359 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-02T14:47:20.463 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-02T14:47:20.464 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-02T14:47:20.477 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi062.front.sepia.ceph.com 2024-04-02T14:47:20.502 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-02T14:47:20.512 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-02T14:47:20.512 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T14:47:20.512 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-04-02T14:47:20.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-02T14:47:20.653 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-02T14:47:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:21 smithi204 ceph-mon[35948]: pgmap v103: 97 pgs: 97 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 460 op/s 2024-04-02T14:47:22.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:21 smithi062 ceph-mon[32116]: pgmap v103: 97 pgs: 97 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 460 op/s 2024-04-02T14:47:22.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:21 smithi113 ceph-mon[36135]: pgmap v103: 97 pgs: 97 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 460 op/s 2024-04-02T14:47:24.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:23 smithi204 ceph-mon[35948]: pgmap v104: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 564 op/s 2024-04-02T14:47:24.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:23 smithi062 ceph-mon[32116]: pgmap v104: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 564 op/s 2024-04-02T14:47:24.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:23 smithi113 ceph-mon[36135]: pgmap v104: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 564 op/s 2024-04-02T14:47:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:25 smithi204 ceph-mon[35948]: pgmap v105: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-04-02T14:47:26.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:25 smithi062 ceph-mon[32116]: pgmap v105: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-04-02T14:47:26.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:25 smithi113 ceph-mon[36135]: pgmap v105: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-04-02T14:47:28.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:27 smithi204 ceph-mon[35948]: pgmap v106: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 440 op/s 2024-04-02T14:47:28.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:27 smithi062 ceph-mon[32116]: pgmap v106: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 440 op/s 2024-04-02T14:47:28.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:27 smithi113 ceph-mon[36135]: pgmap v106: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 440 op/s 2024-04-02T14:47:30.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:29 smithi204 ceph-mon[35948]: pgmap v107: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 451 KiB/s wr, 157 op/s 2024-04-02T14:47:30.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:29 smithi062 ceph-mon[32116]: pgmap v107: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 451 KiB/s wr, 157 op/s 2024-04-02T14:47:30.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:29 smithi113 ceph-mon[36135]: pgmap v107: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 451 KiB/s wr, 157 op/s 2024-04-02T14:47:32.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:31 smithi204 ceph-mon[35948]: pgmap v108: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 446 KiB/s wr, 113 op/s 2024-04-02T14:47:32.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:31 smithi062 ceph-mon[32116]: pgmap v108: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 446 KiB/s wr, 113 op/s 2024-04-02T14:47:32.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:31 smithi113 ceph-mon[36135]: pgmap v108: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 446 KiB/s wr, 113 op/s 2024-04-02T14:47:33.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:47:33.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:47:33.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.696+0000 7f8838000640 1 -- 172.21.15.62:0/1334528047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830076030 msgr2=0x7f8830076410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.696+0000 7f8838000640 1 --2- 172.21.15.62:0/1334528047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830076030 0x7f8830076410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8824009930 tx=0x7f882402bc30 comp rx=0 tx=0).stop 2024-04-02T14:47:33.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.696+0000 7f8838000640 1 -- 172.21.15.62:0/1334528047 shutdown_connections 2024-04-02T14:47:33.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.696+0000 7f8838000640 1 --2- 172.21.15.62:0/1334528047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8830112940 0x7f8830114d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.696+0000 7f8838000640 1 --2- 172.21.15.62:0/1334528047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8830076950 0x7f8830112400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.696+0000 7f8838000640 1 --2- 172.21.15.62:0/1334528047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830076030 0x7f8830076410 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.696+0000 7f8838000640 1 -- 172.21.15.62:0/1334528047 >> 172.21.15.62:0/1334528047 conn(0x7f88300ff770 msgr2=0x7f8830101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:33.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.696+0000 7f8838000640 1 -- 172.21.15.62:0/1334528047 shutdown_connections 2024-04-02T14:47:33.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.696+0000 7f8838000640 1 -- 172.21.15.62:0/1334528047 wait complete. 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.698+0000 7f8838000640 1 Processor -- start 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.698+0000 7f8838000640 1 -- start start 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.698+0000 7f8838000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8830076030 0x7f88301a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.698+0000 7f8838000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8830076950 0x7f88301a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.698+0000 7f8838000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830112940 0x7f883019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.698+0000 7f8838000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8830118570 con 0x7f8830076030 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.698+0000 7f8838000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f88301183f0 con 0x7f8830112940 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.698+0000 7f8838000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88301186f0 con 0x7f8830076950 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8835600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8830076030 0x7f88301a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8835600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8830076030 0x7f88301a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39994/0 (socket says 172.21.15.62:39994) 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8835600640 1 -- 172.21.15.62:0/3527255188 learned_addr learned my addr 172.21.15.62:0/3527255188 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8836000640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830112940 0x7f883019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8834c00640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8830076950 0x7f88301a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8836000640 1 -- 172.21.15.62:0/3527255188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8830076950 msgr2=0x7f88301a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8836000640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8830076950 0x7f88301a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8836000640 1 -- 172.21.15.62:0/3527255188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8830076030 msgr2=0x7f88301a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8836000640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8830076030 0x7f88301a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8836000640 1 -- 172.21.15.62:0/3527255188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883019f090 con 0x7f8830112940 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8835600640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8830076030 0x7f88301a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.699+0000 7f8834c00640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8830076950 0x7f88301a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.700+0000 7f8836000640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830112940 0x7f883019e910 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8828005fd0 tx=0x7f882800dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.700+0000 7f882e000640 1 -- 172.21.15.62:0/3527255188 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882800bd40 con 0x7f8830112940 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.700+0000 7f882e000640 1 -- 172.21.15.62:0/3527255188 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8828004030 con 0x7f8830112940 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.700+0000 7f8838000640 1 -- 172.21.15.62:0/3527255188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883019f380 con 0x7f8830112940 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.700+0000 7f882e000640 1 -- 172.21.15.62:0/3527255188 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828010790 con 0x7f8830112940 2024-04-02T14:47:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.700+0000 7f8838000640 1 -- 172.21.15.62:0/3527255188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88301ab620 con 0x7f8830112940 2024-04-02T14:47:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.704+0000 7f883adb5740 1 Processor -- start 2024-04-02T14:47:33.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.705+0000 7f882e000640 1 -- 172.21.15.62:0/3527255188 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f88280041e0 con 0x7f8830112940 2024-04-02T14:47:33.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.705+0000 7f883adb5740 1 -- start start 2024-04-02T14:47:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.705+0000 7f883adb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 0x561cbbee2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.705+0000 7f883adb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee30d0 0x561cbbeeb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f883adb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbeee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f883adb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561cbbe46770 con 0x561cbbee30d0 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f883adb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561cbbdee700 con 0x561cbbee27b0 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f883adb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561cbbe42720 con 0x561cbbeebcb0 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f8835600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee30d0 0x561cbbeeb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f8834c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 0x561cbbee2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f8834c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 0x561cbbee2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37972/0 (socket says 172.21.15.62:37972) 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f8835600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee30d0 0x561cbbeeb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39998/0 (socket says 172.21.15.62:39998) 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f8834c00640 1 -- 172.21.15.62:0/2368872559 learned_addr learned my addr 172.21.15.62:0/2368872559 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f8834c00640 1 --2- 172.21.15.62:0/2368872559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbeee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f8835600640 1 -- 172.21.15.62:0/2368872559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 msgr2=0x561cbbeee0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f8835600640 1 --2- 172.21.15.62:0/2368872559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbeee0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f8835600640 1 -- 172.21.15.62:0/2368872559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 msgr2=0x561cbbee2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f8835600640 1 --2- 172.21.15.62:0/2368872559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 0x561cbbee2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.706+0000 7f8835600640 1 -- 172.21.15.62:0/2368872559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cbbc247d0 con 0x561cbbee30d0 2024-04-02T14:47:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.707+0000 7f8835600640 1 --2- 172.21.15.62:0/2368872559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee30d0 0x561cbbeeb770 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f883019fa50 tx=0x7f882402d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb4cfdea6a3f715 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.707+0000 7f882e000640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88180a3da0 0x7f88180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.707+0000 7f8822600640 1 -- 172.21.15.62:0/2368872559 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824004040 con 0x561cbbee30d0 2024-04-02T14:47:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.707+0000 7f8822600640 1 -- 172.21.15.62:0/2368872559 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8824005540 con 0x561cbbee30d0 2024-04-02T14:47:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.707+0000 7f8822600640 1 -- 172.21.15.62:0/2368872559 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824004c40 con 0x561cbbee30d0 2024-04-02T14:47:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.707+0000 7f882e000640 1 -- 172.21.15.62:0/3527255188 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f88280d0b10 con 0x7f8830112940 2024-04-02T14:47:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.707+0000 7f8834c00640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88180a3da0 0x7f88180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.708+0000 7f883adb5740 1 -- 172.21.15.62:0/2368872559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee30d0 msgr2=0x561cbbeeb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.708+0000 7f8834c00640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88180a3da0 0x7f88180a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x561cbbede070 tx=0x7f881c009700 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.708+0000 7f883adb5740 1 --2- 172.21.15.62:0/2368872559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee30d0 0x561cbbeeb770 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f883019fa50 tx=0x7f882402d9e0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.708+0000 7f883adb5740 1 -- 172.21.15.62:0/2368872559 shutdown_connections 2024-04-02T14:47:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.708+0000 7f883adb5740 1 --2- 172.21.15.62:0/2368872559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbeee0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.708+0000 7f883adb5740 1 --2- 172.21.15.62:0/2368872559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee30d0 0x561cbbeeb770 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.708+0000 7f883adb5740 1 --2- 172.21.15.62:0/2368872559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 0x561cbbee2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.708+0000 7f883adb5740 1 -- 172.21.15.62:0/2368872559 >> 172.21.15.62:0/2368872559 conn(0x561cbbedcee0 msgr2=0x561cbbedd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.708+0000 7f883adb5740 1 -- 172.21.15.62:0/2368872559 shutdown_connections 2024-04-02T14:47:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.708+0000 7f883adb5740 1 -- 172.21.15.62:0/2368872559 wait complete. 2024-04-02T14:47:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.709+0000 7f883adb5740 1 -- start start 2024-04-02T14:47:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.709+0000 7f883adb5740 1 Processor -- start 2024-04-02T14:47:33.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f883adb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 0x561cbbee8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f883adb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbee30d0 0x561cbbee9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f883adb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbeebcb0 0x561cbbee97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f883adb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561cbbeeea40 con 0x561cbbeebcb0 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f883adb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561cbbe01ac0 con 0x561cbbee27b0 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f883adb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561cbbeeebc0 con 0x561cbbee30d0 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f883adb5740 10 client.0 _ll_register_callbacks cb 0x561cbbe04180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f8835600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 0x561cbbee8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f8836000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbeebcb0 0x561cbbee97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f8835600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 0x561cbbee8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37976/0 (socket says 172.21.15.62:37976) 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f8835600640 1 -- 172.21.15.62:0/4239973171 learned_addr learned my addr 172.21.15.62:0/4239973171 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f8834c00640 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbee30d0 0x561cbbee9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f8836000640 1 -- 172.21.15.62:0/4239973171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbee30d0 msgr2=0x561cbbee9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f8836000640 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbee30d0 0x561cbbee9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f8836000640 1 -- 172.21.15.62:0/4239973171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 msgr2=0x561cbbee8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f8836000640 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 0x561cbbee8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f8836000640 1 -- 172.21.15.62:0/4239973171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cbbb5a3f0 con 0x561cbbeebcb0 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.710+0000 7f8835600640 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 0x561cbbee8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.711+0000 7f8836000640 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbeebcb0 0x561cbbee97c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f88280ce290 tx=0x7f882809c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8260c5433529e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.711+0000 7f8816a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.711+0000 7f8816a00640 1 -- 172.21.15.62:0/4239973171 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88280b6060 con 0x561cbbeebcb0 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.711+0000 7f8816a00640 1 -- 172.21.15.62:0/4239973171 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8828099150 con 0x561cbbeebcb0 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.711+0000 7f883adb5740 1 -- 172.21.15.62:0/4239973171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8824005540 con 0x561cbbeebcb0 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.711+0000 7f8816a00640 1 -- 172.21.15.62:0/4239973171 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f882809cae0 con 0x561cbbeebcb0 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.711+0000 7f883adb5740 10 client.16371 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.711+0000 7f883adb5740 1 -- 172.21.15.62:0/4239973171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561cbbb1a0f0 con 0x561cbbeebcb0 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.711+0000 7f8816a00640 1 -- 172.21.15.62:0/4239973171 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8824005540 con 0x561cbbeebcb0 2024-04-02T14:47:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.712+0000 7f8816a00640 1 -- 172.21.15.62:0/4239973171 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f8828024060 con 0x561cbbeebcb0 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.712+0000 7f883adb5740 10 client.16371 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.712+0000 7f883adb5740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.712+0000 7f883adb5740 20 client.16371 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.712+0000 7f883adb5740 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x561cbbefbda0 0x561cbbefe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.712+0000 7f883adb5740 4 client.16371 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.712+0000 7f883adb5740 1 -- 172.21.15.62:0/4239973171 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cbbde7630 con 0x561cbbefbda0 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.712+0000 7f8835600640 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x561cbbefbda0 0x561cbbefe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.713+0000 7f8835600640 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x561cbbefbda0 0x561cbbefe240 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2342101f7f596f12 server_cookie=9c123ac6f15b3e28 in_seq=0 out_seq=0 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.713+0000 7f8816a00640 10 client.16371 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f8816a00640 1 -- 172.21.15.62:0/4239973171 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561cbbde7630 con 0x561cbbefbda0 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f8816a00640 10 client.16371 handle_command_reply: tid=0 2024-04-02T14:47:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 1 client.16371 shutdown 2024-04-02T14:47:33.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 1 -- 172.21.15.62:0/4239973171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbeebcb0 msgr2=0x561cbbee97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbeebcb0 0x561cbbee97c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f88280ce290 tx=0x7f882809c500 comp rx=0 tx=0).stop 2024-04-02T14:47:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 1 -- 172.21.15.62:0/4239973171 shutdown_connections 2024-04-02T14:47:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x561cbbefbda0 0x561cbbefe240 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbeebcb0 0x561cbbee97c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f8816a00640 0 client.16371 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbee30d0 0x561cbbee9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 1 --2- 172.21.15.62:0/4239973171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee27b0 0x561cbbee8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 1 -- 172.21.15.62:0/4239973171 >> 172.21.15.62:0/4239973171 conn(0x561cbbedcee0 msgr2=0x561cbbedec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 1 -- 172.21.15.62:0/4239973171 shutdown_connections 2024-04-02T14:47:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 1 -- 172.21.15.62:0/4239973171 wait complete. 2024-04-02T14:47:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.714+0000 7f883adb5740 20 client.16371 trim_cache size 0 max 16384 2024-04-02T14:47:33.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f883adb5740 1 Processor -- start 2024-04-02T14:47:33.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f883adb5740 1 -- start start 2024-04-02T14:47:33.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f883adb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 0x561cbbeeb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f883adb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 0x561cbbef7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f883adb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbedbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f883adb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88301b3d10 con 0x561cbbee27b0 2024-04-02T14:47:33.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f883adb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f88301b5a00 con 0x561cbbee30d0 2024-04-02T14:47:33.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f883adb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88301b4440 con 0x561cbbeebcb0 2024-04-02T14:47:33.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f8835600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 0x561cbbeeb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f8834c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 0x561cbbef7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f8836000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbedbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f8834c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 0x561cbbef7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:37988/0 (socket says 172.21.15.62:37988) 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.734+0000 7f8834c00640 1 -- 172.21.15.62:0/3661321642 learned_addr learned my addr 172.21.15.62:0/3661321642 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.735+0000 7f8834c00640 1 -- 172.21.15.62:0/3661321642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 msgr2=0x561cbbedbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.735+0000 7f8834c00640 1 --2- 172.21.15.62:0/3661321642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbedbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.735+0000 7f8834c00640 1 -- 172.21.15.62:0/3661321642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 msgr2=0x561cbbeeb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.735+0000 7f8834c00640 1 --2- 172.21.15.62:0/3661321642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 0x561cbbeeb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.735+0000 7f8834c00640 1 -- 172.21.15.62:0/3661321642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cbbedca10 con 0x561cbbee30d0 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.735+0000 7f8835600640 1 --2- 172.21.15.62:0/3661321642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 0x561cbbeeb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.735+0000 7f8834c00640 1 --2- 172.21.15.62:0/3661321642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 0x561cbbef7900 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f881c010980 tx=0x7f881c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae7aa600b268fdce server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.735+0000 7f8821c00640 1 -- 172.21.15.62:0/3661321642 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c004110 con 0x561cbbee30d0 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.736+0000 7f8821c00640 1 -- 172.21.15.62:0/3661321642 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f881c0042b0 con 0x561cbbee30d0 2024-04-02T14:47:33.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.736+0000 7f8821c00640 1 -- 172.21.15.62:0/3661321642 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c022830 con 0x561cbbee30d0 2024-04-02T14:47:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.736+0000 7f883adb5740 1 -- 172.21.15.62:0/3661321642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 msgr2=0x561cbbef7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.736+0000 7f883adb5740 1 --2- 172.21.15.62:0/3661321642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 0x561cbbef7900 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f881c010980 tx=0x7f881c0046c0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.736+0000 7f883adb5740 1 -- 172.21.15.62:0/3661321642 shutdown_connections 2024-04-02T14:47:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.736+0000 7f883adb5740 1 --2- 172.21.15.62:0/3661321642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbedbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.736+0000 7f883adb5740 1 --2- 172.21.15.62:0/3661321642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 0x561cbbef7900 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.736+0000 7f883adb5740 1 --2- 172.21.15.62:0/3661321642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 0x561cbbeeb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.736+0000 7f883adb5740 1 -- 172.21.15.62:0/3661321642 >> 172.21.15.62:0/3661321642 conn(0x561cbbedcee0 msgr2=0x561cbbee74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.736+0000 7f883adb5740 1 -- 172.21.15.62:0/3661321642 shutdown_connections 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.736+0000 7f883adb5740 1 -- 172.21.15.62:0/3661321642 wait complete. 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.737+0000 7f883adb5740 1 -- start start 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.737+0000 7f883adb5740 1 Processor -- start 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.737+0000 7f883adb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 0x561cbbef3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f883adb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 0x561cbbeef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f883adb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbeef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f883adb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88301b5270 con 0x561cbbee27b0 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f883adb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f88301b4b40 con 0x561cbbee30d0 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f883adb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88301b1cf0 con 0x561cbbeebcb0 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f883adb5740 10 client.0 _ll_register_callbacks cb 0x7f88280092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f8835600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 0x561cbbef3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f8834c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 0x561cbbeef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f8835600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 0x561cbbef3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40034/0 (socket says 172.21.15.62:40034) 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f8835600640 1 -- 172.21.15.62:0/1971852825 learned_addr learned my addr 172.21.15.62:0/1971852825 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f8836000640 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbeef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f8835600640 1 -- 172.21.15.62:0/1971852825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 msgr2=0x561cbbeef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f8835600640 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbeef8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f8835600640 1 -- 172.21.15.62:0/1971852825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 msgr2=0x561cbbeef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f8835600640 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 0x561cbbeef3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.738+0000 7f8835600640 1 -- 172.21.15.62:0/1971852825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cbbeefe30 con 0x561cbbee27b0 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.739+0000 7f8835600640 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 0x561cbbef3ed0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8824005e50 tx=0x7f8824005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af770e26a4aa8bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.739+0000 7f8816a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.739+0000 7f8816a00640 1 -- 172.21.15.62:0/1971852825 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882402dad0 con 0x561cbbee27b0 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.739+0000 7f8816a00640 1 -- 172.21.15.62:0/1971852825 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f8824003800 con 0x561cbbee27b0 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.739+0000 7f883adb5740 1 -- 172.21.15.62:0/1971852825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f881c0042b0 con 0x561cbbee27b0 2024-04-02T14:47:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.739+0000 7f8816a00640 1 -- 172.21.15.62:0/1971852825 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8824003a00 con 0x561cbbee27b0 2024-04-02T14:47:33.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.739+0000 7f883adb5740 10 client.16377 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:33.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.739+0000 7f883adb5740 1 -- 172.21.15.62:0/1971852825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561cbbf620a0 con 0x561cbbee27b0 2024-04-02T14:47:33.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.740+0000 7f8816a00640 1 -- 172.21.15.62:0/1971852825 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f881c0042b0 con 0x561cbbee27b0 2024-04-02T14:47:33.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.740+0000 7f8816a00640 1 -- 172.21.15.62:0/1971852825 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f8824041480 con 0x561cbbee27b0 2024-04-02T14:47:33.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.740+0000 7f883adb5740 10 client.16377 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.740+0000 7f883adb5740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.740+0000 7f883adb5740 20 client.16377 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.740+0000 7f883adb5740 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x561cbbf624f0 0x561cbbf648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.740+0000 7f883adb5740 4 client.16377 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.740+0000 7f883adb5740 1 -- 172.21.15.62:0/1971852825 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561cbbe53cb0 con 0x561cbbf624f0 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.740+0000 7f8834c00640 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x561cbbf624f0 0x561cbbf648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f8834c00640 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x561cbbf624f0 0x561cbbf648e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8980b4ca1bb9f8fa server_cookie=f7a3048900753ac0 in_seq=0 out_seq=0 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f8816a00640 10 client.16377 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f8816a00640 1 -- 172.21.15.62:0/1971852825 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561cbbe53cb0 con 0x561cbbf624f0 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f8816a00640 10 client.16377 handle_command_reply: tid=0 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f883adb5740 1 client.16377 shutdown 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f883adb5740 1 -- 172.21.15.62:0/1971852825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 msgr2=0x561cbbef3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f883adb5740 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 0x561cbbef3ed0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8824005e50 tx=0x7f8824005de0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f883adb5740 1 -- 172.21.15.62:0/1971852825 shutdown_connections 2024-04-02T14:47:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f883adb5740 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x561cbbf624f0 0x561cbbf648e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f883adb5740 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cbbeebcb0 0x561cbbeef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f883adb5740 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561cbbee30d0 0x561cbbeef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f883adb5740 1 --2- 172.21.15.62:0/1971852825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cbbee27b0 0x561cbbef3ed0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f8816a00640 0 client.16377 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.741+0000 7f883adb5740 1 -- 172.21.15.62:0/1971852825 >> 172.21.15.62:0/1971852825 conn(0x561cbbedcee0 msgr2=0x561cbbee9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.742+0000 7f883adb5740 1 -- 172.21.15.62:0/1971852825 shutdown_connections 2024-04-02T14:47:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.742+0000 7f883adb5740 1 -- 172.21.15.62:0/1971852825 wait complete. 2024-04-02T14:47:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.742+0000 7f883adb5740 20 client.16377 trim_cache size 0 max 16384 2024-04-02T14:47:33.739 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:47:33.739 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-02T14:47:33.740 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-02T14:47:33.740 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:47:33.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.742+0000 7f8838000640 1 -- 172.21.15.62:0/3527255188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88180a3da0 msgr2=0x7f88180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.742+0000 7f8838000640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88180a3da0 0x7f88180a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x561cbbede070 tx=0x7f881c009700 comp rx=0 tx=0).stop 2024-04-02T14:47:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.742+0000 7f8838000640 1 -- 172.21.15.62:0/3527255188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830112940 msgr2=0x7f883019e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.742+0000 7f8838000640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830112940 0x7f883019e910 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8828005fd0 tx=0x7f882800dea0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.742+0000 7f8838000640 1 -- 172.21.15.62:0/3527255188 shutdown_connections 2024-04-02T14:47:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.743+0000 7f8838000640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88180a3da0 0x7f88180a6260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.743+0000 7f8838000640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830112940 0x7f883019e910 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.743+0000 7f8838000640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8830076950 0x7f88301a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.743+0000 7f8838000640 1 --2- 172.21.15.62:0/3527255188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8830076030 0x7f88301a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.743+0000 7f8838000640 1 -- 172.21.15.62:0/3527255188 >> 172.21.15.62:0/3527255188 conn(0x7f88300ff770 msgr2=0x7f8830114070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.743+0000 7f8838000640 1 -- 172.21.15.62:0/3527255188 shutdown_connections 2024-04-02T14:47:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.743+0000 7f8838000640 1 -- 172.21.15.62:0/3527255188 wait complete. 2024-04-02T14:47:33.754 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:47:33.754 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:47:33.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:33 smithi062 ceph-mon[32116]: pgmap v109: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 447 KiB/s wr, 118 op/s 2024-04-02T14:47:33.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.986+0000 7f644fe00640 1 -- 172.21.15.62:0/952043955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6450107950 msgr2=0x7f6450107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.986+0000 7f644fe00640 1 --2- 172.21.15.62:0/952043955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6450107950 0x7f6450107d30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6440009930 tx=0x7f644002bc30 comp rx=0 tx=0).stop 2024-04-02T14:47:33.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.986+0000 7f644fe00640 1 -- 172.21.15.62:0/952043955 shutdown_connections 2024-04-02T14:47:33.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.986+0000 7f644fe00640 1 --2- 172.21.15.62:0/952043955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6450069600 0x7f645010e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.986+0000 7f644fe00640 1 --2- 172.21.15.62:0/952043955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6450068c60 0x7f64500690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.986+0000 7f644fe00640 1 --2- 172.21.15.62:0/952043955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6450107950 0x7f6450107d30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.986+0000 7f644fe00640 1 -- 172.21.15.62:0/952043955 >> 172.21.15.62:0/952043955 conn(0x7f6450076410 msgr2=0x7f6450076820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:33.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.986+0000 7f644fe00640 1 -- 172.21.15.62:0/952043955 shutdown_connections 2024-04-02T14:47:33.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.986+0000 7f644fe00640 1 -- 172.21.15.62:0/952043955 wait complete. 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.987+0000 7f644fe00640 1 Processor -- start 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.987+0000 7f644fe00640 1 -- start start 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6450068c60 0x7f64501982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6450069600 0x7f64501987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6450107950 0x7f645019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6450111e00 con 0x7f6450068c60 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6450111c80 con 0x7f6450107950 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6450111f80 con 0x7f6450069600 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6450068c60 0x7f64501982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6450068c60 0x7f64501982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40050/0 (socket says 172.21.15.62:40050) 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644ea00640 1 -- 172.21.15.62:0/3925550261 learned_addr learned my addr 172.21.15.62:0/3925550261 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644e000640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6450069600 0x7f64501987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644f400640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6450107950 0x7f645019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644ea00640 1 -- 172.21.15.62:0/3925550261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6450069600 msgr2=0x7f64501987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.988+0000 7f644ea00640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6450069600 0x7f64501987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.989+0000 7f644ea00640 1 -- 172.21.15.62:0/3925550261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6450107950 msgr2=0x7f645019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:33.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.989+0000 7f644ea00640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6450107950 0x7f645019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:33.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.989+0000 7f644ea00640 1 -- 172.21.15.62:0/3925550261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645019d290 con 0x7f6450068c60 2024-04-02T14:47:33.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.989+0000 7f644ea00640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6450068c60 0x7f64501982a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6440009900 tx=0x7f644002dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:33.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.989+0000 7f6447400640 1 -- 172.21.15.62:0/3925550261 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440043020 con 0x7f6450068c60 2024-04-02T14:47:33.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.989+0000 7f6447400640 1 -- 172.21.15.62:0/3925550261 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6440004250 con 0x7f6450068c60 2024-04-02T14:47:33.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.989+0000 7f6447400640 1 -- 172.21.15.62:0/3925550261 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64400039f0 con 0x7f6450068c60 2024-04-02T14:47:33.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.989+0000 7f644fe00640 1 -- 172.21.15.62:0/3925550261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645019d520 con 0x7f6450068c60 2024-04-02T14:47:33.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.990+0000 7f644fe00640 1 -- 172.21.15.62:0/3925550261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64501a4e10 con 0x7f6450068c60 2024-04-02T14:47:33.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.991+0000 7f644fe00640 1 -- 172.21.15.62:0/3925550261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6450103010 con 0x7f6450068c60 2024-04-02T14:47:33.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.993+0000 7f6447400640 1 -- 172.21.15.62:0/3925550261 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f6440046020 con 0x7f6450068c60 2024-04-02T14:47:33.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.994+0000 7f6447400640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f64340a3da0 0x7f64340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:33.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.994+0000 7f6447400640 1 -- 172.21.15.62:0/3925550261 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f64400f2770 con 0x7f6450068c60 2024-04-02T14:47:33.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.995+0000 7f644e000640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f64340a3da0 0x7f64340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:33.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.995+0000 7f644e000640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f64340a3da0 0x7f64340a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6450100820 tx=0x7f643800b390 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:33.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:33.999+0000 7f6447400640 1 -- 172.21.15.62:0/3925550261 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f644004d020 con 0x7f6450068c60 2024-04-02T14:47:34.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:33 smithi204 ceph-mon[35948]: pgmap v109: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 447 KiB/s wr, 118 op/s 2024-04-02T14:47:34.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:33 smithi113 ceph-mon[36135]: pgmap v109: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 447 KiB/s wr, 118 op/s 2024-04-02T14:47:34.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.250+0000 7f644fe00640 1 -- 172.21.15.62:0/3925550261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6450109e20 con 0x7f6450068c60 2024-04-02T14:47:34.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.251+0000 7f6447400640 1 -- 172.21.15.62:0/3925550261 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11953 (secure 0 0 0) 0x7f64400bef10 con 0x7f6450068c60 2024-04-02T14:47:34.246 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:47:34.246 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:47:10.072109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":93,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":91,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":91,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24553,24565,24533,24535,24584]},"id":1}]} 2024-04-02T14:47:34.246 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 39 2024-04-02T14:47:34.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.254+0000 7f644fe00640 1 -- 172.21.15.62:0/3925550261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f64340a3da0 msgr2=0x7f64340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.254+0000 7f644fe00640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f64340a3da0 0x7f64340a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6450100820 tx=0x7f643800b390 comp rx=0 tx=0).stop 2024-04-02T14:47:34.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.254+0000 7f644fe00640 1 -- 172.21.15.62:0/3925550261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6450068c60 msgr2=0x7f64501982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.254+0000 7f644fe00640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6450068c60 0x7f64501982a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6440009900 tx=0x7f644002dc80 comp rx=0 tx=0).stop 2024-04-02T14:47:34.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.254+0000 7f644fe00640 1 -- 172.21.15.62:0/3925550261 shutdown_connections 2024-04-02T14:47:34.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.254+0000 7f644fe00640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f64340a3da0 0x7f64340a6260 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.254+0000 7f644fe00640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6450107950 0x7f645019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.254+0000 7f644fe00640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6450069600 0x7f64501987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.254+0000 7f644fe00640 1 --2- 172.21.15.62:0/3925550261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6450068c60 0x7f64501982a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.254+0000 7f644fe00640 1 -- 172.21.15.62:0/3925550261 >> 172.21.15.62:0/3925550261 conn(0x7f6450076410 msgr2=0x7f64500feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.254+0000 7f644fe00640 1 -- 172.21.15.62:0/3925550261 shutdown_connections 2024-04-02T14:47:34.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.255+0000 7f644fe00640 1 -- 172.21.15.62:0/3925550261 wait complete. 2024-04-02T14:47:34.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:47:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.498+0000 7f17a1e00640 1 -- 172.21.15.62:0/322590897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f179c1016a0 msgr2=0x7f179c10d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.498+0000 7f17a1e00640 1 --2- 172.21.15.62:0/322590897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f179c1016a0 0x7f179c10d7f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1794009c10 tx=0x7f179402bef0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.498+0000 7f17a1e00640 1 -- 172.21.15.62:0/322590897 shutdown_connections 2024-04-02T14:47:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.498+0000 7f17a1e00640 1 --2- 172.21.15.62:0/322590897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f179c1016a0 0x7f179c10d7f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.498+0000 7f17a1e00640 1 --2- 172.21.15.62:0/322590897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c100ce0 0x7f179c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.498+0000 7f17a1e00640 1 --2- 172.21.15.62:0/322590897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c105960 0x7f179c105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.498+0000 7f17a1e00640 1 -- 172.21.15.62:0/322590897 >> 172.21.15.62:0/322590897 conn(0x7f179c0779e0 msgr2=0x7f179c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.499+0000 7f17a1e00640 1 -- 172.21.15.62:0/322590897 shutdown_connections 2024-04-02T14:47:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.499+0000 7f17a1e00640 1 -- 172.21.15.62:0/322590897 wait complete. 2024-04-02T14:47:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f17a1e00640 1 Processor -- start 2024-04-02T14:47:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f17a1e00640 1 -- start start 2024-04-02T14:47:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f17a1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f179c100ce0 0x7f179c0fe590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f17a1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c1016a0 0x7f179c0fead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f17a1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c105960 0x7f179c0ff010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f17a1e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f179c110f10 con 0x7f179c105960 2024-04-02T14:47:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f17a1e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f179c110d90 con 0x7f179c100ce0 2024-04-02T14:47:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f17a1e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f179c111090 con 0x7f179c1016a0 2024-04-02T14:47:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f179aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f179c100ce0 0x7f179c0fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f179b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c105960 0x7f179c0ff010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f179aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f179c100ce0 0x7f179c0fe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:38030/0 (socket says 172.21.15.62:38030) 2024-04-02T14:47:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f179aa00640 1 -- 172.21.15.62:0/329980489 learned_addr learned my addr 172.21.15.62:0/329980489 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f179b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c105960 0x7f179c0ff010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40060/0 (socket says 172.21.15.62:40060) 2024-04-02T14:47:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.500+0000 7f1793e00640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c1016a0 0x7f179c0fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.501+0000 7f179b400640 1 -- 172.21.15.62:0/329980489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c1016a0 msgr2=0x7f179c0fead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.501+0000 7f179b400640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c1016a0 0x7f179c0fead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.501+0000 7f179b400640 1 -- 172.21.15.62:0/329980489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f179c100ce0 msgr2=0x7f179c0fe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.501+0000 7f179b400640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f179c100ce0 0x7f179c0fe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.501+0000 7f179b400640 1 -- 172.21.15.62:0/329980489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c1a6940 con 0x7f179c105960 2024-04-02T14:47:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.501+0000 7f179aa00640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f179c100ce0 0x7f179c0fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:34.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.501+0000 7f179b400640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c105960 0x7f179c0ff010 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1794036040 tx=0x7f1794033720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.501+0000 7f17a0a00640 1 -- 172.21.15.62:0/329980489 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794043020 con 0x7f179c105960 2024-04-02T14:47:34.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.502+0000 7f17a1e00640 1 -- 172.21.15.62:0/329980489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c1a6bd0 con 0x7f179c105960 2024-04-02T14:47:34.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.502+0000 7f17a0a00640 1 -- 172.21.15.62:0/329980489 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1794004040 con 0x7f179c105960 2024-04-02T14:47:34.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.502+0000 7f17a1e00640 1 -- 172.21.15.62:0/329980489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c1a7140 con 0x7f179c105960 2024-04-02T14:47:34.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.502+0000 7f17a0a00640 1 -- 172.21.15.62:0/329980489 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179402cde0 con 0x7f179c105960 2024-04-02T14:47:34.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.505+0000 7f17a4b6a740 1 Processor -- start 2024-04-02T14:47:34.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.505+0000 7f17a0a00640 1 -- 172.21.15.62:0/329980489 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1794046020 con 0x7f179c105960 2024-04-02T14:47:34.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.505+0000 7f17a4b6a740 1 -- start start 2024-04-02T14:47:34.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c49891740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c49892190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c4989b080 0x564c4989d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564c497d2400 con 0x564c49891d10 2024-04-02T14:47:34.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564c497b6f10 con 0x564c498929f0 2024-04-02T14:47:34.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564c497f66f0 con 0x564c4989b080 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f179aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c49892190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f1793e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c49891740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f179aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c49892190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40076/0 (socket says 172.21.15.62:40076) 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f1793e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c49891740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:38042/0 (socket says 172.21.15.62:38042) 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f179aa00640 1 -- 172.21.15.62:0/529240049 learned_addr learned my addr 172.21.15.62:0/529240049 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.506+0000 7f1793e00640 1 --2- 172.21.15.62:0/529240049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c4989b080 0x564c4989d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f179aa00640 1 -- 172.21.15.62:0/529240049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c4989b080 msgr2=0x564c4989d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f179aa00640 1 --2- 172.21.15.62:0/529240049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c4989b080 0x564c4989d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f179aa00640 1 -- 172.21.15.62:0/529240049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 msgr2=0x564c49891740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f179aa00640 1 --2- 172.21.15.62:0/529240049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c49891740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f179aa00640 1 -- 172.21.15.62:0/529240049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4950a3f0 con 0x564c49891d10 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f1793e00640 1 --2- 172.21.15.62:0/529240049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c49891740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f179aa00640 1 --2- 172.21.15.62:0/529240049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c49892190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f178c005f60 tx=0x7f178c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a0fc9a2bcccf7f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f17a0a00640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f17800a3d50 0x7f17800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f178be00640 1 -- 172.21.15.62:0/529240049 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c005020 con 0x564c49891d10 2024-04-02T14:47:34.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f178be00640 1 -- 172.21.15.62:0/529240049 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f178c0047a0 con 0x564c49891d10 2024-04-02T14:47:34.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f17a0a00640 1 -- 172.21.15.62:0/329980489 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f17940f2760 con 0x7f179c105960 2024-04-02T14:47:34.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f178be00640 1 -- 172.21.15.62:0/529240049 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c00f950 con 0x564c49891d10 2024-04-02T14:47:34.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.507+0000 7f1793e00640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f17800a3d50 0x7f17800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.508+0000 7f17a4b6a740 1 -- 172.21.15.62:0/529240049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 msgr2=0x564c49892190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.508+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/529240049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c49892190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f178c005f60 tx=0x7f178c0119a0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.508+0000 7f1793e00640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f17800a3d50 0x7f17800a6210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f179c0fffd0 tx=0x7f1784004730 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.508+0000 7f17a4b6a740 1 -- 172.21.15.62:0/529240049 shutdown_connections 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.508+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/529240049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c4989b080 0x564c4989d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.508+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/529240049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c49892190 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.508+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/529240049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c49891740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.508+0000 7f17a4b6a740 1 -- 172.21.15.62:0/529240049 >> 172.21.15.62:0/529240049 conn(0x564c49858a50 msgr2=0x564c4988c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.508+0000 7f17a4b6a740 1 -- 172.21.15.62:0/529240049 shutdown_connections 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.508+0000 7f17a4b6a740 1 -- 172.21.15.62:0/529240049 wait complete. 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.509+0000 7f17a4b6a740 1 -- start start 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.509+0000 7f17a4b6a740 1 Processor -- start 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c49891d10 0x564c49898050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c49898590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49898ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564c497f26a0 con 0x564c4989b080 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564c4979e980 con 0x564c498929f0 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564c497b1cb0 con 0x564c49891d10 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f17a4b6a740 10 client.0 _ll_register_callbacks cb 0x564c497b16a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f179b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49898ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f179b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49898ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40090/0 (socket says 172.21.15.62:40090) 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f179b400640 1 -- 172.21.15.62:0/4171942841 learned_addr learned my addr 172.21.15.62:0/4171942841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f179aa00640 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c49891d10 0x564c49898050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f1793e00640 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c49898590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f179b400640 1 -- 172.21.15.62:0/4171942841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c49891d10 msgr2=0x564c49898050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f179b400640 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c49891d10 0x564c49898050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f179b400640 1 -- 172.21.15.62:0/4171942841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 msgr2=0x564c49898590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f179b400640 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c49898590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f179b400640 1 -- 172.21.15.62:0/4171942841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c495d47d0 con 0x564c4989b080 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.510+0000 7f179aa00640 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c49891d10 0x564c49898050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.511+0000 7f179b400640 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49898ad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f17940bb230 tx=0x7f17940be490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8eae90ba93694e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.511+0000 7f1788c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.511+0000 7f1788c00640 1 -- 172.21.15.62:0/4171942841 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940c2c10 con 0x564c4989b080 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.511+0000 7f1788c00640 1 -- 172.21.15.62:0/4171942841 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f17940045f0 con 0x564c4989b080 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.511+0000 7f17a4b6a740 1 -- 172.21.15.62:0/4171942841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178c0047a0 con 0x564c4989b080 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.511+0000 7f1788c00640 1 -- 172.21.15.62:0/4171942841 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f17940c74d0 con 0x564c4989b080 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.511+0000 7f17a4b6a740 10 client.16395 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.511+0000 7f17a4b6a740 1 -- 172.21.15.62:0/4171942841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564c498a9f00 con 0x564c4989b080 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.512+0000 7f1788c00640 1 -- 172.21.15.62:0/4171942841 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f178c0047a0 con 0x564c4989b080 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.512+0000 7f1788c00640 1 -- 172.21.15.62:0/4171942841 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f1794004b40 con 0x564c4989b080 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.512+0000 7f17a4b6a740 10 client.16395 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.512+0000 7f17a4b6a740 10 client.16395 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:47:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.512+0000 7f17a4b6a740 20 client.16395 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:34.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.512+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x564c498ab400 0x564c498ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.512+0000 7f17a4b6a740 4 client.16395 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:47:34.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.512+0000 7f17a4b6a740 1 -- 172.21.15.62:0/4171942841 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c497b6040 con 0x564c498ab400 2024-04-02T14:47:34.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.512+0000 7f179aa00640 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x564c498ab400 0x564c498ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.513+0000 7f179aa00640 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x564c498ab400 0x564c498ad8a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e7ce8a5fb29fbf9 server_cookie=64b628bbd381683f in_seq=0 out_seq=0 2024-04-02T14:47:34.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.513+0000 7f1788c00640 10 client.16395 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:47:34.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f1788c00640 1 -- 172.21.15.62:0/4171942841 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c497b6040 con 0x564c498ab400 2024-04-02T14:47:34.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f1788c00640 10 client.16395 handle_command_reply: tid=0 2024-04-02T14:47:34.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 1 client.16395 shutdown 2024-04-02T14:47:34.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 1 -- 172.21.15.62:0/4171942841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 msgr2=0x564c49898ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49898ad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f17940bb230 tx=0x7f17940be490 comp rx=0 tx=0).stop 2024-04-02T14:47:34.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 1 -- 172.21.15.62:0/4171942841 shutdown_connections 2024-04-02T14:47:34.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x564c498ab400 0x564c498ad8a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49898ad0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c49898590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/4171942841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c49891d10 0x564c49898050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f1788c00640 0 client.16395 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:47:34.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 1 -- 172.21.15.62:0/4171942841 >> 172.21.15.62:0/4171942841 conn(0x564c49858a50 msgr2=0x564c4988e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 1 -- 172.21.15.62:0/4171942841 shutdown_connections 2024-04-02T14:47:34.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 1 -- 172.21.15.62:0/4171942841 wait complete. 2024-04-02T14:47:34.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.514+0000 7f17a4b6a740 20 client.16395 trim_cache size 0 max 16384 2024-04-02T14:47:34.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.529+0000 7f17a4b6a740 1 Processor -- start 2024-04-02T14:47:34.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f17a4b6a740 1 -- start start 2024-04-02T14:47:34.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c49891d10 0x564c498a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c498971b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49899e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1af8d0 con 0x564c4989b080 2024-04-02T14:47:34.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1b15c0 con 0x564c498929f0 2024-04-02T14:47:34.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1b0000 con 0x564c49891d10 2024-04-02T14:47:34.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f179b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49899e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f179b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49899e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40092/0 (socket says 172.21.15.62:40092) 2024-04-02T14:47:34.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f179b400640 1 -- 172.21.15.62:0/3287037246 learned_addr learned my addr 172.21.15.62:0/3287037246 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:34.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f179aa00640 1 --2- 172.21.15.62:0/3287037246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c49891d10 0x564c498a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.530+0000 7f1793e00640 1 --2- 172.21.15.62:0/3287037246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c498971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.531+0000 7f179b400640 1 -- 172.21.15.62:0/3287037246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c49891d10 msgr2=0x564c498a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.531+0000 7f179b400640 1 --2- 172.21.15.62:0/3287037246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c49891d10 0x564c498a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.531+0000 7f179b400640 1 -- 172.21.15.62:0/3287037246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 msgr2=0x564c498971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.531+0000 7f179b400640 1 --2- 172.21.15.62:0/3287037246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c498971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.531+0000 7f179b400640 1 -- 172.21.15.62:0/3287037246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4988d840 con 0x564c4989b080 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.531+0000 7f179b400640 1 --2- 172.21.15.62:0/3287037246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49899e80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17940f00c0 tx=0x7f17940bcd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd7c0ff7c23ee0a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.531+0000 7f178b400640 1 -- 172.21.15.62:0/3287037246 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940bef60 con 0x564c4989b080 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.531+0000 7f178b400640 1 -- 172.21.15.62:0/3287037246 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17940c2c10 con 0x564c4989b080 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.531+0000 7f178b400640 1 -- 172.21.15.62:0/3287037246 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940890c0 con 0x564c4989b080 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.532+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3287037246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 msgr2=0x564c49899e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.532+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/3287037246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49899e80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17940f00c0 tx=0x7f17940bcd70 comp rx=0 tx=0).stop 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.532+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3287037246 shutdown_connections 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.532+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/3287037246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c4989b080 0x564c49899e80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.532+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/3287037246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c498971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.532+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/3287037246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c49891d10 0x564c498a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.532+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3287037246 >> 172.21.15.62:0/3287037246 conn(0x564c49858a50 msgr2=0x564c49899770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.532+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3287037246 shutdown_connections 2024-04-02T14:47:34.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.532+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3287037246 wait complete. 2024-04-02T14:47:34.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.533+0000 7f17a4b6a740 1 -- start start 2024-04-02T14:47:34.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.533+0000 7f17a4b6a740 1 Processor -- start 2024-04-02T14:47:34.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.533+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c498a3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.533+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c498953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.533+0000 7f17a4b6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c4989b080 0x564c498959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.533+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1b0e30 con 0x564c49891d10 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.533+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1b0700 con 0x564c498929f0 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.533+0000 7f17a4b6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1ad8b0 con 0x564c4989b080 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.533+0000 7f17a4b6a740 10 client.0 _ll_register_callbacks cb 0x7f17940babb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f179aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c498a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f1793e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c498953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f179aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c498a3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40104/0 (socket says 172.21.15.62:40104) 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f179b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c4989b080 0x564c498959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f179aa00640 1 -- 172.21.15.62:0/3931648124 learned_addr learned my addr 172.21.15.62:0/3931648124 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f179aa00640 1 -- 172.21.15.62:0/3931648124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c4989b080 msgr2=0x564c498959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f179aa00640 1 --2- 172.21.15.62:0/3931648124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c4989b080 0x564c498959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f179aa00640 1 -- 172.21.15.62:0/3931648124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 msgr2=0x564c498953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f179aa00640 1 --2- 172.21.15.62:0/3931648124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c498953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f179aa00640 1 -- 172.21.15.62:0/3931648124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c49890d40 con 0x564c49891d10 2024-04-02T14:47:34.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f179aa00640 1 --2- 172.21.15.62:0/3931648124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c498a3490 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f178c01b040 tx=0x7f178c005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cb555c05767f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.534+0000 7f1788c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f1788c00640 1 -- 172.21.15.62:0/3931648124 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c004e40 con 0x564c49891d10 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f1788c00640 1 -- 172.21.15.62:0/3931648124 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f178c005310 con 0x564c49891d10 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3931648124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17940c2c10 con 0x564c49891d10 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f1788c00640 1 -- 172.21.15.62:0/3931648124 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f178c005510 con 0x564c49891d10 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f17a4b6a740 10 client.16401 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3931648124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564c49911a80 con 0x564c49891d10 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f1788c00640 1 -- 172.21.15.62:0/3931648124 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17940c2c10 con 0x564c49891d10 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f1788c00640 1 -- 172.21.15.62:0/3931648124 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f178c00f780 con 0x564c49891d10 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f17a4b6a740 10 client.16401 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f17a4b6a740 10 client.16401 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f17a4b6a740 20 client.16401 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/3931648124 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x564c49912090 0x564c499144b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f17a4b6a740 4 client.16401 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.535+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3931648124 --> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c49803c30 con 0x564c49912090 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.536+0000 7f1793e00640 1 --2- 172.21.15.62:0/3931648124 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x564c49912090 0x564c499144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.536+0000 7f1793e00640 1 --2- 172.21.15.62:0/3931648124 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x564c49912090 0x564c499144b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78fa8125a8818b53 server_cookie=6db3540779d7d970 in_seq=0 out_seq=0 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.536+0000 7f1788c00640 10 client.16401 ms_handle_connect on v2:172.21.15.113:6836/1122486361 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f1788c00640 1 -- 172.21.15.62:0/3931648124 <== mds.4 v2:172.21.15.113:6836/1122486361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2082 (crc 0 0 0) 0x564c49803c30 con 0x564c49912090 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f1788c00640 10 client.16401 handle_command_reply: tid=0 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 1 client.16401 shutdown 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3931648124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 msgr2=0x564c498a3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/3931648124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c498a3490 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f178c01b040 tx=0x7f178c005d70 comp rx=0 tx=0).stop 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3931648124 shutdown_connections 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/3931648124 >> [v2:172.21.15.113:6836/1122486361,v1:172.21.15.113:6837/1122486361] conn(0x564c49912090 0x564c499144b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/3931648124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c4989b080 0x564c498959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f1788c00640 0 client.16401 ms_handle_reset on v2:172.21.15.113:6836/1122486361 2024-04-02T14:47:34.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/3931648124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564c498929f0 0x564c498953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 1 --2- 172.21.15.62:0/3931648124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c49891d10 0x564c498a3490 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3931648124 >> 172.21.15.62:0/3931648124 conn(0x564c49858a50 msgr2=0x564c4988e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3931648124 shutdown_connections 2024-04-02T14:47:34.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 1 -- 172.21.15.62:0/3931648124 wait complete. 2024-04-02T14:47:34.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.537+0000 7f17a4b6a740 20 client.16401 trim_cache size 0 max 16384 2024-04-02T14:47:34.535 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-02T14:47:34.535 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:34.535 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:34.535 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 597677847, 2024-04-02T14:47:34.535 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638568, 2024-04-02T14:47:34.535 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/Makefile" 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 2465499163, 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638569, 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c" 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 2749257382, 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638964, 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt" 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 2938148192, 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511639521, 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt" 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3164408192, 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638959, 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt" 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3408690552, 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638571, 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/.gitignore" 2024-04-02T14:47:34.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3880791220, 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638570, 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/timestamping.c" 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 4001013450, 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638567, 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c" 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:47:34.538 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-02T14:47:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 -- 172.21.15.62:0/329980489 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f17800a3d50 msgr2=0x7f17800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f17800a3d50 0x7f17800a6210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f179c0fffd0 tx=0x7f1784004730 comp rx=0 tx=0).stop 2024-04-02T14:47:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 -- 172.21.15.62:0/329980489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c105960 msgr2=0x7f179c0ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c105960 0x7f179c0ff010 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1794036040 tx=0x7f1794033720 comp rx=0 tx=0).stop 2024-04-02T14:47:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 -- 172.21.15.62:0/329980489 shutdown_connections 2024-04-02T14:47:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f17800a3d50 0x7f17800a6210 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f179c105960 0x7f179c0ff010 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c1016a0 0x7f179c0fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 --2- 172.21.15.62:0/329980489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f179c100ce0 0x7f179c0fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 -- 172.21.15.62:0/329980489 >> 172.21.15.62:0/329980489 conn(0x7f179c0779e0 msgr2=0x7f179c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 -- 172.21.15.62:0/329980489 shutdown_connections 2024-04-02T14:47:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.538+0000 7f17a1e00640 1 -- 172.21.15.62:0/329980489 wait complete. 2024-04-02T14:47:34.550 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:47:34.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.772+0000 7f7a29a00640 1 -- 172.21.15.62:0/1242914998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a24103df0 msgr2=0x7f7a241041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.772+0000 7f7a29a00640 1 --2- 172.21.15.62:0/1242914998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a24103df0 0x7f7a241041d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009900 tx=0x7f7a1402bc30 comp rx=0 tx=0).stop 2024-04-02T14:47:34.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.772+0000 7f7a29a00640 1 -- 172.21.15.62:0/1242914998 shutdown_connections 2024-04-02T14:47:34.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.772+0000 7f7a29a00640 1 --2- 172.21.15.62:0/1242914998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a24108ec0 0x7f7a24110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.772+0000 7f7a29a00640 1 --2- 172.21.15.62:0/1242914998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a241047a0 0x7f7a241087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.772+0000 7f7a29a00640 1 --2- 172.21.15.62:0/1242914998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a24103df0 0x7f7a241041d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.772+0000 7f7a29a00640 1 -- 172.21.15.62:0/1242914998 >> 172.21.15.62:0/1242914998 conn(0x7f7a240ff630 msgr2=0x7f7a24101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.772+0000 7f7a29a00640 1 -- 172.21.15.62:0/1242914998 shutdown_connections 2024-04-02T14:47:34.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.772+0000 7f7a29a00640 1 -- 172.21.15.62:0/1242914998 wait complete. 2024-04-02T14:47:34.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.773+0000 7f7a29a00640 1 Processor -- start 2024-04-02T14:47:34.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.773+0000 7f7a29a00640 1 -- start start 2024-04-02T14:47:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.774+0000 7f7a29a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a24103df0 0x7f7a2419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.774+0000 7f7a29a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a24108ec0 0x7f7a2419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.774+0000 7f7a29a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2419ee40 0x7f7a2419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.774+0000 7f7a29a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7a24114060 con 0x7f7a2419ee40 2024-04-02T14:47:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.774+0000 7f7a29a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7a24113ee0 con 0x7f7a24103df0 2024-04-02T14:47:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.774+0000 7f7a29a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241141e0 con 0x7f7a24108ec0 2024-04-02T14:47:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.774+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a24103df0 0x7f7a2419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.774+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a24103df0 0x7f7a2419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:38082/0 (socket says 172.21.15.62:38082) 2024-04-02T14:47:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.774+0000 7f7a23e00640 1 -- 172.21.15.62:0/29245367 learned_addr learned my addr 172.21.15.62:0/29245367 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.774+0000 7f7a29000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2419ee40 0x7f7a2419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.774+0000 7f7a23400640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a24108ec0 0x7f7a2419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.775+0000 7f7a29000640 1 -- 172.21.15.62:0/29245367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a24108ec0 msgr2=0x7f7a2419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.775+0000 7f7a29000640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a24108ec0 0x7f7a2419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.775+0000 7f7a29000640 1 -- 172.21.15.62:0/29245367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a24103df0 msgr2=0x7f7a2419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.775+0000 7f7a29000640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a24103df0 0x7f7a2419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.775+0000 7f7a29000640 1 -- 172.21.15.62:0/29245367 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2419f930 con 0x7f7a2419ee40 2024-04-02T14:47:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.775+0000 7f7a23e00640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a24103df0 0x7f7a2419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.775+0000 7f7a29000640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2419ee40 0x7f7a2419f2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00e970 tx=0x7f7a1c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.775+0000 7f7a20c00640 1 -- 172.21.15.62:0/29245367 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c00ccc0 con 0x7f7a2419ee40 2024-04-02T14:47:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.776+0000 7f7a29a00640 1 -- 172.21.15.62:0/29245367 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2419fc20 con 0x7f7a2419ee40 2024-04-02T14:47:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.776+0000 7f7a20c00640 1 -- 172.21.15.62:0/29245367 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7a1c005020 con 0x7f7a2419ee40 2024-04-02T14:47:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.776+0000 7f7a20c00640 1 -- 172.21.15.62:0/29245367 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c004a20 con 0x7f7a2419ee40 2024-04-02T14:47:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.776+0000 7f7a29a00640 1 -- 172.21.15.62:0/29245367 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a241a73c0 con 0x7f7a2419ee40 2024-04-02T14:47:34.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.780+0000 7f7a2ddb5740 1 Processor -- start 2024-04-02T14:47:34.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.780+0000 7f7a20c00640 1 -- 172.21.15.62:0/29245367 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7a1c004c20 con 0x7f7a2419ee40 2024-04-02T14:47:34.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.780+0000 7f7a2ddb5740 1 -- start start 2024-04-02T14:47:34.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.781+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e44997b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.781+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e44997c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.781+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449987620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.781+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e4498bc400 con 0x55e449985230 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.781+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e4498a0f10 con 0x55e44997bec0 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.781+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e4498e06f0 con 0x55e44997cba0 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.781+0000 7f7a23400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e44997b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.781+0000 7f7a23400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e44997b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50000/0 (socket says 172.21.15.62:50000) 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.781+0000 7f7a23400640 1 -- 172.21.15.62:0/3570039977 learned_addr learned my addr 172.21.15.62:0/3570039977 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.781+0000 7f7a23e00640 1 --2- 172.21.15.62:0/3570039977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e44997c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.781+0000 7f7a23400640 1 --2- 172.21.15.62:0/3570039977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449987620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a23e00640 1 -- 172.21.15.62:0/3570039977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 msgr2=0x55e44997b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a23e00640 1 --2- 172.21.15.62:0/3570039977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e44997b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a23e00640 1 -- 172.21.15.62:0/3570039977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 msgr2=0x55e449987620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a23e00640 1 --2- 172.21.15.62:0/3570039977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449987620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a23e00640 1 -- 172.21.15.62:0/3570039977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4495f43f0 con 0x55e44997bec0 2024-04-02T14:47:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a23e00640 1 --2- 172.21.15.62:0/3570039977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e44997c340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7a14005ec0 tx=0x7f7a1402da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc8dd431b4bec61 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a20c00640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7a080a3d20 0x7f7a080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a20c00640 1 -- 172.21.15.62:0/29245367 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f7a1c0d07b0 con 0x7f7a2419ee40 2024-04-02T14:47:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a23400640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7a080a3d20 0x7f7a080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a19600640 1 -- 172.21.15.62:0/3570039977 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1402c9d0 con 0x55e44997bec0 2024-04-02T14:47:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a19600640 1 -- 172.21.15.62:0/3570039977 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7a1402cb70 con 0x55e44997bec0 2024-04-02T14:47:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.782+0000 7f7a19600640 1 -- 172.21.15.62:0/3570039977 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a140048c0 con 0x55e44997bec0 2024-04-02T14:47:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.783+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/3570039977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 msgr2=0x55e44997c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.783+0000 7f7a23400640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7a080a3d20 0x7f7a080a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7a10005f40 tx=0x7f7a10009680 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.783+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/3570039977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e44997c340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7a14005ec0 tx=0x7f7a1402da60 comp rx=0 tx=0).stop 2024-04-02T14:47:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.783+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/3570039977 shutdown_connections 2024-04-02T14:47:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.783+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/3570039977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449987620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.783+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/3570039977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e44997c340 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.783+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/3570039977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e44997b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.783+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/3570039977 >> 172.21.15.62:0/3570039977 conn(0x55e449942a50 msgr2=0x55e449976910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.783+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/3570039977 shutdown_connections 2024-04-02T14:47:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.783+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/3570039977 wait complete. 2024-04-02T14:47:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.784+0000 7f7a2ddb5740 1 -- start start 2024-04-02T14:47:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.784+0000 7f7a2ddb5740 1 Processor -- start 2024-04-02T14:47:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.784+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e449982240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.784+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e449982780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.784+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449982cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e4498dc6a0 con 0x55e449985230 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e449888980 con 0x55e44997bec0 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e44989bcb0 con 0x55e44997cba0 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a2ddb5740 10 client.0 _ll_register_callbacks cb 0x55e44989b6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a29000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449982cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e449982240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a29000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449982cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40132/0 (socket says 172.21.15.62:40132) 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e449982240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:38088/0 (socket says 172.21.15.62:38088) 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a29000640 1 -- 172.21.15.62:0/6378349 learned_addr learned my addr 172.21.15.62:0/6378349 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a23400640 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e449982780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a29000640 1 -- 172.21.15.62:0/6378349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 msgr2=0x55e449982780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a29000640 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e449982780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a29000640 1 -- 172.21.15.62:0/6378349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 msgr2=0x55e449982240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a29000640 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e449982240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a29000640 1 -- 172.21.15.62:0/6378349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4496be7d0 con 0x55e449985230 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a23400640 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e449982780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a29000640 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449982cc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009230 tx=0x7f7a1c0056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b87eb8922fcf58 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.785+0000 7f7a0e000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.786+0000 7f7a0e000640 1 -- 172.21.15.62:0/6378349 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c01c070 con 0x55e449985230 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.786+0000 7f7a0e000640 1 -- 172.21.15.62:0/6378349 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f7a1c09c9a0 con 0x55e449985230 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.786+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/6378349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a1402cb70 con 0x55e449985230 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.786+0000 7f7a0e000640 1 -- 172.21.15.62:0/6378349 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c0a0af0 con 0x55e449985230 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.786+0000 7f7a2ddb5740 10 client.16413 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.786+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/6378349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e4499940e0 con 0x55e449985230 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.786+0000 7f7a0e000640 1 -- 172.21.15.62:0/6378349 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7a1402cb70 con 0x55e449985230 2024-04-02T14:47:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.786+0000 7f7a0e000640 1 -- 172.21.15.62:0/6378349 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f7a1c09bb90 con 0x55e449985230 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.787+0000 7f7a2ddb5740 10 client.16413 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.787+0000 7f7a2ddb5740 10 client.16413 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.787+0000 7f7a2ddb5740 20 client.16413 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.787+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55e449995640 0x55e449997ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.787+0000 7f7a2ddb5740 4 client.16413 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.787+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/6378349 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4498a0040 con 0x55e449995640 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.787+0000 7f7a23e00640 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55e449995640 0x55e449997ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.788+0000 7f7a23e00640 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55e449995640 0x55e449997ab0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f061db218f6f44e4 server_cookie=83d904c3aa2dbdcc in_seq=0 out_seq=0 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.788+0000 7f7a0e000640 10 client.16413 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a0e000640 1 -- 172.21.15.62:0/6378349 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4498a0040 con 0x55e449995640 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a0e000640 10 client.16413 handle_command_reply: tid=0 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 1 client.16413 shutdown 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/6378349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 msgr2=0x55e449982cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449982cc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009230 tx=0x7f7a1c0056b0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/6378349 shutdown_connections 2024-04-02T14:47:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55e449995640 0x55e449997ab0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449982cc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e449982780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/6378349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e449982240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/6378349 >> 172.21.15.62:0/6378349 conn(0x55e449942a50 msgr2=0x55e449978660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a0e000640 0 client.16413 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:47:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/6378349 shutdown_connections 2024-04-02T14:47:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/6378349 wait complete. 2024-04-02T14:47:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.789+0000 7f7a2ddb5740 20 client.16413 trim_cache size 0 max 16384 2024-04-02T14:47:34.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.804+0000 7f7a2ddb5740 1 Processor -- start 2024-04-02T14:47:34.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.804+0000 7f7a2ddb5740 1 -- start start 2024-04-02T14:47:34.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.805+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e449990c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.805+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e449976440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.805+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e44999d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.805+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241af750 con 0x55e449985230 2024-04-02T14:47:34.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.805+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241b1440 con 0x55e44997bec0 2024-04-02T14:47:34.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.805+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241afe80 con 0x55e44997cba0 2024-04-02T14:47:34.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.805+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e449990c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.805+0000 7f7a29000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e44999d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.805+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e449990c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:38098/0 (socket says 172.21.15.62:38098) 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.805+0000 7f7a23e00640 1 -- 172.21.15.62:0/1922350368 learned_addr learned my addr 172.21.15.62:0/1922350368 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.805+0000 7f7a23400640 1 --2- 172.21.15.62:0/1922350368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e449976440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.806+0000 7f7a23e00640 1 -- 172.21.15.62:0/1922350368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 msgr2=0x55e449976440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.806+0000 7f7a23e00640 1 --2- 172.21.15.62:0/1922350368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e449976440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.806+0000 7f7a23e00640 1 -- 172.21.15.62:0/1922350368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 msgr2=0x55e44999d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.806+0000 7f7a23e00640 1 --2- 172.21.15.62:0/1922350368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e44999d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.806+0000 7f7a23e00640 1 -- 172.21.15.62:0/1922350368 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e449976b20 con 0x55e44997bec0 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.806+0000 7f7a29000640 1 --2- 172.21.15.62:0/1922350368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e44999d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.806+0000 7f7a23e00640 1 --2- 172.21.15.62:0/1922350368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e449990c40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7a14005ec0 tx=0x7f7a14005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c8c60b9563d704 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.806+0000 7f7a18c00640 1 -- 172.21.15.62:0/1922350368 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14004040 con 0x55e44997bec0 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.806+0000 7f7a18c00640 1 -- 172.21.15.62:0/1922350368 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7a140041e0 con 0x55e44997bec0 2024-04-02T14:47:34.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.806+0000 7f7a18c00640 1 -- 172.21.15.62:0/1922350368 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14003be0 con 0x55e44997bec0 2024-04-02T14:47:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.807+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/1922350368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 msgr2=0x55e449990c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.807+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/1922350368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e449990c40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7a14005ec0 tx=0x7f7a14005fd0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.807+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/1922350368 shutdown_connections 2024-04-02T14:47:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.807+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/1922350368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e44999d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.807+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/1922350368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997cba0 0x55e449976440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.807+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/1922350368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997bec0 0x55e449990c40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.807+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/1922350368 >> 172.21.15.62:0/1922350368 conn(0x55e449942a50 msgr2=0x55e449981270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.807+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/1922350368 shutdown_connections 2024-04-02T14:47:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.807+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/1922350368 wait complete. 2024-04-02T14:47:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.808+0000 7f7a2ddb5740 1 -- start start 2024-04-02T14:47:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.808+0000 7f7a2ddb5740 1 Processor -- start 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.808+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997bec0 0x55e44998d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.808+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997cba0 0x55e449985a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.808+0000 7f7a2ddb5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449985f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.808+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241b0cb0 con 0x55e449985230 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.808+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241b0580 con 0x55e44997cba0 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a2ddb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241ad750 con 0x55e44997bec0 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a2ddb5740 10 client.0 _ll_register_callbacks cb 0x7f7a1c09c0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997bec0 0x55e44998d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a29000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449985f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997bec0 0x55e44998d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50036/0 (socket says 172.21.15.62:50036) 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a23e00640 1 -- 172.21.15.62:0/4117938898 learned_addr learned my addr 172.21.15.62:0/4117938898 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a23400640 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997cba0 0x55e449985a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a29000640 1 -- 172.21.15.62:0/4117938898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997bec0 msgr2=0x55e44998d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a29000640 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997bec0 0x55e44998d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a29000640 1 -- 172.21.15.62:0/4117938898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997cba0 msgr2=0x55e449985a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a29000640 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997cba0 0x55e449985a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a29000640 1 -- 172.21.15.62:0/4117938898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4499865a0 con 0x55e449985230 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.809+0000 7f7a23e00640 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997bec0 0x55e44998d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.810+0000 7f7a29000640 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449985f60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c09bf80 tx=0x7f7a1c004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6085930c4105104 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.810+0000 7f7a0e000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.810+0000 7f7a0e000640 1 -- 172.21.15.62:0/4117938898 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c01c070 con 0x55e449985230 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.810+0000 7f7a0e000640 1 -- 172.21.15.62:0/4117938898 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f7a1c09c9a0 con 0x55e449985230 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.810+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/4117938898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a140041e0 con 0x55e449985230 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.810+0000 7f7a0e000640 1 -- 172.21.15.62:0/4117938898 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c09c280 con 0x55e449985230 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.810+0000 7f7a2ddb5740 10 client.16419 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:34.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.810+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/4117938898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e44998cf80 con 0x55e449985230 2024-04-02T14:47:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.810+0000 7f7a0e000640 1 -- 172.21.15.62:0/4117938898 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7a140041e0 con 0x55e449985230 2024-04-02T14:47:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.811+0000 7f7a0e000640 1 -- 172.21.15.62:0/4117938898 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f7a1c0a1e10 con 0x55e449985230 2024-04-02T14:47:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.811+0000 7f7a2ddb5740 10 client.16419 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.811+0000 7f7a2ddb5740 10 client.16419 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-04-02T14:47:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.811+0000 7f7a2ddb5740 20 client.16419 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.811+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55e449a049e0 0x55e449a06e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.811+0000 7f7a2ddb5740 4 client.16419 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.811+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/4117938898 --> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4498edc30 con 0x55e449a049e0 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.811+0000 7f7a23e00640 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55e449a049e0 0x55e449a06e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a23e00640 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55e449a049e0 0x55e449a06e00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=67bd9fc78594f8d3 server_cookie=bb69481d10620 in_seq=0 out_seq=0 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a0e000640 10 client.16419 ms_handle_connect on v2:172.21.15.204:6838/631749651 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a0e000640 1 -- 172.21.15.62:0/4117938898 <== mds.3 v2:172.21.15.204:6838/631749651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4498edc30 con 0x55e449a049e0 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a0e000640 10 client.16419 handle_command_reply: tid=0 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a2ddb5740 1 client.16419 shutdown 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/4117938898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 msgr2=0x55e449985f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449985f60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c09bf80 tx=0x7f7a1c004550 comp rx=0 tx=0).stop 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/4117938898 shutdown_connections 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.204:6838/631749651,v1:172.21.15.204:6839/631749651] conn(0x55e449a049e0 0x55e449a06e00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e449985230 0x55e449985f60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e44997cba0 0x55e449985a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a2ddb5740 1 --2- 172.21.15.62:0/4117938898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e44997bec0 0x55e44998d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a0e000640 0 client.16419 ms_handle_reset on v2:172.21.15.204:6838/631749651 2024-04-02T14:47:34.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/4117938898 >> 172.21.15.62:0/4117938898 conn(0x55e449942a50 msgr2=0x55e449975260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/4117938898 shutdown_connections 2024-04-02T14:47:34.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.812+0000 7f7a2ddb5740 1 -- 172.21.15.62:0/4117938898 wait complete. 2024-04-02T14:47:34.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a2ddb5740 20 client.16419 trim_cache size 0 max 16384 2024-04-02T14:47:34.810 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:47:34.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a29a00640 1 -- 172.21.15.62:0/29245367 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7a080a3d20 msgr2=0x7f7a080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a29a00640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7a080a3d20 0x7f7a080a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7a10005f40 tx=0x7f7a10009680 comp rx=0 tx=0).stop 2024-04-02T14:47:34.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a29a00640 1 -- 172.21.15.62:0/29245367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2419ee40 msgr2=0x7f7a2419f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:34.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a29a00640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2419ee40 0x7f7a2419f2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00e970 tx=0x7f7a1c00ee30 comp rx=0 tx=0).stop 2024-04-02T14:47:34.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a29a00640 1 -- 172.21.15.62:0/29245367 shutdown_connections 2024-04-02T14:47:34.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a29a00640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f7a080a3d20 0x7f7a080a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a29a00640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2419ee40 0x7f7a2419f2f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a29a00640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a24108ec0 0x7f7a2419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a29a00640 1 --2- 172.21.15.62:0/29245367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a24103df0 0x7f7a2419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:34.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a29a00640 1 -- 172.21.15.62:0/29245367 >> 172.21.15.62:0/29245367 conn(0x7f7a240ff630 msgr2=0x7f7a24101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:34.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.813+0000 7f7a29a00640 1 -- 172.21.15.62:0/29245367 shutdown_connections 2024-04-02T14:47:34.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:34.814+0000 7f7a29a00640 1 -- 172.21.15.62:0/29245367 wait complete. 2024-04-02T14:47:34.822 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:34 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3925550261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:47:34.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:47:35.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:34 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3925550261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:47:35.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.039+0000 7f0482a00640 1 -- 172.21.15.62:0/3975085803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c103df0 msgr2=0x7f047c1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.039+0000 7f0482a00640 1 --2- 172.21.15.62:0/3975085803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c103df0 0x7f047c1041d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0470009900 tx=0x7f047002bc30 comp rx=0 tx=0).stop 2024-04-02T14:47:35.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.039+0000 7f0482a00640 1 -- 172.21.15.62:0/3975085803 shutdown_connections 2024-04-02T14:47:35.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.039+0000 7f0482a00640 1 --2- 172.21.15.62:0/3975085803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c108ec0 0x7f047c110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.039+0000 7f0482a00640 1 --2- 172.21.15.62:0/3975085803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c1047a0 0x7f047c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.039+0000 7f0482a00640 1 --2- 172.21.15.62:0/3975085803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c103df0 0x7f047c1041d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.039+0000 7f0482a00640 1 -- 172.21.15.62:0/3975085803 >> 172.21.15.62:0/3975085803 conn(0x7f047c0ff630 msgr2=0x7f047c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.040+0000 7f0482a00640 1 -- 172.21.15.62:0/3975085803 shutdown_connections 2024-04-02T14:47:35.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.040+0000 7f0482a00640 1 -- 172.21.15.62:0/3975085803 wait complete. 2024-04-02T14:47:35.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.041+0000 7f0482a00640 1 Processor -- start 2024-04-02T14:47:35.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.041+0000 7f0482a00640 1 -- start start 2024-04-02T14:47:35.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.041+0000 7f0482a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c103df0 0x7f047c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0482a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c108ec0 0x7f047c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0482a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c19ee90 0x7f047c19f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0482a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f047c114060 con 0x7f047c108ec0 2024-04-02T14:47:35.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0482a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f047c113ee0 con 0x7f047c103df0 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0482a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1141e0 con 0x7f047c19ee90 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0480c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c108ec0 0x7f047c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0480c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c108ec0 0x7f047c19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40170/0 (socket says 172.21.15.62:40170) 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0480c00640 1 -- 172.21.15.62:0/800290192 learned_addr learned my addr 172.21.15.62:0/800290192 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0481600640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c103df0 0x7f047c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0482000640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c19ee90 0x7f047c19f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0480c00640 1 -- 172.21.15.62:0/800290192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c19ee90 msgr2=0x7f047c19f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0480c00640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c19ee90 0x7f047c19f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0480c00640 1 -- 172.21.15.62:0/800290192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c103df0 msgr2=0x7f047c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0480c00640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c103df0 0x7f047c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0480c00640 1 -- 172.21.15.62:0/800290192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c19f980 con 0x7f047c108ec0 2024-04-02T14:47:35.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.042+0000 7f0481600640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c103df0 0x7f047c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:47:35.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.043+0000 7f0480c00640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c108ec0 0x7f047c19ab60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f046800e9c0 tx=0x7f046800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.043+0000 7f047a000640 1 -- 172.21.15.62:0/800290192 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046800cd80 con 0x7f047c108ec0 2024-04-02T14:47:35.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.043+0000 7f047a000640 1 -- 172.21.15.62:0/800290192 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0468005520 con 0x7f047c108ec0 2024-04-02T14:47:35.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.043+0000 7f0482a00640 1 -- 172.21.15.62:0/800290192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047c19fc70 con 0x7f047c108ec0 2024-04-02T14:47:35.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.043+0000 7f047a000640 1 -- 172.21.15.62:0/800290192 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468010690 con 0x7f047c108ec0 2024-04-02T14:47:35.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.043+0000 7f0482a00640 1 -- 172.21.15.62:0/800290192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047c1a74b0 con 0x7f047c108ec0 2024-04-02T14:47:35.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.047+0000 7f047a000640 1 -- 172.21.15.62:0/800290192 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f04680046c0 con 0x7f047c108ec0 2024-04-02T14:47:35.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.047+0000 7f0486c0f740 1 Processor -- start 2024-04-02T14:47:35.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.047+0000 7f0486c0f740 1 -- start start 2024-04-02T14:47:35.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.047+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277d98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.047+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277d9ec0 0x55da277da340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.047+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277e3230 0x55da277e5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.047+0000 7f0486c0f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55da2771a400 con 0x55da277e3230 2024-04-02T14:47:35.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.047+0000 7f0486c0f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55da276fef10 con 0x55da277daba0 2024-04-02T14:47:35.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.047+0000 7f0486c0f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55da2773e6f0 con 0x55da277d9ec0 2024-04-02T14:47:35.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.048+0000 7f0480c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277e3230 0x55da277e5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.048+0000 7f0482000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277d98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.048+0000 7f0481600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277d9ec0 0x55da277da340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.048+0000 7f0482000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277d98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:38146/0 (socket says 172.21.15.62:38146) 2024-04-02T14:47:35.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.048+0000 7f0482000640 1 -- 172.21.15.62:0/1505832274 learned_addr learned my addr 172.21.15.62:0/1505832274 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.048+0000 7f0481600640 1 -- 172.21.15.62:0/1505832274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 msgr2=0x55da277d98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.048+0000 7f0481600640 1 --2- 172.21.15.62:0/1505832274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277d98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.048+0000 7f0481600640 1 -- 172.21.15.62:0/1505832274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277e3230 msgr2=0x55da277e5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.048+0000 7f0481600640 1 --2- 172.21.15.62:0/1505832274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277e3230 0x55da277e5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.048+0000 7f0481600640 1 -- 172.21.15.62:0/1505832274 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da274523f0 con 0x55da277d9ec0 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.048+0000 7f0482000640 1 --2- 172.21.15.62:0/1505832274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277d98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.049+0000 7f047a000640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f04640a3da0 0x7f04640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.049+0000 7f0481600640 1 --2- 172.21.15.62:0/1505832274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277d9ec0 0x55da277da340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0470009230 tx=0x7f0470005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bcdfab630d7a37e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.049+0000 7f047a000640 1 -- 172.21.15.62:0/800290192 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f04680d0c70 con 0x7f047c108ec0 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.049+0000 7f046e600640 1 -- 172.21.15.62:0/1505832274 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047002c9d0 con 0x55da277d9ec0 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.049+0000 7f0482000640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f04640a3da0 0x7f04640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.049+0000 7f046e600640 1 -- 172.21.15.62:0/1505832274 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f047002cb70 con 0x55da277d9ec0 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.049+0000 7f046e600640 1 -- 172.21.15.62:0/1505832274 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470004920 con 0x55da277d9ec0 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.050+0000 7f0486c0f740 1 -- 172.21.15.62:0/1505832274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277d9ec0 msgr2=0x55da277da340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.050+0000 7f0486c0f740 1 --2- 172.21.15.62:0/1505832274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277d9ec0 0x55da277da340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0470009230 tx=0x7f0470005e80 comp rx=0 tx=0).stop 2024-04-02T14:47:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.050+0000 7f0482000640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f04640a3da0 0x7f04640a6260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f04740117b0 tx=0x7f0474009500 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.050+0000 7f0486c0f740 1 -- 172.21.15.62:0/1505832274 shutdown_connections 2024-04-02T14:47:35.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.050+0000 7f0486c0f740 1 --2- 172.21.15.62:0/1505832274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277e3230 0x55da277e5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.050+0000 7f0486c0f740 1 --2- 172.21.15.62:0/1505832274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277d9ec0 0x55da277da340 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.050+0000 7f0486c0f740 1 --2- 172.21.15.62:0/1505832274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277d98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.050+0000 7f0486c0f740 1 -- 172.21.15.62:0/1505832274 >> 172.21.15.62:0/1505832274 conn(0x55da277a0a50 msgr2=0x55da277d4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.050+0000 7f0486c0f740 1 -- 172.21.15.62:0/1505832274 shutdown_connections 2024-04-02T14:47:35.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.050+0000 7f0486c0f740 1 -- 172.21.15.62:0/1505832274 wait complete. 2024-04-02T14:47:35.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.050+0000 7f0486c0f740 1 -- start start 2024-04-02T14:47:35.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0486c0f740 1 Processor -- start 2024-04-02T14:47:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277e01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277e0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277e3230 0x55da277e0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0486c0f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55da2773a6a0 con 0x55da277d9ec0 2024-04-02T14:47:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0486c0f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55da276e6980 con 0x55da277daba0 2024-04-02T14:47:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0486c0f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55da276f9cb0 con 0x55da277e3230 2024-04-02T14:47:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0486c0f740 10 client.0 _ll_register_callbacks cb 0x55da276f96a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0481600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277e01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0481600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277e01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40190/0 (socket says 172.21.15.62:40190) 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0481600640 1 -- 172.21.15.62:0/3067597143 learned_addr learned my addr 172.21.15.62:0/3067597143 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.051+0000 7f0480c00640 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277e0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.052+0000 7f0482000640 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277e3230 0x55da277e0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.052+0000 7f0481600640 1 -- 172.21.15.62:0/3067597143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277e3230 msgr2=0x55da277e0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.052+0000 7f0481600640 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277e3230 0x55da277e0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.052+0000 7f0481600640 1 -- 172.21.15.62:0/3067597143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 msgr2=0x55da277e0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.052+0000 7f0481600640 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277e0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.052+0000 7f0481600640 1 -- 172.21.15.62:0/3067597143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2751c7d0 con 0x55da277d9ec0 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.052+0000 7f0480c00640 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.052+0000 7f0481600640 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277e01e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0470036040 tx=0x7f04700045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17df1aa4111ff98b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.052+0000 7f0462a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.053+0000 7f0462a00640 1 -- 172.21.15.62:0/3067597143 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047004e020 con 0x55da277d9ec0 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.053+0000 7f0462a00640 1 -- 172.21.15.62:0/3067597143 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f04700035f0 con 0x55da277d9ec0 2024-04-02T14:47:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.053+0000 7f0486c0f740 1 -- 172.21.15.62:0/3067597143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f047002cb70 con 0x55da277d9ec0 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.053+0000 7f0462a00640 1 -- 172.21.15.62:0/3067597143 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0470059050 con 0x55da277d9ec0 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.053+0000 7f0486c0f740 10 client.16431 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.053+0000 7f0486c0f740 1 -- 172.21.15.62:0/3067597143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55da277f2040 con 0x55da277d9ec0 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.053+0000 7f0462a00640 1 -- 172.21.15.62:0/3067597143 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f047002cb70 con 0x55da277d9ec0 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.054+0000 7f0462a00640 1 -- 172.21.15.62:0/3067597143 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f04700347b0 con 0x55da277d9ec0 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.054+0000 7f0486c0f740 10 client.16431 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.054+0000 7f0486c0f740 10 client.16431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.054+0000 7f0486c0f740 20 client.16431 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.054+0000 7f0486c0f740 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55da277f3390 0x55da277f5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.054+0000 7f0486c0f740 4 client.16431 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.054+0000 7f0486c0f740 1 -- 172.21.15.62:0/3067597143 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da276fe040 con 0x55da277f3390 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.054+0000 7f0480c00640 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55da277f3390 0x55da277f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.055+0000 7f0480c00640 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55da277f3390 0x55da277f5780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ace62603bffdc7c server_cookie=741652b5fac8c9ee in_seq=0 out_seq=0 2024-04-02T14:47:35.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.055+0000 7f0462a00640 10 client.16431 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:47:35.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.055+0000 7f0462a00640 1 -- 172.21.15.62:0/3067597143 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da276fe040 con 0x55da277f3390 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.055+0000 7f0462a00640 10 client.16431 handle_command_reply: tid=0 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 1 client.16431 shutdown 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 1 -- 172.21.15.62:0/3067597143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 msgr2=0x55da277e01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277e01e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0470036040 tx=0x7f04700045e0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 1 -- 172.21.15.62:0/3067597143 shutdown_connections 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55da277f3390 0x55da277f5780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277e3230 0x55da277e0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0462a00640 0 client.16431 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 1 --2- 172.21.15.62:0/3067597143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277e01e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 1 -- 172.21.15.62:0/3067597143 >> 172.21.15.62:0/3067597143 conn(0x55da277a0a50 msgr2=0x55da277d6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 1 -- 172.21.15.62:0/3067597143 shutdown_connections 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 1 -- 172.21.15.62:0/3067597143 wait complete. 2024-04-02T14:47:35.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.056+0000 7f0486c0f740 20 client.16431 trim_cache size 0 max 16384 2024-04-02T14:47:35.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.071+0000 7f0486c0f740 1 Processor -- start 2024-04-02T14:47:35.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.071+0000 7f0486c0f740 1 -- start start 2024-04-02T14:47:35.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.072+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277e2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.072+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277eec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.072+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277e3230 0x55da277d3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.072+0000 7f0486c0f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1af900 con 0x55da277d9ec0 2024-04-02T14:47:35.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.072+0000 7f0486c0f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b15f0 con 0x55da277daba0 2024-04-02T14:47:35.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.072+0000 7f0486c0f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b0030 con 0x55da277e3230 2024-04-02T14:47:35.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.072+0000 7f0482000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277e3230 0x55da277d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.072+0000 7f0482000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277e3230 0x55da277d3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50080/0 (socket says 172.21.15.62:50080) 2024-04-02T14:47:35.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.072+0000 7f0482000640 1 -- 172.21.15.62:0/573375106 learned_addr learned my addr 172.21.15.62:0/573375106 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.072+0000 7f0481600640 1 --2- 172.21.15.62:0/573375106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277e2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.072+0000 7f0480c00640 1 --2- 172.21.15.62:0/573375106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277eec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.073+0000 7f0480c00640 1 -- 172.21.15.62:0/573375106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277e3230 msgr2=0x55da277d3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.073+0000 7f0480c00640 1 --2- 172.21.15.62:0/573375106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277e3230 0x55da277d3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.073+0000 7f0480c00640 1 -- 172.21.15.62:0/573375106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 msgr2=0x55da277e2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.073+0000 7f0480c00640 1 --2- 172.21.15.62:0/573375106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277e2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.073+0000 7f0480c00640 1 -- 172.21.15.62:0/573375106 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da277d8ef0 con 0x55da277daba0 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.073+0000 7f0480c00640 1 --2- 172.21.15.62:0/573375106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277eec40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f04680ce3b0 tx=0x7f04680bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce3806a647a4fa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.073+0000 7f046dc00640 1 -- 172.21.15.62:0/573375106 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680b6030 con 0x55da277daba0 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.073+0000 7f046dc00640 1 -- 172.21.15.62:0/573375106 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f046809d300 con 0x55da277daba0 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.073+0000 7f046dc00640 1 -- 172.21.15.62:0/573375106 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680a0fb0 con 0x55da277daba0 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.074+0000 7f0486c0f740 1 -- 172.21.15.62:0/573375106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 msgr2=0x55da277eec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.074+0000 7f0486c0f740 1 --2- 172.21.15.62:0/573375106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277eec40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f04680ce3b0 tx=0x7f04680bd040 comp rx=0 tx=0).stop 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.074+0000 7f0486c0f740 1 -- 172.21.15.62:0/573375106 shutdown_connections 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.074+0000 7f0486c0f740 1 --2- 172.21.15.62:0/573375106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277e3230 0x55da277d3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.074+0000 7f0486c0f740 1 --2- 172.21.15.62:0/573375106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277daba0 0x55da277eec40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.074+0000 7f0486c0f740 1 --2- 172.21.15.62:0/573375106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277e2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.074+0000 7f0486c0f740 1 -- 172.21.15.62:0/573375106 >> 172.21.15.62:0/573375106 conn(0x55da277a0a50 msgr2=0x55da277d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.074+0000 7f0486c0f740 1 -- 172.21.15.62:0/573375106 shutdown_connections 2024-04-02T14:47:35.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.074+0000 7f0486c0f740 1 -- 172.21.15.62:0/573375106 wait complete. 2024-04-02T14:47:35.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.074+0000 7f0486c0f740 1 -- start start 2024-04-02T14:47:35.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.075+0000 7f0486c0f740 1 Processor -- start 2024-04-02T14:47:35.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.075+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.075+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277daba0 0x55da277dd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.075+0000 7f0486c0f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277e3230 0x55da277ddd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.075+0000 7f0486c0f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b0e60 con 0x55da277d9ec0 2024-04-02T14:47:35.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.075+0000 7f0486c0f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b0730 con 0x55da277e3230 2024-04-02T14:47:35.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.075+0000 7f0486c0f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1ad900 con 0x55da277daba0 2024-04-02T14:47:35.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.075+0000 7f0486c0f740 10 client.0 _ll_register_callbacks cb 0x7f04740093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:35.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0481600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0481600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277eb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40202/0 (socket says 172.21.15.62:40202) 2024-04-02T14:47:35.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0481600640 1 -- 172.21.15.62:0/277598393 learned_addr learned my addr 172.21.15.62:0/277598393 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0482000640 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277e3230 0x55da277ddd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0480c00640 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277daba0 0x55da277dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0481600640 1 -- 172.21.15.62:0/277598393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277daba0 msgr2=0x55da277dd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0481600640 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277daba0 0x55da277dd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0481600640 1 -- 172.21.15.62:0/277598393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277e3230 msgr2=0x55da277ddd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0481600640 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277e3230 0x55da277ddd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0481600640 1 -- 172.21.15.62:0/277598393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da277e40d0 con 0x55da277d9ec0 2024-04-02T14:47:35.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0482000640 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277e3230 0x55da277ddd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0481600640 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277eb620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f047002f020 tx=0x7f0470005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34141cab587cc4cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.076+0000 7f0462a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.077+0000 7f0462a00640 1 -- 172.21.15.62:0/277598393 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047004e040 con 0x55da277d9ec0 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.077+0000 7f0462a00640 1 -- 172.21.15.62:0/277598393 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f04700344e0 con 0x55da277d9ec0 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.077+0000 7f0486c0f740 1 -- 172.21.15.62:0/277598393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f046809d300 con 0x55da277d9ec0 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.077+0000 7f0462a00640 1 -- 172.21.15.62:0/277598393 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f047002cde0 con 0x55da277d9ec0 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.077+0000 7f0486c0f740 10 client.16437 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.077+0000 7f0486c0f740 1 -- 172.21.15.62:0/277598393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55da278599c0 con 0x55da277d9ec0 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.077+0000 7f0462a00640 1 -- 172.21.15.62:0/277598393 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f046809d300 con 0x55da277d9ec0 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.078+0000 7f0462a00640 1 -- 172.21.15.62:0/277598393 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f047005e440 con 0x55da277d9ec0 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.078+0000 7f0486c0f740 10 client.16437 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.078+0000 7f0486c0f740 10 client.16437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.078+0000 7f0486c0f740 20 client.16437 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.078+0000 7f0486c0f740 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55da2785a0e0 0x55da2785c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.078+0000 7f0486c0f740 4 client.16437 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:47:35.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.078+0000 7f0486c0f740 1 -- 172.21.15.62:0/277598393 --> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da2774bc30 con 0x55da2785a0e0 2024-04-02T14:47:35.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.078+0000 7f0480c00640 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55da2785a0e0 0x55da2785c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.078+0000 7f0480c00640 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55da2785a0e0 0x55da2785c500 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db710a704708079a server_cookie=501ae719ec0a7448 in_seq=0 out_seq=0 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.078+0000 7f0462a00640 10 client.16437 ms_handle_connect on v2:172.21.15.62:6838/3188264140 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0462a00640 1 -- 172.21.15.62:0/277598393 <== mds.2 v2:172.21.15.62:6838/3188264140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+503 (crc 0 0 0) 0x55da2774bc30 con 0x55da2785a0e0 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0462a00640 10 client.16437 handle_command_reply: tid=0 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 1 client.16437 shutdown 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 1 -- 172.21.15.62:0/277598393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 msgr2=0x55da277eb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277eb620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f047002f020 tx=0x7f0470005740 comp rx=0 tx=0).stop 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 1 -- 172.21.15.62:0/277598393 shutdown_connections 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.62:6838/3188264140,v1:172.21.15.62:6839/3188264140] conn(0x55da2785a0e0 0x55da2785c500 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0462a00640 0 client.16437 ms_handle_reset on v2:172.21.15.62:6838/3188264140 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55da277e3230 0x55da277ddd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da277daba0 0x55da277dd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 1 --2- 172.21.15.62:0/277598393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da277d9ec0 0x55da277eb620 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 1 -- 172.21.15.62:0/277598393 >> 172.21.15.62:0/277598393 conn(0x55da277a0a50 msgr2=0x55da277e0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 1 -- 172.21.15.62:0/277598393 shutdown_connections 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 1 -- 172.21.15.62:0/277598393 wait complete. 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.079+0000 7f0486c0f740 20 client.16437 trim_cache size 0 max 16384 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 294003381, 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638951, 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/compartmentalisation.txt" 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:35.077 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:35.078 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:35.078 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 2019150572, 2024-04-02T14:47:35.078 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638952, 2024-04-02T14:47:35.078 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/ABI.txt" 2024-04-02T14:47:35.078 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:47:35.078 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-02T14:47:35.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 -- 172.21.15.62:0/800290192 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f04640a3da0 msgr2=0x7f04640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f04640a3da0 0x7f04640a6260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f04740117b0 tx=0x7f0474009500 comp rx=0 tx=0).stop 2024-04-02T14:47:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 -- 172.21.15.62:0/800290192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c108ec0 msgr2=0x7f047c19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c108ec0 0x7f047c19ab60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f046800e9c0 tx=0x7f046800ee80 comp rx=0 tx=0).stop 2024-04-02T14:47:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 -- 172.21.15.62:0/800290192 shutdown_connections 2024-04-02T14:47:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f04640a3da0 0x7f04640a6260 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047c19ee90 0x7f047c19f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f047c108ec0 0x7f047c19ab60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 --2- 172.21.15.62:0/800290192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c103df0 0x7f047c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 -- 172.21.15.62:0/800290192 >> 172.21.15.62:0/800290192 conn(0x7f047c0ff630 msgr2=0x7f047c101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 -- 172.21.15.62:0/800290192 shutdown_connections 2024-04-02T14:47:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.080+0000 7f0482a00640 1 -- 172.21.15.62:0/800290192 wait complete. 2024-04-02T14:47:35.092 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:47:35.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:34 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3925550261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:47:35.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.325+0000 7f341a600640 1 -- 172.21.15.62:0/1687080219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3414108ec0 msgr2=0x7f3414110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.325+0000 7f341a600640 1 --2- 172.21.15.62:0/1687080219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3414108ec0 0x7f3414110a90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f340c009be0 tx=0x7f340c02bef0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.326+0000 7f341a600640 1 -- 172.21.15.62:0/1687080219 shutdown_connections 2024-04-02T14:47:35.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.326+0000 7f341a600640 1 --2- 172.21.15.62:0/1687080219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3414108ec0 0x7f3414110a90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.326+0000 7f341a600640 1 --2- 172.21.15.62:0/1687080219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34141047a0 0x7f34141087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.326+0000 7f341a600640 1 --2- 172.21.15.62:0/1687080219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3414103df0 0x7f34141041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.326+0000 7f341a600640 1 -- 172.21.15.62:0/1687080219 >> 172.21.15.62:0/1687080219 conn(0x7f34140ff630 msgr2=0x7f3414101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.326+0000 7f341a600640 1 -- 172.21.15.62:0/1687080219 shutdown_connections 2024-04-02T14:47:35.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.326+0000 7f341a600640 1 -- 172.21.15.62:0/1687080219 wait complete. 2024-04-02T14:47:35.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.327+0000 7f341a600640 1 Processor -- start 2024-04-02T14:47:35.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.327+0000 7f341a600640 1 -- start start 2024-04-02T14:47:35.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.327+0000 7f341a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3414103df0 0x7f341419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.327+0000 7f341a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108ec0 0x7f341419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.327+0000 7f341a600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f341419edb0 0x7f341419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.327+0000 7f341a600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3414114090 con 0x7f341419edb0 2024-04-02T14:47:35.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.327+0000 7f341a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3414113f10 con 0x7f3414103df0 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.327+0000 7f341a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3414114210 con 0x7f3414108ec0 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f3419c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f341419edb0 0x7f341419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f3419200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3414103df0 0x7f341419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f340be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108ec0 0x7f341419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f3419200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3414103df0 0x7f341419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:38200/0 (socket says 172.21.15.62:38200) 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f3419c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f341419edb0 0x7f341419f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40222/0 (socket says 172.21.15.62:40222) 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f340be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108ec0 0x7f341419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50110/0 (socket says 172.21.15.62:50110) 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f3419200640 1 -- 172.21.15.62:0/4059407671 learned_addr learned my addr 172.21.15.62:0/4059407671 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f340be00640 1 -- 172.21.15.62:0/4059407671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3414103df0 msgr2=0x7f341419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f340be00640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3414103df0 0x7f341419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f340be00640 1 -- 172.21.15.62:0/4059407671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f341419edb0 msgr2=0x7f341419f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f340be00640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f341419edb0 0x7f341419f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f340be00640 1 -- 172.21.15.62:0/4059407671 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341419f8a0 con 0x7f3414108ec0 2024-04-02T14:47:35.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.328+0000 7f340be00640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108ec0 0x7f341419aa30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00e9c0 tx=0x7f33fc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.329+0000 7f3412000640 1 -- 172.21.15.62:0/4059407671 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc00ccc0 con 0x7f3414108ec0 2024-04-02T14:47:35.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.329+0000 7f341a600640 1 -- 172.21.15.62:0/4059407671 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341419fb90 con 0x7f3414108ec0 2024-04-02T14:47:35.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.329+0000 7f3412000640 1 -- 172.21.15.62:0/4059407671 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f33fc010930 con 0x7f3414108ec0 2024-04-02T14:47:35.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.329+0000 7f3412000640 1 -- 172.21.15.62:0/4059407671 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc018840 con 0x7f3414108ec0 2024-04-02T14:47:35.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.329+0000 7f341a600640 1 -- 172.21.15.62:0/4059407671 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34141a73d0 con 0x7f3414108ec0 2024-04-02T14:47:35.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.332+0000 7f341e98d740 1 Processor -- start 2024-04-02T14:47:35.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.332+0000 7f3412000640 1 -- 172.21.15.62:0/4059407671 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f33fc010ae0 con 0x7f3414108ec0 2024-04-02T14:47:35.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.332+0000 7f341e98d740 1 -- start start 2024-04-02T14:47:35.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f341e98d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1fba0 0x5625bef1e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f341e98d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1eec0 0x5625bef1f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f341e98d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef28230 0x5625bef2a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f341e98d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5625bee5f400 con 0x5625bef1eec0 2024-04-02T14:47:35.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f341e98d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5625bee43f10 con 0x5625bef28230 2024-04-02T14:47:35.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f341e98d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5625bee836f0 con 0x5625bef1fba0 2024-04-02T14:47:35.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f3419c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1fba0 0x5625bef1e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f3419200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1eec0 0x5625bef1f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f340be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef28230 0x5625bef2a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f3419c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1fba0 0x5625bef1e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50126/0 (socket says 172.21.15.62:50126) 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f3419c00640 1 -- 172.21.15.62:0/3364069498 learned_addr learned my addr 172.21.15.62:0/3364069498 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f3419c00640 1 -- 172.21.15.62:0/3364069498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef28230 msgr2=0x5625bef2a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f3419c00640 1 --2- 172.21.15.62:0/3364069498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef28230 0x5625bef2a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f3419c00640 1 -- 172.21.15.62:0/3364069498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1eec0 msgr2=0x5625bef1f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f3419c00640 1 --2- 172.21.15.62:0/3364069498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1eec0 0x5625bef1f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f3419c00640 1 -- 172.21.15.62:0/3364069498 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625beb973f0 con 0x5625bef1fba0 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.333+0000 7f3419200640 1 --2- 172.21.15.62:0/3364069498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1eec0 0x5625bef1f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.334+0000 7f3419c00640 1 --2- 172.21.15.62:0/3364069498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1fba0 0x5625bef1e8f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f340c009be0 tx=0x7f340c009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=749bfa3d86258e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.334+0000 7f3412000640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33f80a3da0 0x7f33f80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.334+0000 7f340a000640 1 -- 172.21.15.62:0/3364069498 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c043030 con 0x5625bef1fba0 2024-04-02T14:47:35.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.334+0000 7f340a000640 1 -- 172.21.15.62:0/3364069498 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f340c033c40 con 0x5625bef1fba0 2024-04-02T14:47:35.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.334+0000 7f340a000640 1 -- 172.21.15.62:0/3364069498 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c02c890 con 0x5625bef1fba0 2024-04-02T14:47:35.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.334+0000 7f3412000640 1 -- 172.21.15.62:0/4059407671 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f33fc0d0a80 con 0x7f3414108ec0 2024-04-02T14:47:35.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.334+0000 7f3419200640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33f80a3da0 0x7f33f80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.335+0000 7f341e98d740 1 -- 172.21.15.62:0/3364069498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1fba0 msgr2=0x5625bef1e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.335+0000 7f341e98d740 1 --2- 172.21.15.62:0/3364069498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1fba0 0x5625bef1e8f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f340c009be0 tx=0x7f340c009330 comp rx=0 tx=0).stop 2024-04-02T14:47:35.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.335+0000 7f341e98d740 1 -- 172.21.15.62:0/3364069498 shutdown_connections 2024-04-02T14:47:35.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.335+0000 7f341e98d740 1 --2- 172.21.15.62:0/3364069498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef28230 0x5625bef2a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.335+0000 7f341e98d740 1 --2- 172.21.15.62:0/3364069498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1eec0 0x5625bef1f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.335+0000 7f341e98d740 1 --2- 172.21.15.62:0/3364069498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1fba0 0x5625bef1e8f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.335+0000 7f341e98d740 1 -- 172.21.15.62:0/3364069498 >> 172.21.15.62:0/3364069498 conn(0x5625beee5a50 msgr2=0x5625bef19910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.335+0000 7f341e98d740 1 -- 172.21.15.62:0/3364069498 shutdown_connections 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.335+0000 7f3419200640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33f80a3da0 0x7f33f80a6260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5625bef200f0 tx=0x7f3404009260 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.335+0000 7f341e98d740 1 -- 172.21.15.62:0/3364069498 wait complete. 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.336+0000 7f341e98d740 1 -- start start 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.336+0000 7f341e98d740 1 Processor -- start 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.336+0000 7f341e98d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 0x5625bef25250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.336+0000 7f341e98d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef1fba0 0x5625bef25790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.336+0000 7f341e98d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef28230 0x5625bef25cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.336+0000 7f341e98d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5625bee7f6a0 con 0x5625bef28230 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f341e98d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5625bee2b980 con 0x5625bef1fba0 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f341e98d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5625bee3ecb0 con 0x5625bef1eec0 2024-04-02T14:47:35.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f341e98d740 10 client.0 _ll_register_callbacks cb 0x5625bee3e6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f3419c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef28230 0x5625bef25cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f3419200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 0x5625bef25250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f3419200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 0x5625bef25250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50132/0 (socket says 172.21.15.62:50132) 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f3419200640 1 -- 172.21.15.62:0/3459843429 learned_addr learned my addr 172.21.15.62:0/3459843429 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f340be00640 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef1fba0 0x5625bef25790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f3419c00640 1 -- 172.21.15.62:0/3459843429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 msgr2=0x5625bef25250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f3419c00640 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 0x5625bef25250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f3419c00640 1 -- 172.21.15.62:0/3459843429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef1fba0 msgr2=0x5625bef25790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f3419c00640 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef1fba0 0x5625bef25790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f3419c00640 1 -- 172.21.15.62:0/3459843429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625bec617d0 con 0x5625bef28230 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f3419200640 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 0x5625bef25250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.337+0000 7f3419c00640 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef28230 0x5625bef25cd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f340c033490 tx=0x7f340c0338e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3fe5ee912548f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.338+0000 7f3402a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:35.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.338+0000 7f3402a00640 1 -- 172.21.15.62:0/3459843429 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c043040 con 0x5625bef28230 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.338+0000 7f3402a00640 1 -- 172.21.15.62:0/3459843429 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f340c003970 con 0x5625bef28230 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.338+0000 7f341e98d740 1 -- 172.21.15.62:0/3459843429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f340c033c40 con 0x5625bef28230 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.338+0000 7f3402a00640 1 -- 172.21.15.62:0/3459843429 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f340c02c890 con 0x5625bef28230 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.338+0000 7f341e98d740 10 client.16449 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.338+0000 7f341e98d740 1 -- 172.21.15.62:0/3459843429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5625bef370b0 con 0x5625bef28230 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.338+0000 7f3402a00640 1 -- 172.21.15.62:0/3459843429 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f340c033c40 con 0x5625bef28230 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.339+0000 7f3402a00640 1 -- 172.21.15.62:0/3459843429 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f340c03f440 con 0x5625bef28230 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.339+0000 7f341e98d740 10 client.16449 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.339+0000 7f341e98d740 10 client.16449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.339+0000 7f341e98d740 20 client.16449 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.339+0000 7f341e98d740 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5625bef38400 0x5625bef3a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.339+0000 7f341e98d740 4 client.16449 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.339+0000 7f341e98d740 1 -- 172.21.15.62:0/3459843429 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625bee43040 con 0x5625bef38400 2024-04-02T14:47:35.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.339+0000 7f3419200640 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5625bef38400 0x5625bef3a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.340+0000 7f3419200640 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5625bef38400 0x5625bef3a7f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28d23d2a62131af5 server_cookie=218962b67f3a91f4 in_seq=0 out_seq=0 2024-04-02T14:47:35.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.340+0000 7f3402a00640 10 client.16449 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:47:35.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.340+0000 7f3402a00640 1 -- 172.21.15.62:0/3459843429 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625bee43040 con 0x5625bef38400 2024-04-02T14:47:35.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.340+0000 7f3402a00640 10 client.16449 handle_command_reply: tid=0 2024-04-02T14:47:35.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 1 client.16449 shutdown 2024-04-02T14:47:35.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 1 -- 172.21.15.62:0/3459843429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef28230 msgr2=0x5625bef25cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef28230 0x5625bef25cd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f340c033490 tx=0x7f340c0338e0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 1 -- 172.21.15.62:0/3459843429 shutdown_connections 2024-04-02T14:47:35.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5625bef38400 0x5625bef3a7f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef28230 0x5625bef25cd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef1fba0 0x5625bef25790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 1 --2- 172.21.15.62:0/3459843429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 0x5625bef25250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f3402a00640 0 client.16449 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:47:35.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 1 -- 172.21.15.62:0/3459843429 >> 172.21.15.62:0/3459843429 conn(0x5625beee5a50 msgr2=0x5625bef1b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 1 -- 172.21.15.62:0/3459843429 shutdown_connections 2024-04-02T14:47:35.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 1 -- 172.21.15.62:0/3459843429 wait complete. 2024-04-02T14:47:35.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.341+0000 7f341e98d740 20 client.16449 trim_cache size 0 max 16384 2024-04-02T14:47:35.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.356+0000 7f341e98d740 1 Processor -- start 2024-04-02T14:47:35.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.356+0000 7f341e98d740 1 -- start start 2024-04-02T14:47:35.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f341e98d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef1eec0 0x5625bef27980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f341e98d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef25af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f341e98d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef28230 0x5625bef33860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f341e98d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34141af770 con 0x5625bef1fba0 2024-04-02T14:47:35.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f341e98d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f34141b1460 con 0x5625bef1eec0 2024-04-02T14:47:35.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f341e98d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34141afea0 con 0x5625bef28230 2024-04-02T14:47:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f340be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef25af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f340be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef25af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40252/0 (socket says 172.21.15.62:40252) 2024-04-02T14:47:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f340be00640 1 -- 172.21.15.62:0/3488965819 learned_addr learned my addr 172.21.15.62:0/3488965819 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f3419200640 1 --2- 172.21.15.62:0/3488965819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef1eec0 0x5625bef27980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f3419c00640 1 --2- 172.21.15.62:0/3488965819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef28230 0x5625bef33860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f340be00640 1 -- 172.21.15.62:0/3488965819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef28230 msgr2=0x5625bef33860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f340be00640 1 --2- 172.21.15.62:0/3488965819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef28230 0x5625bef33860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.357+0000 7f340be00640 1 -- 172.21.15.62:0/3488965819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef1eec0 msgr2=0x5625bef27980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.358+0000 7f340be00640 1 --2- 172.21.15.62:0/3488965819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef1eec0 0x5625bef27980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.358+0000 7f340be00640 1 -- 172.21.15.62:0/3488965819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625bef237a0 con 0x5625bef1fba0 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.358+0000 7f3419200640 1 --2- 172.21.15.62:0/3488965819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef1eec0 0x5625bef27980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.358+0000 7f340be00640 1 --2- 172.21.15.62:0/3488965819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef25af0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0d42d0 tx=0x7f33fc087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b6e6fac71507d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.358+0000 7f3409600640 1 -- 172.21.15.62:0/3488965819 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0da040 con 0x5625bef1fba0 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.358+0000 7f3409600640 1 -- 172.21.15.62:0/3488965819 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f33fc09cac0 con 0x5625bef1fba0 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.358+0000 7f3409600640 1 -- 172.21.15.62:0/3488965819 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0ce440 con 0x5625bef1fba0 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.359+0000 7f341e98d740 1 -- 172.21.15.62:0/3488965819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 msgr2=0x5625bef25af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.359+0000 7f341e98d740 1 --2- 172.21.15.62:0/3488965819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef25af0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0d42d0 tx=0x7f33fc087040 comp rx=0 tx=0).stop 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.359+0000 7f341e98d740 1 -- 172.21.15.62:0/3488965819 shutdown_connections 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.359+0000 7f341e98d740 1 --2- 172.21.15.62:0/3488965819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef28230 0x5625bef33860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.359+0000 7f341e98d740 1 --2- 172.21.15.62:0/3488965819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef25af0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.359+0000 7f341e98d740 1 --2- 172.21.15.62:0/3488965819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef1eec0 0x5625bef27980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.359+0000 7f341e98d740 1 -- 172.21.15.62:0/3488965819 >> 172.21.15.62:0/3488965819 conn(0x5625beee5a50 msgr2=0x5625bef33150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.359+0000 7f341e98d740 1 -- 172.21.15.62:0/3488965819 shutdown_connections 2024-04-02T14:47:35.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.359+0000 7f341e98d740 1 -- 172.21.15.62:0/3488965819 wait complete. 2024-04-02T14:47:35.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.359+0000 7f341e98d740 1 -- start start 2024-04-02T14:47:35.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f341e98d740 1 Processor -- start 2024-04-02T14:47:35.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f341e98d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 0x5625bef392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f341e98d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef39800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f341e98d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef28230 0x5625bef39d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f341e98d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34141b0cd0 con 0x5625bef1fba0 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f341e98d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f34141b05a0 con 0x5625bef28230 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f341e98d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34141ad750 con 0x5625bef1eec0 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f341e98d740 10 client.0 _ll_register_callbacks cb 0x7f340c009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f340be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef39800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f3419c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef28230 0x5625bef39d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f340be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef39800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40260/0 (socket says 172.21.15.62:40260) 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f340be00640 1 -- 172.21.15.62:0/1081721297 learned_addr learned my addr 172.21.15.62:0/1081721297 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.360+0000 7f3419200640 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 0x5625bef392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f340be00640 1 -- 172.21.15.62:0/1081721297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 msgr2=0x5625bef392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f340be00640 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 0x5625bef392c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f340be00640 1 -- 172.21.15.62:0/1081721297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef28230 msgr2=0x5625bef39d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f340be00640 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef28230 0x5625bef39d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f340be00640 1 -- 172.21.15.62:0/1081721297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625bef9d650 con 0x5625bef1fba0 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f340be00640 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef39800 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0a5150 tx=0x7f33fc009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64b474ad74116ec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f3402a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f3402a00640 1 -- 172.21.15.62:0/1081721297 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc09c600 con 0x5625bef1fba0 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f3402a00640 1 -- 172.21.15.62:0/1081721297 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f33fc0a53f0 con 0x5625bef1fba0 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f341e98d740 1 -- 172.21.15.62:0/1081721297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33fc09cac0 con 0x5625bef1fba0 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f3402a00640 1 -- 172.21.15.62:0/1081721297 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f33fc0ce650 con 0x5625bef1fba0 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.361+0000 7f341e98d740 10 client.16455 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.362+0000 7f341e98d740 1 -- 172.21.15.62:0/1081721297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5625bef9e930 con 0x5625bef1fba0 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.362+0000 7f3402a00640 1 -- 172.21.15.62:0/1081721297 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f33fc09cac0 con 0x5625bef1fba0 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.362+0000 7f3402a00640 1 -- 172.21.15.62:0/1081721297 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f33fc09b980 con 0x5625bef1fba0 2024-04-02T14:47:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.362+0000 7f341e98d740 10 client.16455 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.362+0000 7f341e98d740 10 client.16455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.362+0000 7f341e98d740 20 client.16455 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.362+0000 7f341e98d740 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5625bef9f060 0x5625befa1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.362+0000 7f341e98d740 4 client.16455 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.362+0000 7f341e98d740 1 -- 172.21.15.62:0/1081721297 --> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625bee90c30 con 0x5625bef9f060 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.362+0000 7f3419200640 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5625bef9f060 0x5625befa1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.363+0000 7f3419200640 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5625bef9f060 0x5625befa1480 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51464e2bf85476af server_cookie=b252d0c76cc93446 in_seq=0 out_seq=0 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.363+0000 7f3402a00640 10 client.16455 ms_handle_connect on v2:172.21.15.62:6840/450176372 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.363+0000 7f3402a00640 1 -- 172.21.15.62:0/1081721297 <== mds.1 v2:172.21.15.62:6840/450176372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1472 (crc 0 0 0) 0x5625bee90c30 con 0x5625bef9f060 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.363+0000 7f3402a00640 10 client.16455 handle_command_reply: tid=0 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.363+0000 7f341e98d740 1 client.16455 shutdown 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.363+0000 7f341e98d740 1 -- 172.21.15.62:0/1081721297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 msgr2=0x5625bef39800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.363+0000 7f341e98d740 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef39800 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0a5150 tx=0x7f33fc009700 comp rx=0 tx=0).stop 2024-04-02T14:47:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.363+0000 7f341e98d740 1 -- 172.21.15.62:0/1081721297 shutdown_connections 2024-04-02T14:47:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.363+0000 7f341e98d740 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.62:6840/450176372,v1:172.21.15.62:6841/450176372] conn(0x5625bef9f060 0x5625befa1480 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f341e98d740 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5625bef28230 0x5625bef39d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f341e98d740 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5625bef1fba0 0x5625bef39800 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f341e98d740 1 --2- 172.21.15.62:0/1081721297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625bef1eec0 0x5625bef392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f3402a00640 0 client.16455 ms_handle_reset on v2:172.21.15.62:6840/450176372 2024-04-02T14:47:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f341e98d740 1 -- 172.21.15.62:0/1081721297 >> 172.21.15.62:0/1081721297 conn(0x5625beee5a50 msgr2=0x5625bef18af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f341e98d740 1 -- 172.21.15.62:0/1081721297 shutdown_connections 2024-04-02T14:47:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f341e98d740 1 -- 172.21.15.62:0/1081721297 wait complete. 2024-04-02T14:47:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f341e98d740 20 client.16455 trim_cache size 0 max 16384 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 1074492612, 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638939, 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/omap-hdq" 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 1510157187, 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638941, 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/mxc-w1" 2024-04-02T14:47:35.363 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 1590141041, 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638938, 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/w1-gpio" 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3629657775, 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638940, 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2482" 2024-04-02T14:47:35.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3660942034, 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638936, 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/00-INDEX" 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3812859424, 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511638937, 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2490" 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:47:35.365 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-02T14:47:35.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f341a600640 1 -- 172.21.15.62:0/4059407671 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33f80a3da0 msgr2=0x7f33f80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f341a600640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33f80a3da0 0x7f33f80a6260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5625bef200f0 tx=0x7f3404009260 comp rx=0 tx=0).stop 2024-04-02T14:47:35.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f341a600640 1 -- 172.21.15.62:0/4059407671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108ec0 msgr2=0x7f341419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.364+0000 7f341a600640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108ec0 0x7f341419aa30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00e9c0 tx=0x7f33fc00ee80 comp rx=0 tx=0).stop 2024-04-02T14:47:35.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.365+0000 7f341a600640 1 -- 172.21.15.62:0/4059407671 shutdown_connections 2024-04-02T14:47:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.365+0000 7f341a600640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33f80a3da0 0x7f33f80a6260 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.365+0000 7f341a600640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f341419edb0 0x7f341419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.365+0000 7f341a600640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108ec0 0x7f341419aa30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.365+0000 7f341a600640 1 --2- 172.21.15.62:0/4059407671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3414103df0 0x7f341419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.365+0000 7f341a600640 1 -- 172.21.15.62:0/4059407671 >> 172.21.15.62:0/4059407671 conn(0x7f34140ff630 msgr2=0x7f3414100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.365+0000 7f341a600640 1 -- 172.21.15.62:0/4059407671 shutdown_connections 2024-04-02T14:47:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.365+0000 7f341a600640 1 -- 172.21.15.62:0/4059407671 wait complete. 2024-04-02T14:47:35.377 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:47:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.602+0000 7f541be00640 1 -- 172.21.15.62:0/3943051932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c1082b0 msgr2=0x7f541c108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.602+0000 7f541be00640 1 --2- 172.21.15.62:0/3943051932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c1082b0 0x7f541c108690 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f540c009ca0 tx=0x7f540c02c100 comp rx=0 tx=0).stop 2024-04-02T14:47:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.602+0000 7f541be00640 1 -- 172.21.15.62:0/3943051932 shutdown_connections 2024-04-02T14:47:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.602+0000 7f541be00640 1 --2- 172.21.15.62:0/3943051932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f541c10d300 0x7f541c114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.602+0000 7f541be00640 1 --2- 172.21.15.62:0/3943051932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f541c108c60 0x7f541c10cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.602+0000 7f541be00640 1 --2- 172.21.15.62:0/3943051932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c1082b0 0x7f541c108690 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.602+0000 7f541be00640 1 -- 172.21.15.62:0/3943051932 >> 172.21.15.62:0/3943051932 conn(0x7f541c077920 msgr2=0x7f541c106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.603+0000 7f541be00640 1 -- 172.21.15.62:0/3943051932 shutdown_connections 2024-04-02T14:47:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.603+0000 7f541be00640 1 -- 172.21.15.62:0/3943051932 wait complete. 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.604+0000 7f541be00640 1 Processor -- start 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.604+0000 7f541be00640 1 -- start start 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.604+0000 7f541be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f541c1082b0 0x7f541c1a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.604+0000 7f541be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c10d300 0x7f541c1a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.604+0000 7f541be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f541c19e750 0x7f541c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.604+0000 7f541be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1184f0 con 0x7f541c19e750 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.604+0000 7f541be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f541c118370 con 0x7f541c10d300 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.604+0000 7f541be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f541c118670 con 0x7f541c1082b0 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.604+0000 7f541a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c10d300 0x7f541c1a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.604+0000 7f541a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c10d300 0x7f541c1a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:38244/0 (socket says 172.21.15.62:38244) 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.604+0000 7f541a000640 1 -- 172.21.15.62:0/2265564896 learned_addr learned my addr 172.21.15.62:0/2265564896 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.605+0000 7f541b400640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f541c19e750 0x7f541c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.605+0000 7f541aa00640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f541c1082b0 0x7f541c1a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.605+0000 7f541a000640 1 -- 172.21.15.62:0/2265564896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f541c1082b0 msgr2=0x7f541c1a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.605+0000 7f541a000640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f541c1082b0 0x7f541c1a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.605+0000 7f541a000640 1 -- 172.21.15.62:0/2265564896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f541c19e750 msgr2=0x7f541c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.605+0000 7f541a000640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f541c19e750 0x7f541c19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.605+0000 7f541a000640 1 -- 172.21.15.62:0/2265564896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c19f350 con 0x7f541c10d300 2024-04-02T14:47:35.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.605+0000 7f541a000640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c10d300 0x7f541c1a4bd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f540400e9f0 tx=0x7f540400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.605+0000 7f5413400640 1 -- 172.21.15.62:0/2265564896 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540400ccc0 con 0x7f541c10d300 2024-04-02T14:47:35.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.606+0000 7f541be00640 1 -- 172.21.15.62:0/2265564896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c1ab430 con 0x7f541c10d300 2024-04-02T14:47:35.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.606+0000 7f5413400640 1 -- 172.21.15.62:0/2265564896 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5404010930 con 0x7f541c10d300 2024-04-02T14:47:35.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.606+0000 7f5413400640 1 -- 172.21.15.62:0/2265564896 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404018a30 con 0x7f541c10d300 2024-04-02T14:47:35.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.606+0000 7f541be00640 1 -- 172.21.15.62:0/2265564896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c1ab890 con 0x7f541c10d300 2024-04-02T14:47:35.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.609+0000 7f542441b740 1 Processor -- start 2024-04-02T14:47:35.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.609+0000 7f542441b740 1 -- start start 2024-04-02T14:47:35.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.609+0000 7f542441b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.609+0000 7f542441b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de36ec0 0x55849de37340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.609+0000 7f542441b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de42620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.609+0000 7f542441b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55849dd77400 con 0x55849de37ba0 2024-04-02T14:47:35.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.609+0000 7f542441b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55849dd5bf10 con 0x55849de40230 2024-04-02T14:47:35.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.609+0000 7f542441b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55849dd9b6f0 con 0x55849de36ec0 2024-04-02T14:47:35.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.609+0000 7f5413400640 1 -- 172.21.15.62:0/2265564896 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5404010ae0 con 0x7f541c10d300 2024-04-02T14:47:35.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.610+0000 7f541b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.610+0000 7f541aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de36ec0 0x55849de37340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.610+0000 7f541aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de36ec0 0x55849de37340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50170/0 (socket says 172.21.15.62:50170) 2024-04-02T14:47:35.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.610+0000 7f541b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de368f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40280/0 (socket says 172.21.15.62:40280) 2024-04-02T14:47:35.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.610+0000 7f541aa00640 1 -- 172.21.15.62:0/2828748022 learned_addr learned my addr 172.21.15.62:0/2828748022 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.610+0000 7f541a000640 1 --2- 172.21.15.62:0/2828748022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de42620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.610+0000 7f541b400640 1 -- 172.21.15.62:0/2828748022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de36ec0 msgr2=0x55849de37340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.610+0000 7f541b400640 1 --2- 172.21.15.62:0/2828748022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de36ec0 0x55849de37340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.610+0000 7f541b400640 1 -- 172.21.15.62:0/2828748022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 msgr2=0x55849de42620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.610+0000 7f541b400640 1 --2- 172.21.15.62:0/2828748022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de42620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.610+0000 7f541b400640 1 -- 172.21.15.62:0/2828748022 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55849daaf3f0 con 0x55849de37ba0 2024-04-02T14:47:35.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.611+0000 7f541b400640 1 --2- 172.21.15.62:0/2828748022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de368f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f54140045b0 tx=0x7f541400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdf27c949e17244 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.611+0000 7f540b400640 1 -- 172.21.15.62:0/2828748022 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414003a70 con 0x55849de37ba0 2024-04-02T14:47:35.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.611+0000 7f540b400640 1 -- 172.21.15.62:0/2828748022 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5414015070 con 0x55849de37ba0 2024-04-02T14:47:35.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.611+0000 7f540b400640 1 -- 172.21.15.62:0/2828748022 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414004d90 con 0x55849de37ba0 2024-04-02T14:47:35.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.611+0000 7f5413400640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f54000a3d20 0x7f54000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.611+0000 7f5413400640 1 -- 172.21.15.62:0/2265564896 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f54040d0ad0 con 0x7f541c10d300 2024-04-02T14:47:35.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.611+0000 7f541aa00640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f54000a3d20 0x7f54000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.612+0000 7f542441b740 1 -- 172.21.15.62:0/2828748022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 msgr2=0x55849de368f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.612+0000 7f542441b740 1 --2- 172.21.15.62:0/2828748022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de368f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f54140045b0 tx=0x7f541400ba90 comp rx=0 tx=0).stop 2024-04-02T14:47:35.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.612+0000 7f542441b740 1 -- 172.21.15.62:0/2828748022 shutdown_connections 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.612+0000 7f542441b740 1 --2- 172.21.15.62:0/2828748022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de42620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.612+0000 7f542441b740 1 --2- 172.21.15.62:0/2828748022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de36ec0 0x55849de37340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.612+0000 7f542441b740 1 --2- 172.21.15.62:0/2828748022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de368f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.612+0000 7f542441b740 1 -- 172.21.15.62:0/2828748022 >> 172.21.15.62:0/2828748022 conn(0x55849ddfda50 msgr2=0x55849de31910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.612+0000 7f542441b740 1 -- 172.21.15.62:0/2828748022 shutdown_connections 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.612+0000 7f542441b740 1 -- 172.21.15.62:0/2828748022 wait complete. 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.612+0000 7f542441b740 1 -- start start 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.612+0000 7f541aa00640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f54000a3d20 0x7f54000a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f540c006010 tx=0x7f540c009360 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f542441b740 1 Processor -- start 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f542441b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 0x55849de3d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f542441b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de3d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f542441b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de3dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f542441b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55849dd976a0 con 0x55849de36ec0 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f542441b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55849dd43980 con 0x55849de40230 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f542441b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55849dd56cb0 con 0x55849de37ba0 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f542441b740 10 client.0 _ll_register_callbacks cb 0x55849dd566a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f541a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de3d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f541aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 0x55849de3d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f541a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de3d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50174/0 (socket says 172.21.15.62:50174) 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.613+0000 7f541a000640 1 -- 172.21.15.62:0/4280790536 learned_addr learned my addr 172.21.15.62:0/4280790536 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.614+0000 7f541b400640 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de3dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.614+0000 7f541a000640 1 -- 172.21.15.62:0/4280790536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 msgr2=0x55849de3dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.614+0000 7f541a000640 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de3dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.614+0000 7f541a000640 1 -- 172.21.15.62:0/4280790536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 msgr2=0x55849de3d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.614+0000 7f541a000640 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 0x55849de3d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.614+0000 7f541a000640 1 -- 172.21.15.62:0/4280790536 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55849db797d0 con 0x55849de37ba0 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.614+0000 7f541a000640 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de3d770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5404098f70 tx=0x7f5404098ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fcda7516e127aed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.614+0000 7f53ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-04-02T14:47:35.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.614+0000 7f53ffe00640 1 -- 172.21.15.62:0/4280790536 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54040d7060 con 0x55849de37ba0 2024-04-02T14:47:35.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.614+0000 7f53ffe00640 1 -- 172.21.15.62:0/4280790536 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f5404018bd0 con 0x55849de37ba0 2024-04-02T14:47:35.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.614+0000 7f542441b740 1 -- 172.21.15.62:0/4280790536 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5414015070 con 0x55849de37ba0 2024-04-02T14:47:35.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.615+0000 7f53ffe00640 1 -- 172.21.15.62:0/4280790536 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f54040ce690 con 0x55849de37ba0 2024-04-02T14:47:35.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.615+0000 7f542441b740 10 client.26560 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:35.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.615+0000 7f542441b740 1 -- 172.21.15.62:0/4280790536 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55849de4f130 con 0x55849de37ba0 2024-04-02T14:47:35.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.615+0000 7f53ffe00640 1 -- 172.21.15.62:0/4280790536 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5414015070 con 0x55849de37ba0 2024-04-02T14:47:35.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.615+0000 7f53ffe00640 1 -- 172.21.15.62:0/4280790536 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f540409c080 con 0x55849de37ba0 2024-04-02T14:47:35.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.615+0000 7f542441b740 10 client.26560 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:35.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.615+0000 7f542441b740 10 client.26560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:47:35.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.615+0000 7f542441b740 20 client.26560 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:35.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.615+0000 7f542441b740 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55849de505f0 0x55849de52a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.615+0000 7f542441b740 4 client.26560 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:47:35.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.615+0000 7f542441b740 1 -- 172.21.15.62:0/4280790536 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55849dd5b040 con 0x55849de505f0 2024-04-02T14:47:35.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.616+0000 7f541aa00640 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55849de505f0 0x55849de52a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.616+0000 7f541aa00640 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55849de505f0 0x55849de52a60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e13b0f985419798 server_cookie=dd39b7a77bad4054 in_seq=0 out_seq=0 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.616+0000 7f53ffe00640 10 client.26560 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.617+0000 7f53ffe00640 1 -- 172.21.15.62:0/4280790536 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55849dd5b040 con 0x55849de505f0 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.617+0000 7f53ffe00640 10 client.26560 handle_command_reply: tid=0 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.617+0000 7f542441b740 1 client.26560 shutdown 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.617+0000 7f542441b740 1 -- 172.21.15.62:0/4280790536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 msgr2=0x55849de3d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.617+0000 7f542441b740 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de3d770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5404098f70 tx=0x7f5404098ee0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.618+0000 7f542441b740 1 -- 172.21.15.62:0/4280790536 shutdown_connections 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.618+0000 7f542441b740 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55849de505f0 0x55849de52a60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.618+0000 7f542441b740 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de3dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.618+0000 7f542441b740 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de3d770 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.618+0000 7f53ffe00640 0 client.26560 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.618+0000 7f542441b740 1 --2- 172.21.15.62:0/4280790536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 0x55849de3d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.618+0000 7f542441b740 1 -- 172.21.15.62:0/4280790536 >> 172.21.15.62:0/4280790536 conn(0x55849ddfda50 msgr2=0x55849de33660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.618+0000 7f542441b740 1 -- 172.21.15.62:0/4280790536 shutdown_connections 2024-04-02T14:47:35.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.618+0000 7f542441b740 1 -- 172.21.15.62:0/4280790536 wait complete. 2024-04-02T14:47:35.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.618+0000 7f542441b740 20 client.26560 trim_cache size 0 max 16384 2024-04-02T14:47:35.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f542441b740 1 Processor -- start 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f542441b740 1 -- start start 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f542441b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 0x55849de3f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f542441b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de4bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f542441b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de33400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f542441b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1b3c70 con 0x55849de36ec0 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f542441b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1b5960 con 0x55849de40230 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f542441b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1b43a0 con 0x55849de37ba0 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f541aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 0x55849de3f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f541a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de4bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f541a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de4bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:50182/0 (socket says 172.21.15.62:50182) 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f541a000640 1 -- 172.21.15.62:0/341205976 learned_addr learned my addr 172.21.15.62:0/341205976 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f541aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 0x55849de3f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40298/0 (socket says 172.21.15.62:40298) 2024-04-02T14:47:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.633+0000 7f541b400640 1 --2- 172.21.15.62:0/341205976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de33400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.634+0000 7f541aa00640 1 -- 172.21.15.62:0/341205976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 msgr2=0x55849de4bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.634+0000 7f541aa00640 1 --2- 172.21.15.62:0/341205976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de4bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.634+0000 7f541aa00640 1 -- 172.21.15.62:0/341205976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 msgr2=0x55849de33400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.634+0000 7f541aa00640 1 --2- 172.21.15.62:0/341205976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de33400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.634+0000 7f541aa00640 1 -- 172.21.15.62:0/341205976 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55849de2d1d0 con 0x55849de36ec0 2024-04-02T14:47:35.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.634+0000 7f541a000640 1 --2- 172.21.15.62:0/341205976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de4bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:47:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.634+0000 7f541aa00640 1 --2- 172.21.15.62:0/341205976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 0x55849de3f960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f540c009430 tx=0x7f540c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8e366cd0f7efaa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.634+0000 7f540aa00640 1 -- 172.21.15.62:0/341205976 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c039040 con 0x55849de36ec0 2024-04-02T14:47:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.634+0000 7f540aa00640 1 -- 172.21.15.62:0/341205976 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f540c058070 con 0x55849de36ec0 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.634+0000 7f540aa00640 1 -- 172.21.15.62:0/341205976 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c0042a0 con 0x55849de36ec0 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.635+0000 7f542441b740 1 -- 172.21.15.62:0/341205976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 msgr2=0x55849de3f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.635+0000 7f542441b740 1 --2- 172.21.15.62:0/341205976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 0x55849de3f960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f540c009430 tx=0x7f540c041000 comp rx=0 tx=0).stop 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.635+0000 7f542441b740 1 -- 172.21.15.62:0/341205976 shutdown_connections 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.635+0000 7f542441b740 1 --2- 172.21.15.62:0/341205976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de40230 0x55849de33400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.635+0000 7f542441b740 1 --2- 172.21.15.62:0/341205976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de37ba0 0x55849de4bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.635+0000 7f542441b740 1 --2- 172.21.15.62:0/341205976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de36ec0 0x55849de3f960 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.635+0000 7f542441b740 1 -- 172.21.15.62:0/341205976 >> 172.21.15.62:0/341205976 conn(0x55849ddfda50 msgr2=0x55849de32cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.635+0000 7f542441b740 1 -- 172.21.15.62:0/341205976 shutdown_connections 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.635+0000 7f542441b740 1 -- 172.21.15.62:0/341205976 wait complete. 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.636+0000 7f542441b740 1 -- start start 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.636+0000 7f542441b740 1 Processor -- start 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.636+0000 7f542441b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de36ec0 0x55849de41110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.636+0000 7f542441b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de41650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.636+0000 7f542441b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de40230 0x55849de41b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.636+0000 7f542441b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1b51d0 con 0x55849de37ba0 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.636+0000 7f542441b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1b4aa0 con 0x55849de36ec0 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.636+0000 7f542441b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1b1c70 con 0x55849de40230 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.636+0000 7f542441b740 10 client.0 _ll_register_callbacks cb 0x7f54040096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.636+0000 7f541a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de41650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f541a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de41650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40308/0 (socket says 172.21.15.62:40308) 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f541a000640 1 -- 172.21.15.62:0/1829390349 learned_addr learned my addr 172.21.15.62:0/1829390349 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f541b400640 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de40230 0x55849de41b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f541aa00640 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de36ec0 0x55849de41110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f541a000640 1 -- 172.21.15.62:0/1829390349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de40230 msgr2=0x55849de41b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f541a000640 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de40230 0x55849de41b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f541a000640 1 -- 172.21.15.62:0/1829390349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de36ec0 msgr2=0x55849de41110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f541a000640 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de36ec0 0x55849de41110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f541a000640 1 -- 172.21.15.62:0/1829390349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55849de46850 con 0x55849de37ba0 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f541a000640 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de41650 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5404099490 tx=0x7f5404018e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e1cf90ea7dee82 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f53ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f53ffe00640 1 -- 172.21.15.62:0/1829390349 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54040d4020 con 0x55849de37ba0 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f53ffe00640 1 -- 172.21.15.62:0/1829390349 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8440+0+0 (secure 0 0 0) 0x7f5404010430 con 0x55849de37ba0 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.637+0000 7f542441b740 1 -- 172.21.15.62:0/1829390349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f540c058070 con 0x55849de37ba0 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.638+0000 7f53ffe00640 1 -- 172.21.15.62:0/1829390349 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f540409cb10 con 0x55849de37ba0 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.638+0000 7f542441b740 10 client.16473 fetch_fsmap learned FSMap version 39 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.638+0000 7f542441b740 1 -- 172.21.15.62:0/1829390349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55849deb7130 con 0x55849de37ba0 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.638+0000 7f53ffe00640 1 -- 172.21.15.62:0/1829390349 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f540c058070 con 0x55849de37ba0 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.638+0000 7f53ffe00640 1 -- 172.21.15.62:0/1829390349 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f54040a1fa0 con 0x55849de37ba0 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.638+0000 7f542441b740 10 client.16473 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T14:47:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.638+0000 7f542441b740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:47:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.638+0000 7f542441b740 20 client.16473 populate_metadata read hostname 'smithi062' 2024-04-02T14:47:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.638+0000 7f542441b740 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55849deb79c0 0x55849deb9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:47:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.638+0000 7f542441b740 4 client.16473 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:47:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.638+0000 7f542441b740 1 -- 172.21.15.62:0/1829390349 --> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55849dda8c30 con 0x55849deb79c0 2024-04-02T14:47:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.639+0000 7f541aa00640 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55849deb79c0 0x55849deb9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:47:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.639+0000 7f541aa00640 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55849deb79c0 0x55849deb9de0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2898e0c277352caa server_cookie=30ccd43aebd03f69 in_seq=0 out_seq=0 2024-04-02T14:47:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.639+0000 7f53ffe00640 10 client.16473 ms_handle_connect on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.639+0000 7f53ffe00640 1 -- 172.21.15.62:0/1829390349 <== mds.0 v2:172.21.15.113:6838/1791174899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55849dda8c30 con 0x55849deb79c0 2024-04-02T14:47:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.639+0000 7f53ffe00640 10 client.16473 handle_command_reply: tid=0 2024-04-02T14:47:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.639+0000 7f542441b740 1 client.16473 shutdown 2024-04-02T14:47:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f542441b740 1 -- 172.21.15.62:0/1829390349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 msgr2=0x55849de41650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f542441b740 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de41650 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5404099490 tx=0x7f5404018e80 comp rx=0 tx=0).stop 2024-04-02T14:47:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f542441b740 1 -- 172.21.15.62:0/1829390349 shutdown_connections 2024-04-02T14:47:35.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f542441b740 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.113:6838/1791174899,v1:172.21.15.113:6839/1791174899] conn(0x55849deb79c0 0x55849deb9de0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f542441b740 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55849de40230 0x55849de41b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f542441b740 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55849de37ba0 0x55849de41650 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f53ffe00640 0 client.16473 ms_handle_reset on v2:172.21.15.113:6838/1791174899 2024-04-02T14:47:35.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f542441b740 1 --2- 172.21.15.62:0/1829390349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55849de36ec0 0x55849de41110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f542441b740 1 -- 172.21.15.62:0/1829390349 >> 172.21.15.62:0/1829390349 conn(0x55849ddfda50 msgr2=0x55849de3d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f542441b740 1 -- 172.21.15.62:0/1829390349 shutdown_connections 2024-04-02T14:47:35.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f542441b740 1 -- 172.21.15.62:0/1829390349 wait complete. 2024-04-02T14:47:35.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.640+0000 7f542441b740 20 client.16473 trim_cache size 0 max 16384 2024-04-02T14:47:35.638 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-02T14:47:35.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 -- 172.21.15.62:0/2265564896 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f54000a3d20 msgr2=0x7f54000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f54000a3d20 0x7f54000a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f540c006010 tx=0x7f540c009360 comp rx=0 tx=0).stop 2024-04-02T14:47:35.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 -- 172.21.15.62:0/2265564896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c10d300 msgr2=0x7f541c1a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:47:35.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c10d300 0x7f541c1a4bd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f540400e9f0 tx=0x7f540400eeb0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 -- 172.21.15.62:0/2265564896 shutdown_connections 2024-04-02T14:47:35.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f54000a3d20 0x7f54000a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f541c19e750 0x7f541c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c10d300 0x7f541c1a4bd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 --2- 172.21.15.62:0/2265564896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f541c1082b0 0x7f541c1a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:47:35.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 -- 172.21.15.62:0/2265564896 >> 172.21.15.62:0/2265564896 conn(0x7f541c077920 msgr2=0x7f541c077f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:47:35.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 -- 172.21.15.62:0/2265564896 shutdown_connections 2024-04-02T14:47:35.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:47:35.641+0000 7f541be00640 1 -- 172.21.15.62:0/2265564896 wait complete. 2024-04-02T14:47:35.653 ERROR:tasks.fwd_scrub:rank 4 damaged: [{'damage_type': 'backtrace', 'id': 597677847, 'ino': 1099511638568, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/Makefile'}, {'damage_type': 'backtrace', 'id': 2465499163, 'ino': 1099511638569, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c'}, {'damage_type': 'backtrace', 'id': 2749257382, 'ino': 1099511638964, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt'}, {'damage_type': 'backtrace', 'id': 2938148192, 'ino': 1099511639521, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt'}, {'damage_type': 'backtrace', 'id': 3164408192, 'ino': 1099511638959, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt'}, {'damage_type': 'backtrace', 'id': 3408690552, 'ino': 1099511638571, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/.gitignore'}, {'damage_type': 'backtrace', 'id': 3880791220, 'ino': 1099511638570, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/timestamping.c'}, {'damage_type': 'backtrace', 'id': 4001013450, 'ino': 1099511638567, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c'}] 2024-04-02T14:47:35.653 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 294003381, 'ino': 1099511638951, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/compartmentalisation.txt'}, {'damage_type': 'backtrace', 'id': 2019150572, 'ino': 1099511638952, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/ABI.txt'}] 2024-04-02T14:47:35.653 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 1074492612, 'ino': 1099511638939, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/omap-hdq'}, {'damage_type': 'backtrace', 'id': 1510157187, 'ino': 1099511638941, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/mxc-w1'}, {'damage_type': 'backtrace', 'id': 1590141041, 'ino': 1099511638938, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/w1-gpio'}, {'damage_type': 'backtrace', 'id': 3629657775, 'ino': 1099511638940, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2482'}, {'damage_type': 'backtrace', 'id': 3660942034, 'ino': 1099511638936, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/00-INDEX'}, {'damage_type': 'backtrace', 'id': 3812859424, 'ino': 1099511638937, 'path': '/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/w1/masters/ds2490'}] 2024-04-02T14:47:35.653 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-04-02T14:47:36.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:35 smithi204 ceph-mon[35948]: pgmap v110: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-02T14:47:36.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:35 smithi062 ceph-mon[32116]: pgmap v110: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-02T14:47:36.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:35 smithi113 ceph-mon[36135]: pgmap v110: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-02T14:47:38.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:37 smithi204 ceph-mon[35948]: pgmap v111: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-04-02T14:47:38.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:37 smithi062 ceph-mon[32116]: pgmap v111: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-04-02T14:47:38.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:37 smithi113 ceph-mon[36135]: pgmap v111: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-04-02T14:47:40.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:39 smithi204 ceph-mon[35948]: pgmap v112: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-02T14:47:40.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:39 smithi062 ceph-mon[32116]: pgmap v112: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-02T14:47:40.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:39 smithi113 ceph-mon[36135]: pgmap v112: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-02T14:47:42.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:41 smithi204 ceph-mon[35948]: pgmap v113: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-02T14:47:42.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:41 smithi062 ceph-mon[32116]: pgmap v113: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-02T14:47:42.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:41 smithi113 ceph-mon[36135]: pgmap v113: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-02T14:47:44.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:43 smithi204 ceph-mon[35948]: pgmap v114: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-04-02T14:47:44.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:43 smithi062 ceph-mon[32116]: pgmap v114: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-04-02T14:47:44.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:43 smithi113 ceph-mon[36135]: pgmap v114: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-04-02T14:47:46.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:45 smithi204 ceph-mon[35948]: pgmap v115: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:47:46.089 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:45 smithi062 ceph-mon[32116]: pgmap v115: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:47:46.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:45 smithi113 ceph-mon[36135]: pgmap v115: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:47:47.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:46 smithi204 ceph-mon[35948]: pgmap v116: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-02T14:47:47.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:46 smithi062 ceph-mon[32116]: pgmap v116: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-02T14:47:47.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:46 smithi113 ceph-mon[36135]: pgmap v116: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-02T14:47:49.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:49 smithi062 ceph-mon[32116]: pgmap v117: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:47:49.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:49 smithi113 ceph-mon[36135]: pgmap v117: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:47:49.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:49 smithi204 ceph-mon[35948]: pgmap v117: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:47:51.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:51 smithi062 ceph-mon[32116]: pgmap v118: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:47:51.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:51 smithi113 ceph-mon[36135]: pgmap v118: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:47:51.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:51 smithi204 ceph-mon[35948]: pgmap v118: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:47:53.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:53 smithi062 ceph-mon[32116]: pgmap v119: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-02T14:47:53.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:53 smithi113 ceph-mon[36135]: pgmap v119: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-02T14:47:53.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:53 smithi204 ceph-mon[35948]: pgmap v119: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-02T14:47:55.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:55 smithi062 ceph-mon[32116]: pgmap v120: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-02T14:47:55.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:55 smithi113 ceph-mon[36135]: pgmap v120: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-02T14:47:55.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:55 smithi204 ceph-mon[35948]: pgmap v120: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-02T14:47:57.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:57 smithi062 ceph-mon[32116]: pgmap v121: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-02T14:47:57.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:57 smithi113 ceph-mon[36135]: pgmap v121: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-02T14:47:57.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:57 smithi204 ceph-mon[35948]: pgmap v121: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-02T14:47:59.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:47:59 smithi062 ceph-mon[32116]: pgmap v122: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:47:59.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:47:59 smithi113 ceph-mon[36135]: pgmap v122: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:47:59.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:47:59 smithi204 ceph-mon[35948]: pgmap v122: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:48:01.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:01 smithi062 ceph-mon[32116]: pgmap v123: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:48:01.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:01 smithi113 ceph-mon[36135]: pgmap v123: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:48:01.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:01 smithi204 ceph-mon[35948]: pgmap v123: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T14:48:04.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:03 smithi204 ceph-mon[35948]: pgmap v124: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-02T14:48:04.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:03 smithi062 ceph-mon[32116]: pgmap v124: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-02T14:48:04.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:03 smithi113 ceph-mon[36135]: pgmap v124: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-02T14:48:06.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:05 smithi204 ceph-mon[35948]: pgmap v125: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:48:06.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:05 smithi062 ceph-mon[32116]: pgmap v125: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:48:06.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:05 smithi113 ceph-mon[36135]: pgmap v125: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:48:08.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:07 smithi204 ceph-mon[35948]: pgmap v126: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-02T14:48:08.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:07 smithi062 ceph-mon[32116]: pgmap v126: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-02T14:48:08.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:07 smithi113 ceph-mon[36135]: pgmap v126: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-02T14:48:10.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:09 smithi204 ceph-mon[35948]: pgmap v127: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:48:10.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:09 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:48:10.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:09 smithi062 ceph-mon[32116]: pgmap v127: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:48:10.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:09 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:48:10.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:09 smithi113 ceph-mon[36135]: pgmap v127: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:48:10.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:09 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:48:12.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:11 smithi204 ceph-mon[35948]: pgmap v128: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:48:12.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:11 smithi062 ceph-mon[32116]: pgmap v128: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:48:12.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:11 smithi113 ceph-mon[36135]: pgmap v128: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:48:14.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:13 smithi204 ceph-mon[35948]: pgmap v129: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-02T14:48:14.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:13 smithi062 ceph-mon[32116]: pgmap v129: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-02T14:48:14.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:13 smithi113 ceph-mon[36135]: pgmap v129: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-02T14:48:15.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:14 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:48:15.017 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:14 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:48:15.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:14 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:48:16.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:15 smithi204 ceph-mon[35948]: pgmap v130: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T14:48:16.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:15 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:48:16.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:15 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:48:16.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:15 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:48:16.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:15 smithi062 ceph-mon[32116]: pgmap v130: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T14:48:16.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:15 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:48:16.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:15 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:48:16.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:15 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:48:16.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:15 smithi113 ceph-mon[36135]: pgmap v130: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T14:48:16.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:15 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:48:16.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:15 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:48:16.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:15 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:48:18.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:17 smithi204 ceph-mon[35948]: pgmap v131: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-02T14:48:18.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:17 smithi062 ceph-mon[32116]: pgmap v131: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-02T14:48:18.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:17 smithi113 ceph-mon[36135]: pgmap v131: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-02T14:48:20.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:19 smithi204 ceph-mon[35948]: pgmap v132: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:48:20.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:19 smithi062 ceph-mon[32116]: pgmap v132: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:48:20.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:19 smithi113 ceph-mon[36135]: pgmap v132: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:48:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:21 smithi204 ceph-mon[35948]: pgmap v133: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:48:22.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:21 smithi062 ceph-mon[32116]: pgmap v133: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:48:22.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:21 smithi113 ceph-mon[36135]: pgmap v133: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:48:24.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:23 smithi204 ceph-mon[35948]: pgmap v134: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:48:24.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:23 smithi062 ceph-mon[32116]: pgmap v134: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:48:24.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:23 smithi113 ceph-mon[36135]: pgmap v134: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:48:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:25 smithi204 ceph-mon[35948]: pgmap v135: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:26.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:25 smithi062 ceph-mon[32116]: pgmap v135: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:26.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:25 smithi113 ceph-mon[36135]: pgmap v135: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:28.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:27 smithi204 ceph-mon[35948]: pgmap v136: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:28.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:27 smithi062 ceph-mon[32116]: pgmap v136: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:28.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:27 smithi113 ceph-mon[36135]: pgmap v136: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:30.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:29 smithi204 ceph-mon[35948]: pgmap v137: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:30.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:29 smithi062 ceph-mon[32116]: pgmap v137: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:30.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:29 smithi113 ceph-mon[36135]: pgmap v137: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:32.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:31 smithi204 ceph-mon[35948]: pgmap v138: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:32.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:31 smithi062 ceph-mon[32116]: pgmap v138: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:32.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:31 smithi113 ceph-mon[36135]: pgmap v138: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:34.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:33 smithi204 ceph-mon[35948]: pgmap v139: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:34.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:33 smithi062 ceph-mon[32116]: pgmap v139: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:34.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:33 smithi113 ceph-mon[36135]: pgmap v139: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:36.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:35 smithi204 ceph-mon[35948]: pgmap v140: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:36.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:35 smithi062 ceph-mon[32116]: pgmap v140: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:36.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:35 smithi113 ceph-mon[36135]: pgmap v140: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:38.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:37 smithi204 ceph-mon[35948]: pgmap v141: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:38.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:37 smithi062 ceph-mon[32116]: pgmap v141: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:38.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:37 smithi113 ceph-mon[36135]: pgmap v141: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:40.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:39 smithi204 ceph-mon[35948]: pgmap v142: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:40.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:39 smithi062 ceph-mon[32116]: pgmap v142: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:40.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:39 smithi113 ceph-mon[36135]: pgmap v142: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:42.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:41 smithi204 ceph-mon[35948]: pgmap v143: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:42.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:41 smithi062 ceph-mon[32116]: pgmap v143: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:42.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:41 smithi113 ceph-mon[36135]: pgmap v143: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:44.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:43 smithi204 ceph-mon[35948]: pgmap v144: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:44.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:43 smithi062 ceph-mon[32116]: pgmap v144: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:44.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:43 smithi113 ceph-mon[36135]: pgmap v144: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:46.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:45 smithi204 ceph-mon[35948]: pgmap v145: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:46.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:45 smithi062 ceph-mon[32116]: pgmap v145: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:46.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:45 smithi113 ceph-mon[36135]: pgmap v145: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:47.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:46 smithi062 ceph-mon[32116]: pgmap v146: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:47.114 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:46 smithi113 ceph-mon[36135]: pgmap v146: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:47.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:46 smithi204 ceph-mon[35948]: pgmap v146: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:49.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:49 smithi062 ceph-mon[32116]: pgmap v147: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:49.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:49 smithi113 ceph-mon[36135]: pgmap v147: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:49.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:49 smithi204 ceph-mon[35948]: pgmap v147: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:51.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:51 smithi062 ceph-mon[32116]: pgmap v148: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:51.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:51 smithi113 ceph-mon[36135]: pgmap v148: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:51.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:51 smithi204 ceph-mon[35948]: pgmap v148: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:53.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:53 smithi062 ceph-mon[32116]: pgmap v149: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:53.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:53 smithi113 ceph-mon[36135]: pgmap v149: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:53.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:53 smithi204 ceph-mon[35948]: pgmap v149: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:55.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:55 smithi062 ceph-mon[32116]: pgmap v150: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:55.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:55 smithi113 ceph-mon[36135]: pgmap v150: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:55 smithi204 ceph-mon[35948]: pgmap v150: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:57.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:57 smithi062 ceph-mon[32116]: pgmap v151: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:57.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:57 smithi113 ceph-mon[36135]: pgmap v151: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:57 smithi204 ceph-mon[35948]: pgmap v151: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:48:58.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:58 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:48:58.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:58 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:48:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:58 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:48:59.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:48:59 smithi062 ceph-mon[32116]: pgmap v152: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:59.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:48:59 smithi113 ceph-mon[36135]: pgmap v152: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:48:59.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:48:59 smithi204 ceph-mon[35948]: pgmap v152: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:01.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:01 smithi062 ceph-mon[32116]: pgmap v153: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:01.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:01 smithi113 ceph-mon[36135]: pgmap v153: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:01.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:01 smithi204 ceph-mon[35948]: pgmap v153: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:03.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:03 smithi113 ceph-mon[36135]: pgmap v154: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T14:49:03.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:03 smithi204 ceph-mon[35948]: pgmap v154: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T14:49:03.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:03 smithi062 ceph-mon[32116]: pgmap v154: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T14:49:05.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:05 smithi113 ceph-mon[36135]: pgmap v155: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-02T14:49:05.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:05 smithi204 ceph-mon[35948]: pgmap v155: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-02T14:49:05.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:05 smithi062 ceph-mon[32116]: pgmap v155: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-02T14:49:07.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:07 smithi113 ceph-mon[36135]: pgmap v156: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:49:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:07 smithi204 ceph-mon[35948]: pgmap v156: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:49:07.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:07 smithi062 ceph-mon[32116]: pgmap v156: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:49:09.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:09 smithi113 ceph-mon[36135]: pgmap v157: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:49:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:09 smithi204 ceph-mon[35948]: pgmap v157: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:49:09.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:09 smithi062 ceph-mon[32116]: pgmap v157: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:49:11.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:11 smithi113 ceph-mon[36135]: pgmap v158: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:49:11.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:11 smithi204 ceph-mon[35948]: pgmap v158: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:49:11.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:11 smithi062 ceph-mon[32116]: pgmap v158: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:49:13.614 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:13 smithi113 ceph-mon[36135]: pgmap v159: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:49:13.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:13 smithi204 ceph-mon[35948]: pgmap v159: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:49:13.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:13 smithi062 ceph-mon[32116]: pgmap v159: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:49:15.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:15 smithi204 ceph-mon[35948]: pgmap v160: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-02T14:49:15.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:15 smithi062 ceph-mon[32116]: pgmap v160: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-02T14:49:15.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:15 smithi113 ceph-mon[36135]: pgmap v160: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-02T14:49:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:16 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:49:16.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:16 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:49:16.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:16 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:49:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:17 smithi204 ceph-mon[35948]: pgmap v161: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-02T14:49:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:17 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:17 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:49:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:17 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:49:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:17 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:49:17.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:17 smithi062 ceph-mon[32116]: pgmap v161: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-02T14:49:17.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:17 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:17.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:17 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:49:17.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:17 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:49:17.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:17 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:49:17.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:17 smithi113 ceph-mon[36135]: pgmap v161: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-02T14:49:17.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:17 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:17.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:17 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:49:17.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:17 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:49:17.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:17 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:49:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:19 smithi204 ceph-mon[35948]: pgmap v162: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:49:19.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:19 smithi062 ceph-mon[32116]: pgmap v162: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:49:19.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:19 smithi113 ceph-mon[36135]: pgmap v162: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:49:21.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:21 smithi204 ceph-mon[35948]: pgmap v163: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:49:21.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:21 smithi062 ceph-mon[32116]: pgmap v163: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:49:21.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:21 smithi113 ceph-mon[36135]: pgmap v163: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:49:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:23 smithi204 ceph-mon[35948]: pgmap v164: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:49:23.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:23 smithi062 ceph-mon[32116]: pgmap v164: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:49:23.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:23 smithi113 ceph-mon[36135]: pgmap v164: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:49:25.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:25 smithi204 ceph-mon[35948]: pgmap v165: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:25.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:25 smithi062 ceph-mon[32116]: pgmap v165: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:25.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:25 smithi113 ceph-mon[36135]: pgmap v165: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:26.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:26 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:26.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:26 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:26.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:26 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:27 smithi204 ceph-mon[35948]: pgmap v166: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:27.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:27 smithi062 ceph-mon[32116]: pgmap v166: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:27.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:27 smithi113 ceph-mon[36135]: pgmap v166: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:29.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:29 smithi204 ceph-mon[35948]: pgmap v167: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:29.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:29 smithi062 ceph-mon[32116]: pgmap v167: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:29.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:29 smithi113 ceph-mon[36135]: pgmap v167: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:31.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:31 smithi204 ceph-mon[35948]: pgmap v168: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:31.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:31 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:31.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:31 smithi062 ceph-mon[32116]: pgmap v168: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:31.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:31 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:31.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:31 smithi113 ceph-mon[36135]: pgmap v168: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:31.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:31 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:33.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:33 smithi204 ceph-mon[35948]: pgmap v169: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:33.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:33 smithi062 ceph-mon[32116]: pgmap v169: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:33.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:33 smithi113 ceph-mon[36135]: pgmap v169: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:35.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:35 smithi204 ceph-mon[35948]: pgmap v170: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:35.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:35 smithi062 ceph-mon[32116]: pgmap v170: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:35.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:35 smithi113 ceph-mon[36135]: pgmap v170: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:36.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:36 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:36.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:36 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:36.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:36 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:37 smithi204 ceph-mon[35948]: pgmap v171: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:37.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:37 smithi062 ceph-mon[32116]: pgmap v171: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:37.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:37 smithi113 ceph-mon[36135]: pgmap v171: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:39.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:39 smithi204 ceph-mon[35948]: pgmap v172: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:39.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:39 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:39.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:39 smithi062 ceph-mon[32116]: pgmap v172: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:39.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:39 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:39.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:39 smithi113 ceph-mon[36135]: pgmap v172: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:39.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:39 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:41.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:41 smithi204 ceph-mon[35948]: pgmap v173: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:41.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:41 smithi062 ceph-mon[32116]: pgmap v173: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:41.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:41 smithi113 ceph-mon[36135]: pgmap v173: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:43.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:43 smithi204 ceph-mon[35948]: pgmap v174: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:43.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:43 smithi062 ceph-mon[32116]: pgmap v174: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:43.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:43 smithi113 ceph-mon[36135]: pgmap v174: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:45.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:45 smithi204 ceph-mon[35948]: pgmap v175: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:45.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:45 smithi062 ceph-mon[32116]: pgmap v175: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:45.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:45 smithi113 ceph-mon[36135]: pgmap v175: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:46.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:46 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:46.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:46 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:47.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:46 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:47.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:47 smithi062 ceph-mon[32116]: pgmap v176: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:47.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:47 smithi113 ceph-mon[36135]: pgmap v176: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:48.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:47 smithi204 ceph-mon[35948]: pgmap v176: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:49.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:49 smithi062 ceph-mon[32116]: pgmap v177: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:49.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:49 smithi113 ceph-mon[36135]: pgmap v177: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:50.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:49 smithi204 ceph-mon[35948]: pgmap v177: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:51.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:51 smithi062 ceph-mon[32116]: pgmap v178: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:51.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:51 smithi113 ceph-mon[36135]: pgmap v178: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:52.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:51 smithi204 ceph-mon[35948]: pgmap v178: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:53.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:53 smithi062 ceph-mon[32116]: pgmap v179: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:53.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:53 smithi113 ceph-mon[36135]: pgmap v179: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:54.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:53 smithi204 ceph-mon[35948]: pgmap v179: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:55.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:55 smithi062 ceph-mon[32116]: pgmap v180: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:55.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:55 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:55.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:55 smithi113 ceph-mon[36135]: pgmap v180: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:55.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:55 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:56.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:55 smithi204 ceph-mon[35948]: pgmap v180: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:56.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:55 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:57.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:57 smithi062 ceph-mon[32116]: pgmap v181: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:57.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:57 smithi113 ceph-mon[36135]: pgmap v181: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:58.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:57 smithi204 ceph-mon[35948]: pgmap v181: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:49:58.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:58 smithi062 ceph-mon[32116]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:58.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:58 smithi113 ceph-mon[36135]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:59.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:58 smithi204 ceph-mon[35948]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T14:49:59.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:49:59 smithi062 ceph-mon[32116]: pgmap v182: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:49:59.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:49:59 smithi113 ceph-mon[36135]: pgmap v182: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:00.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:49:59 smithi204 ceph-mon[35948]: pgmap v182: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:00.336 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[32112]: 2024-04-02T14:49:59.999+0000 7f7747c00700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2024-04-02T14:50:00.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[32112]: 2024-04-02T14:49:59.999+0000 7f7747c00700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2024-04-02T14:50:00.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[32112]: 2024-04-02T14:49:59.999+0000 7f7747c00700 -1 log_channel(cluster) log [ERR] : mds.h(mds.4): Metadata damage detected 2024-04-02T14:50:00.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[32112]: 2024-04-02T14:49:59.999+0000 7f7747c00700 -1 log_channel(cluster) log [ERR] : mds.g(mds.2): Metadata damage detected 2024-04-02T14:50:00.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[32112]: 2024-04-02T14:49:59.999+0000 7f7747c00700 -1 log_channel(cluster) log [ERR] : mds.a(mds.1): Metadata damage detected 2024-04-02T14:50:00.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:00 smithi113 ceph-mon[36135]: Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2024-04-02T14:50:00.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:00 smithi113 ceph-mon[36135]: [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2024-04-02T14:50:00.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:00 smithi113 ceph-mon[36135]: mds.h(mds.4): Metadata damage detected 2024-04-02T14:50:00.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:00 smithi113 ceph-mon[36135]: mds.g(mds.2): Metadata damage detected 2024-04-02T14:50:00.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:00 smithi113 ceph-mon[36135]: mds.a(mds.1): Metadata damage detected 2024-04-02T14:50:01.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:00 smithi204 ceph-mon[35948]: Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2024-04-02T14:50:01.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:00 smithi204 ceph-mon[35948]: [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2024-04-02T14:50:01.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:00 smithi204 ceph-mon[35948]: mds.h(mds.4): Metadata damage detected 2024-04-02T14:50:01.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:00 smithi204 ceph-mon[35948]: mds.g(mds.2): Metadata damage detected 2024-04-02T14:50:01.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:00 smithi204 ceph-mon[35948]: mds.a(mds.1): Metadata damage detected 2024-04-02T14:50:01.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:00 smithi062 ceph-mon[32116]: Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2024-04-02T14:50:01.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:00 smithi062 ceph-mon[32116]: [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2024-04-02T14:50:01.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:00 smithi062 ceph-mon[32116]: mds.h(mds.4): Metadata damage detected 2024-04-02T14:50:01.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:00 smithi062 ceph-mon[32116]: mds.g(mds.2): Metadata damage detected 2024-04-02T14:50:01.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:00 smithi062 ceph-mon[32116]: mds.a(mds.1): Metadata damage detected 2024-04-02T14:50:01.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:01 smithi113 ceph-mon[36135]: pgmap v183: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:02.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:01 smithi204 ceph-mon[35948]: pgmap v183: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:02.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:01 smithi062 ceph-mon[32116]: pgmap v183: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:03.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:03 smithi113 ceph-mon[36135]: pgmap v184: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:50:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:03 smithi204 ceph-mon[35948]: pgmap v184: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:50:04.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:03 smithi062 ceph-mon[32116]: pgmap v184: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:50:05.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:05 smithi113 ceph-mon[36135]: pgmap v185: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:50:06.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:05 smithi204 ceph-mon[35948]: pgmap v185: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:50:06.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:05 smithi062 ceph-mon[32116]: pgmap v185: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:50:07.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:07 smithi113 ceph-mon[36135]: pgmap v186: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:50:08.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:07 smithi204 ceph-mon[35948]: pgmap v186: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:50:08.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:07 smithi062 ceph-mon[32116]: pgmap v186: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:50:10.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:09 smithi204 ceph-mon[35948]: pgmap v187: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:50:10.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:09 smithi062 ceph-mon[32116]: pgmap v187: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:50:10.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:09 smithi113 ceph-mon[36135]: pgmap v187: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:50:12.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:11 smithi204 ceph-mon[35948]: pgmap v188: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:50:12.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:11 smithi062 ceph-mon[32116]: pgmap v188: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:50:12.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:11 smithi113 ceph-mon[36135]: pgmap v188: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T14:50:13.956 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:13 smithi113 ceph-mon[36135]: pgmap v189: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:50:14.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:13 smithi204 ceph-mon[35948]: pgmap v189: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:50:14.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:13 smithi062 ceph-mon[32116]: pgmap v189: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T14:50:16.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:15 smithi204 ceph-mon[35948]: pgmap v190: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:50:16.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:15 smithi062 ceph-mon[32116]: pgmap v190: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:50:16.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:15 smithi113 ceph-mon[36135]: pgmap v190: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T14:50:18.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:17 smithi204 ceph-mon[35948]: pgmap v191: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:50:18.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:17 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:50:18.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:17 smithi062 ceph-mon[32116]: pgmap v191: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:50:18.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:17 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:50:18.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:17 smithi113 ceph-mon[36135]: pgmap v191: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T14:50:18.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:17 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:50:19.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:18 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:50:19.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:18 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:50:19.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:18 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:50:19.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:18 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:50:19.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:18 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:50:19.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:18 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:50:19.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:18 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:50:19.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:18 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:50:19.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:18 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:50:20.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:19 smithi204 ceph-mon[35948]: pgmap v192: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:20.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:19 smithi062 ceph-mon[32116]: pgmap v192: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:20.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:19 smithi113 ceph-mon[36135]: pgmap v192: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:22.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:21 smithi204 ceph-mon[35948]: pgmap v193: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:22.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:21 smithi062 ceph-mon[32116]: pgmap v193: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:22.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:21 smithi113 ceph-mon[36135]: pgmap v193: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:24.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:23 smithi204 ceph-mon[35948]: pgmap v194: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:24.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:23 smithi062 ceph-mon[32116]: pgmap v194: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:24.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:23 smithi113 ceph-mon[36135]: pgmap v194: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:26.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:25 smithi204 ceph-mon[35948]: pgmap v195: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:26.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:25 smithi062 ceph-mon[32116]: pgmap v195: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:26.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:25 smithi113 ceph-mon[36135]: pgmap v195: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:28.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:27 smithi204 ceph-mon[35948]: pgmap v196: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:28.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:27 smithi062 ceph-mon[32116]: pgmap v196: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:28.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:27 smithi113 ceph-mon[36135]: pgmap v196: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:30.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:29 smithi204 ceph-mon[35948]: pgmap v197: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:30.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:29 smithi062 ceph-mon[32116]: pgmap v197: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:30.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:29 smithi113 ceph-mon[36135]: pgmap v197: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:32.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:31 smithi204 ceph-mon[35948]: pgmap v198: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:32.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:31 smithi062 ceph-mon[32116]: pgmap v198: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:32.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:31 smithi113 ceph-mon[36135]: pgmap v198: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:34.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:33 smithi204 ceph-mon[35948]: pgmap v199: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:34.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:33 smithi062 ceph-mon[32116]: pgmap v199: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:34.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:33 smithi113 ceph-mon[36135]: pgmap v199: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:36.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:35 smithi204 ceph-mon[35948]: pgmap v200: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:36.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:35 smithi062 ceph-mon[32116]: pgmap v200: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:36.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:35 smithi113 ceph-mon[36135]: pgmap v200: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:38.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:37 smithi204 ceph-mon[35948]: pgmap v201: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:38.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:37 smithi062 ceph-mon[32116]: pgmap v201: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:38.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:37 smithi113 ceph-mon[36135]: pgmap v201: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:40.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:39 smithi204 ceph-mon[35948]: pgmap v202: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:40.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:39 smithi062 ceph-mon[32116]: pgmap v202: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:40.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:39 smithi113 ceph-mon[36135]: pgmap v202: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:42.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:41 smithi204 ceph-mon[35948]: pgmap v203: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:42.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:41 smithi062 ceph-mon[32116]: pgmap v203: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:42.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:41 smithi113 ceph-mon[36135]: pgmap v203: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:44.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:43 smithi204 ceph-mon[35948]: pgmap v204: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:44.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:43 smithi062 ceph-mon[32116]: pgmap v204: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:44.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:43 smithi113 ceph-mon[36135]: pgmap v204: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:46.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:45 smithi204 ceph-mon[35948]: pgmap v205: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:46.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:45 smithi062 ceph-mon[32116]: pgmap v205: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:46.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:45 smithi113 ceph-mon[36135]: pgmap v205: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:48.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:47 smithi204 ceph-mon[35948]: pgmap v206: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:48.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:47 smithi062 ceph-mon[32116]: pgmap v206: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:48.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:47 smithi113 ceph-mon[36135]: pgmap v206: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:50.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:49 smithi204 ceph-mon[35948]: pgmap v207: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:50.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:49 smithi062 ceph-mon[32116]: pgmap v207: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:50.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:49 smithi113 ceph-mon[36135]: pgmap v207: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:52.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:51 smithi204 ceph-mon[35948]: pgmap v208: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:52.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:51 smithi062 ceph-mon[32116]: pgmap v208: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:52.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:51 smithi113 ceph-mon[36135]: pgmap v208: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:54.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:53 smithi204 ceph-mon[35948]: pgmap v209: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:54.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:53 smithi062 ceph-mon[32116]: pgmap v209: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:54.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:53 smithi113 ceph-mon[36135]: pgmap v209: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:56.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:55 smithi204 ceph-mon[35948]: pgmap v210: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:56.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:55 smithi062 ceph-mon[32116]: pgmap v210: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:56.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:55 smithi113 ceph-mon[36135]: pgmap v210: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:50:58.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:57 smithi204 ceph-mon[35948]: pgmap v211: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:58.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:57 smithi062 ceph-mon[32116]: pgmap v211: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:50:58.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:57 smithi113 ceph-mon[36135]: pgmap v211: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:00.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:50:59 smithi062 ceph-mon[32116]: pgmap v212: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:00.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:50:59 smithi113 ceph-mon[36135]: pgmap v212: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:00.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:50:59 smithi204 ceph-mon[35948]: pgmap v212: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:02.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:01 smithi062 ceph-mon[32116]: pgmap v213: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:02.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:01 smithi113 ceph-mon[36135]: pgmap v213: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:02.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:01 smithi204 ceph-mon[35948]: pgmap v213: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:04.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:03 smithi062 ceph-mon[32116]: pgmap v214: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:51:04.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:03 smithi113 ceph-mon[36135]: pgmap v214: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:51:04.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:03 smithi204 ceph-mon[35948]: pgmap v214: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:51:05.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:04 smithi062 ceph-mon[32116]: pgmap v215: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:51:05.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:04 smithi113 ceph-mon[36135]: pgmap v215: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:51:05.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:04 smithi204 ceph-mon[35948]: pgmap v215: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:51:07.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:07 smithi113 ceph-mon[36135]: pgmap v216: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:51:07.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:07 smithi204 ceph-mon[35948]: pgmap v216: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:51:07.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:07 smithi062 ceph-mon[32116]: pgmap v216: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:51:09.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:09 smithi113 ceph-mon[36135]: pgmap v217: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:51:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:09 smithi204 ceph-mon[35948]: pgmap v217: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:51:09.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:09 smithi062 ceph-mon[32116]: pgmap v217: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:51:11.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:11 smithi113 ceph-mon[36135]: pgmap v218: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:51:11.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:11 smithi204 ceph-mon[35948]: pgmap v218: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:51:11.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:11 smithi062 ceph-mon[32116]: pgmap v218: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:51:12.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:12 smithi204 ceph-mon[35948]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T14:51:12.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:12 smithi062 ceph-mon[32116]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T14:51:12.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:12 smithi113 ceph-mon[36135]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T14:51:13.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:13 smithi204 ceph-mon[35948]: pgmap v220: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:51:13.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:13 smithi062 ceph-mon[32116]: pgmap v220: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:51:13.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:13 smithi113 ceph-mon[36135]: pgmap v220: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:51:15.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:15 smithi204 ceph-mon[35948]: pgmap v221: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:51:15.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:15 smithi062 ceph-mon[32116]: pgmap v221: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:51:15.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:15 smithi113 ceph-mon[36135]: pgmap v221: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:51:18.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:17 smithi204 ceph-mon[35948]: pgmap v222: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:51:18.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:17 smithi204 ceph-mon[35948]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T14:51:18.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:17 smithi062 ceph-mon[32116]: pgmap v222: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:51:18.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:17 smithi062 ceph-mon[32116]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T14:51:18.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:17 smithi113 ceph-mon[36135]: pgmap v222: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:51:18.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:17 smithi113 ceph-mon[36135]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T14:51:20.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:19 smithi204 ceph-mon[35948]: pgmap v224: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:51:20.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:19 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:51:20.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:19 smithi062 ceph-mon[32116]: pgmap v224: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:51:20.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:19 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:51:20.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:19 smithi113 ceph-mon[36135]: pgmap v224: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:51:20.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:19 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:51:21.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:20 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:51:21.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:20 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:51:21.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:20 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:51:21.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:20 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:51:21.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:20 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:51:21.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:20 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:51:21.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:20 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:51:21.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:20 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:51:21.113 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:20 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:51:22.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:21 smithi204 ceph-mon[35948]: pgmap v225: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 114 B/s wr, 5 op/s 2024-04-02T14:51:22.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:21 smithi062 ceph-mon[32116]: pgmap v225: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 114 B/s wr, 5 op/s 2024-04-02T14:51:22.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:21 smithi113 ceph-mon[36135]: pgmap v225: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 114 B/s wr, 5 op/s 2024-04-02T14:51:24.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:23 smithi204 ceph-mon[35948]: pgmap v226: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:51:24.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:23 smithi062 ceph-mon[32116]: pgmap v226: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:51:24.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:23 smithi113 ceph-mon[36135]: pgmap v226: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:51:26.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:25 smithi204 ceph-mon[35948]: pgmap v227: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:51:26.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:25 smithi062 ceph-mon[32116]: pgmap v227: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:51:26.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:25 smithi113 ceph-mon[36135]: pgmap v227: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:51:28.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:27 smithi204 ceph-mon[35948]: pgmap v228: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:51:28.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:27 smithi062 ceph-mon[32116]: pgmap v228: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:51:28.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:27 smithi113 ceph-mon[36135]: pgmap v228: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:51:30.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:29 smithi204 ceph-mon[35948]: pgmap v229: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:51:30.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:29 smithi062 ceph-mon[32116]: pgmap v229: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:51:30.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:29 smithi113 ceph-mon[36135]: pgmap v229: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:51:32.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:31 smithi204 ceph-mon[35948]: pgmap v230: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:31 smithi062 ceph-mon[32116]: pgmap v230: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:32.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:31 smithi113 ceph-mon[36135]: pgmap v230: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:34.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:33 smithi204 ceph-mon[35948]: pgmap v231: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:34.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:33 smithi062 ceph-mon[32116]: pgmap v231: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:34.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:33 smithi113 ceph-mon[36135]: pgmap v231: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:36.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:35 smithi204 ceph-mon[35948]: pgmap v232: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:36.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:35 smithi062 ceph-mon[32116]: pgmap v232: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:36.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:35 smithi113 ceph-mon[36135]: pgmap v232: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:38.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:37 smithi204 ceph-mon[35948]: pgmap v233: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:38.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:37 smithi062 ceph-mon[32116]: pgmap v233: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:38.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:37 smithi113 ceph-mon[36135]: pgmap v233: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:40.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:39 smithi062 ceph-mon[32116]: pgmap v234: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:40.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:39 smithi113 ceph-mon[36135]: pgmap v234: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:40.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:39 smithi204 ceph-mon[35948]: pgmap v234: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:42.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:41 smithi062 ceph-mon[32116]: pgmap v235: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:42.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:41 smithi113 ceph-mon[36135]: pgmap v235: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:42.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:41 smithi204 ceph-mon[35948]: pgmap v235: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:44.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:43 smithi062 ceph-mon[32116]: pgmap v236: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:44.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:43 smithi113 ceph-mon[36135]: pgmap v236: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:44.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:43 smithi204 ceph-mon[35948]: pgmap v236: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:46.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:45 smithi062 ceph-mon[32116]: pgmap v237: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:46.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:45 smithi113 ceph-mon[36135]: pgmap v237: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:46.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:45 smithi204 ceph-mon[35948]: pgmap v237: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:48.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:47 smithi062 ceph-mon[32116]: pgmap v238: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:48.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:47 smithi113 ceph-mon[36135]: pgmap v238: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:48.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:47 smithi204 ceph-mon[35948]: pgmap v238: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:50.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:49 smithi062 ceph-mon[32116]: pgmap v239: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:50.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:49 smithi113 ceph-mon[36135]: pgmap v239: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:50.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:49 smithi204 ceph-mon[35948]: pgmap v239: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:52.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:51 smithi062 ceph-mon[32116]: pgmap v240: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:52.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:51 smithi113 ceph-mon[36135]: pgmap v240: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:52.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:51 smithi204 ceph-mon[35948]: pgmap v240: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:53.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:52 smithi062 ceph-mon[32116]: pgmap v241: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:53.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:52 smithi113 ceph-mon[36135]: pgmap v241: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:53.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:52 smithi204 ceph-mon[35948]: pgmap v241: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:55.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:55 smithi204 ceph-mon[35948]: pgmap v242: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:55.789 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:55 smithi062 ceph-mon[32116]: pgmap v242: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:55.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:55 smithi113 ceph-mon[36135]: pgmap v242: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:51:57.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:57 smithi204 ceph-mon[35948]: pgmap v243: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:57.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:57 smithi062 ceph-mon[32116]: pgmap v243: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:57.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:57 smithi113 ceph-mon[36135]: pgmap v243: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:51:59.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:51:59 smithi204 ceph-mon[35948]: pgmap v244: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:59.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:51:59 smithi062 ceph-mon[32116]: pgmap v244: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:51:59.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:51:59 smithi113 ceph-mon[36135]: pgmap v244: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:01.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:01 smithi204 ceph-mon[35948]: pgmap v245: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:01.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:01 smithi062 ceph-mon[32116]: pgmap v245: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:01.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:01 smithi113 ceph-mon[36135]: pgmap v245: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:03 smithi204 ceph-mon[35948]: pgmap v246: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:52:04.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:03 smithi062 ceph-mon[32116]: pgmap v246: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:52:04.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:03 smithi113 ceph-mon[36135]: pgmap v246: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:52:06.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:05 smithi204 ceph-mon[35948]: pgmap v247: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:52:06.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:05 smithi062 ceph-mon[32116]: pgmap v247: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:52:06.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:05 smithi113 ceph-mon[36135]: pgmap v247: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:52:08.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:07 smithi204 ceph-mon[35948]: pgmap v248: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:52:08.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:07 smithi062 ceph-mon[32116]: pgmap v248: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:52:08.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:07 smithi113 ceph-mon[36135]: pgmap v248: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:52:10.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:09 smithi204 ceph-mon[35948]: pgmap v249: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:52:10.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:09 smithi062 ceph-mon[32116]: pgmap v249: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:52:10.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:09 smithi113 ceph-mon[36135]: pgmap v249: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:52:12.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:11 smithi204 ceph-mon[35948]: pgmap v250: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:52:12.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:11 smithi204 ceph-mon[35948]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T14:52:12.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:11 smithi062 ceph-mon[32116]: pgmap v250: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:52:12.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:11 smithi062 ceph-mon[32116]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T14:52:12.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:11 smithi113 ceph-mon[36135]: pgmap v250: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:52:12.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:11 smithi113 ceph-mon[36135]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T14:52:14.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:13 smithi062 ceph-mon[32116]: pgmap v252: 97 pgs: 22 active+clean+snaptrim, 34 active+clean, 41 active+clean+snaptrim_wait; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:14.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:13 smithi113 ceph-mon[36135]: pgmap v252: 97 pgs: 22 active+clean+snaptrim, 34 active+clean, 41 active+clean+snaptrim_wait; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:14.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:13 smithi204 ceph-mon[35948]: pgmap v252: 97 pgs: 22 active+clean+snaptrim, 34 active+clean, 41 active+clean+snaptrim_wait; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:16.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:15 smithi062 ceph-mon[32116]: pgmap v253: 97 pgs: 22 active+clean+snaptrim, 34 active+clean, 41 active+clean+snaptrim_wait; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:16.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:15 smithi113 ceph-mon[36135]: pgmap v253: 97 pgs: 22 active+clean+snaptrim, 34 active+clean, 41 active+clean+snaptrim_wait; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:16.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:15 smithi204 ceph-mon[35948]: pgmap v253: 97 pgs: 22 active+clean+snaptrim, 34 active+clean, 41 active+clean+snaptrim_wait; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:18.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:17 smithi062 ceph-mon[32116]: pgmap v254: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:18.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:17 smithi113 ceph-mon[36135]: pgmap v254: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:18.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:17 smithi204 ceph-mon[35948]: pgmap v254: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:20.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:19 smithi062 ceph-mon[32116]: pgmap v255: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:20.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:19 smithi113 ceph-mon[36135]: pgmap v255: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:20.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:19 smithi204 ceph-mon[35948]: pgmap v255: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:21.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:20 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:52:21.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:20 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:52:21.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:20 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:52:22.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:21 smithi062 ceph-mon[32116]: pgmap v256: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:22.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:21 smithi062 ceph-mon[32116]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T14:52:22.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:21 smithi113 ceph-mon[36135]: pgmap v256: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:22.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:21 smithi113 ceph-mon[36135]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T14:52:22.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:21 smithi204 ceph-mon[35948]: pgmap v256: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:52:22.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:21 smithi204 ceph-mon[35948]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T14:52:23.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:22 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:52:23.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:22 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:52:23.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:22 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:52:23.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:22 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:52:23.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:22 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:52:23.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:22 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:52:23.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:22 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:52:23.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:22 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:52:23.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:22 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:52:24.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:23 smithi062 ceph-mon[32116]: pgmap v258: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:52:24.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:23 smithi113 ceph-mon[36135]: pgmap v258: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:52:24.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:23 smithi204 ceph-mon[35948]: pgmap v258: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:52:26.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:25 smithi062 ceph-mon[32116]: pgmap v259: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:52:26.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:25 smithi113 ceph-mon[36135]: pgmap v259: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:52:26.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:25 smithi204 ceph-mon[35948]: pgmap v259: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:52:27.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:26 smithi062 ceph-mon[32116]: pgmap v260: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:52:27.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:26 smithi113 ceph-mon[36135]: pgmap v260: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:52:27.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:26 smithi204 ceph-mon[35948]: pgmap v260: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:52:29.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:29 smithi204 ceph-mon[35948]: pgmap v261: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:52:29.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:29 smithi062 ceph-mon[32116]: pgmap v261: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:52:29.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:29 smithi113 ceph-mon[36135]: pgmap v261: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:52:31.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:31 smithi204 ceph-mon[35948]: pgmap v262: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:52:31.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:31 smithi062 ceph-mon[32116]: pgmap v262: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:52:31.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:31 smithi113 ceph-mon[36135]: pgmap v262: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:52:33.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:33 smithi204 ceph-mon[35948]: pgmap v263: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T14:52:33.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:33 smithi062 ceph-mon[32116]: pgmap v263: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T14:52:33.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:33 smithi113 ceph-mon[36135]: pgmap v263: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T14:52:35.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:35 smithi204 ceph-mon[35948]: pgmap v264: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:52:35.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:35 smithi062 ceph-mon[32116]: pgmap v264: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:52:35.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:35 smithi113 ceph-mon[36135]: pgmap v264: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:52:37.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:37 smithi204 ceph-mon[35948]: pgmap v265: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:37.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:37 smithi062 ceph-mon[32116]: pgmap v265: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:37.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:37 smithi113 ceph-mon[36135]: pgmap v265: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:39.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:39 smithi204 ceph-mon[35948]: pgmap v266: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:39.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:39 smithi062 ceph-mon[32116]: pgmap v266: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:39.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:39 smithi113 ceph-mon[36135]: pgmap v266: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:41.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:41 smithi204 ceph-mon[35948]: pgmap v267: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:41.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:41 smithi062 ceph-mon[32116]: pgmap v267: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:41.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:41 smithi113 ceph-mon[36135]: pgmap v267: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:43.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:43 smithi204 ceph-mon[35948]: pgmap v268: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:43.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:43 smithi062 ceph-mon[32116]: pgmap v268: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:43.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:43 smithi113 ceph-mon[36135]: pgmap v268: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:45.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:45 smithi204 ceph-mon[35948]: pgmap v269: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:45.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:45 smithi062 ceph-mon[32116]: pgmap v269: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:45.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:45 smithi113 ceph-mon[36135]: pgmap v269: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:47.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:47 smithi204 ceph-mon[35948]: pgmap v270: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:47.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:47 smithi062 ceph-mon[32116]: pgmap v270: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:47.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:47 smithi113 ceph-mon[36135]: pgmap v270: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:49.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:49 smithi204 ceph-mon[35948]: pgmap v271: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:49.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:49 smithi062 ceph-mon[32116]: pgmap v271: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:49.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:49 smithi113 ceph-mon[36135]: pgmap v271: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:51.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:51 smithi204 ceph-mon[35948]: pgmap v272: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:51.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:51 smithi062 ceph-mon[32116]: pgmap v272: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:51.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:51 smithi113 ceph-mon[36135]: pgmap v272: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:53.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:53 smithi204 ceph-mon[35948]: pgmap v273: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:53.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:53 smithi062 ceph-mon[32116]: pgmap v273: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:53.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:53 smithi113 ceph-mon[36135]: pgmap v273: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:55.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:55 smithi204 ceph-mon[35948]: pgmap v274: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:55.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:55 smithi062 ceph-mon[32116]: pgmap v274: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:55.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:55 smithi113 ceph-mon[36135]: pgmap v274: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:57.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:57 smithi204 ceph-mon[35948]: pgmap v275: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:57.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:57 smithi062 ceph-mon[32116]: pgmap v275: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:57.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:57 smithi113 ceph-mon[36135]: pgmap v275: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:52:59.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:52:59 smithi204 ceph-mon[35948]: pgmap v276: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:59.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:52:59 smithi062 ceph-mon[32116]: pgmap v276: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:52:59.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:52:59 smithi113 ceph-mon[36135]: pgmap v276: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:01.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:01 smithi204 ceph-mon[35948]: pgmap v277: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:01.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:01 smithi062 ceph-mon[32116]: pgmap v277: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:01.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:01 smithi113 ceph-mon[36135]: pgmap v277: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:03.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:03 smithi204 ceph-mon[35948]: pgmap v278: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:53:03.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:03 smithi062 ceph-mon[32116]: pgmap v278: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:53:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:03 smithi113 ceph-mon[36135]: pgmap v278: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:53:05.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:05 smithi204 ceph-mon[35948]: pgmap v279: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:53:05.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:05 smithi062 ceph-mon[32116]: pgmap v279: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:53:05.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:05 smithi113 ceph-mon[36135]: pgmap v279: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:53:07.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:07 smithi204 ceph-mon[35948]: pgmap v280: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:53:07.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:07 smithi062 ceph-mon[32116]: pgmap v280: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:53:07.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:07 smithi113 ceph-mon[36135]: pgmap v280: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:53:09.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:09 smithi204 ceph-mon[35948]: pgmap v281: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:53:09.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:09 smithi062 ceph-mon[32116]: pgmap v281: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:53:09.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:09 smithi113 ceph-mon[36135]: pgmap v281: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:53:11.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:11 smithi204 ceph-mon[35948]: pgmap v282: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:53:11.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:11 smithi204 ceph-mon[35948]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T14:53:11.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:11 smithi062 ceph-mon[32116]: pgmap v282: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:53:11.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:11 smithi062 ceph-mon[32116]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T14:53:11.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:11 smithi113 ceph-mon[36135]: pgmap v282: 97 pgs: 97 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:53:11.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:11 smithi113 ceph-mon[36135]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T14:53:13.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:13 smithi204 ceph-mon[35948]: pgmap v284: 97 pgs: 17 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 47 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:53:13.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:13 smithi062 ceph-mon[32116]: pgmap v284: 97 pgs: 17 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 47 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:53:13.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:13 smithi113 ceph-mon[36135]: pgmap v284: 97 pgs: 17 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 47 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:53:15.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:15 smithi062 ceph-mon[32116]: pgmap v285: 97 pgs: 17 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 47 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:53:15.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:15 smithi113 ceph-mon[36135]: pgmap v285: 97 pgs: 17 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 47 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:53:16.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:15 smithi204 ceph-mon[35948]: pgmap v285: 97 pgs: 17 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 47 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:53:18.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:17 smithi204 ceph-mon[35948]: pgmap v286: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:53:18.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:17 smithi204 ceph-mon[35948]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T14:53:18.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:17 smithi062 ceph-mon[32116]: pgmap v286: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:53:18.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:17 smithi062 ceph-mon[32116]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T14:53:18.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:17 smithi113 ceph-mon[36135]: pgmap v286: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:53:18.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:17 smithi113 ceph-mon[36135]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T14:53:20.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:19 smithi204 ceph-mon[35948]: pgmap v288: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:53:20.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:19 smithi062 ceph-mon[32116]: pgmap v288: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:53:20.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:19 smithi113 ceph-mon[36135]: pgmap v288: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:53:22.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:21 smithi062 ceph-mon[32116]: pgmap v289: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2024-04-02T14:53:22.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:21 smithi113 ceph-mon[36135]: pgmap v289: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2024-04-02T14:53:22.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:21 smithi204 ceph-mon[35948]: pgmap v289: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2024-04-02T14:53:23.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:22 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:53:23.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:22 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:53:23.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:22 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:53:24.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:23 smithi062 ceph-mon[32116]: pgmap v290: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:53:24.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:23 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:53:24.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:23 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:53:24.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:23 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:53:24.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:23 smithi113 ceph-mon[36135]: pgmap v290: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:53:24.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:23 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:53:24.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:23 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:53:24.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:23 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:53:24.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:23 smithi204 ceph-mon[35948]: pgmap v290: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:53:24.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:23 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:53:24.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:23 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:53:24.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:23 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:53:26.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:25 smithi062 ceph-mon[32116]: pgmap v291: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:53:26.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:25 smithi113 ceph-mon[36135]: pgmap v291: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:53:26.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:25 smithi204 ceph-mon[35948]: pgmap v291: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:53:28.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:27 smithi062 ceph-mon[32116]: pgmap v292: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-02T14:53:28.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:27 smithi113 ceph-mon[36135]: pgmap v292: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-02T14:53:28.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:27 smithi204 ceph-mon[35948]: pgmap v292: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-02T14:53:30.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:29 smithi062 ceph-mon[32116]: pgmap v293: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T14:53:30.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:29 smithi113 ceph-mon[36135]: pgmap v293: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T14:53:30.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:29 smithi204 ceph-mon[35948]: pgmap v293: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T14:53:32.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:31 smithi062 ceph-mon[32116]: pgmap v294: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:32.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:31 smithi113 ceph-mon[36135]: pgmap v294: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:32.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:31 smithi204 ceph-mon[35948]: pgmap v294: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:34.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:33 smithi062 ceph-mon[32116]: pgmap v295: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-02T14:53:34.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:33 smithi113 ceph-mon[36135]: pgmap v295: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-02T14:53:34.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:33 smithi204 ceph-mon[35948]: pgmap v295: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-02T14:53:36.053 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:35 smithi204 ceph-mon[35948]: pgmap v296: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:36.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:35 smithi062 ceph-mon[32116]: pgmap v296: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:36.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:35 smithi113 ceph-mon[36135]: pgmap v296: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:38.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:37 smithi062 ceph-mon[32116]: pgmap v297: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:38.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:37 smithi113 ceph-mon[36135]: pgmap v297: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:38.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:37 smithi204 ceph-mon[35948]: pgmap v297: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:40.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:39 smithi062 ceph-mon[32116]: pgmap v298: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:40.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:39 smithi113 ceph-mon[36135]: pgmap v298: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:40.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:39 smithi204 ceph-mon[35948]: pgmap v298: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:41.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:40 smithi062 ceph-mon[32116]: pgmap v299: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:41.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:40 smithi113 ceph-mon[36135]: pgmap v299: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:41.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:40 smithi204 ceph-mon[35948]: pgmap v299: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:43.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:43 smithi204 ceph-mon[35948]: pgmap v300: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:43.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:43 smithi062 ceph-mon[32116]: pgmap v300: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:43.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:43 smithi113 ceph-mon[36135]: pgmap v300: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:45.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:45 smithi204 ceph-mon[35948]: pgmap v301: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:45.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:45 smithi062 ceph-mon[32116]: pgmap v301: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:45.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:45 smithi113 ceph-mon[36135]: pgmap v301: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:47.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:47 smithi204 ceph-mon[35948]: pgmap v302: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:47.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:47 smithi062 ceph-mon[32116]: pgmap v302: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:47.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:47 smithi113 ceph-mon[36135]: pgmap v302: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:49.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:49 smithi204 ceph-mon[35948]: pgmap v303: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:49.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:49 smithi062 ceph-mon[32116]: pgmap v303: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:49.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:49 smithi113 ceph-mon[36135]: pgmap v303: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:51.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:51 smithi204 ceph-mon[35948]: pgmap v304: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:51.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:51 smithi062 ceph-mon[32116]: pgmap v304: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:51.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:51 smithi113 ceph-mon[36135]: pgmap v304: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:53.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:53 smithi204 ceph-mon[35948]: pgmap v305: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:53.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:53 smithi062 ceph-mon[32116]: pgmap v305: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:53.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:53 smithi113 ceph-mon[36135]: pgmap v305: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:55.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:55 smithi204 ceph-mon[35948]: pgmap v306: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:55.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:55 smithi062 ceph-mon[32116]: pgmap v306: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:55.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:55 smithi113 ceph-mon[36135]: pgmap v306: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:57.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:57 smithi204 ceph-mon[35948]: pgmap v307: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:57.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:57 smithi062 ceph-mon[32116]: pgmap v307: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:57.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:57 smithi113 ceph-mon[36135]: pgmap v307: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:53:59.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:53:59 smithi204 ceph-mon[35948]: pgmap v308: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:59.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:53:59 smithi062 ceph-mon[32116]: pgmap v308: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:53:59.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:53:59 smithi113 ceph-mon[36135]: pgmap v308: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:01.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:01 smithi204 ceph-mon[35948]: pgmap v309: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:01.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:01 smithi062 ceph-mon[32116]: pgmap v309: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:01.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:01 smithi113 ceph-mon[36135]: pgmap v309: 97 pgs: 97 active+clean; 18 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:04.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:03 smithi062 ceph-mon[32116]: pgmap v310: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:54:04.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:03 smithi113 ceph-mon[36135]: pgmap v310: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:54:04.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:03 smithi204 ceph-mon[35948]: pgmap v310: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:54:06.052 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:05 smithi204 ceph-mon[35948]: pgmap v311: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:54:06.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:05 smithi062 ceph-mon[32116]: pgmap v311: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:54:06.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:05 smithi113 ceph-mon[36135]: pgmap v311: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:54:07.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:06 smithi062 ceph-mon[32116]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T14:54:07.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:06 smithi113 ceph-mon[36135]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T14:54:07.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:06 smithi204 ceph-mon[35948]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T14:54:08.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:07 smithi062 ceph-mon[32116]: pgmap v313: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T14:54:08.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:07 smithi113 ceph-mon[36135]: pgmap v313: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T14:54:08.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:07 smithi204 ceph-mon[35948]: pgmap v313: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T14:54:10.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:09 smithi062 ceph-mon[32116]: pgmap v314: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T14:54:10.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:09 smithi113 ceph-mon[36135]: pgmap v314: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T14:54:10.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:09 smithi204 ceph-mon[35948]: pgmap v314: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T14:54:12.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:11 smithi062 ceph-mon[32116]: pgmap v315: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T14:54:12.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:11 smithi062 ceph-mon[32116]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T14:54:12.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:11 smithi113 ceph-mon[36135]: pgmap v315: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T14:54:12.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:11 smithi113 ceph-mon[36135]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T14:54:12.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:11 smithi204 ceph-mon[35948]: pgmap v315: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T14:54:12.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:11 smithi204 ceph-mon[35948]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T14:54:12.765 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-02T14:54:12.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.003+0000 7fa338200640 1 -- 172.21.15.62:0/385382246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3301048d0 msgr2=0x7fa330108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:12.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.003+0000 7fa338200640 1 --2- 172.21.15.62:0/385382246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3301048d0 0x7fa330108910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009900 tx=0x7fa31c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:54:12.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.003+0000 7fa338200640 1 -- 172.21.15.62:0/385382246 shutdown_connections 2024-04-02T14:54:12.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.003+0000 7fa338200640 1 --2- 172.21.15.62:0/385382246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330108ff0 0x7fa330110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:12.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.003+0000 7fa338200640 1 --2- 172.21.15.62:0/385382246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3301048d0 0x7fa330108910 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:12.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.003+0000 7fa338200640 1 --2- 172.21.15.62:0/385382246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa330103f20 0x7fa330104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:12.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.003+0000 7fa338200640 1 -- 172.21.15.62:0/385382246 >> 172.21.15.62:0/385382246 conn(0x7fa3300ff780 msgr2=0x7fa330101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:12.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.003+0000 7fa338200640 1 -- 172.21.15.62:0/385382246 shutdown_connections 2024-04-02T14:54:12.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.003+0000 7fa338200640 1 -- 172.21.15.62:0/385382246 wait complete. 2024-04-02T14:54:12.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.004+0000 7fa338200640 1 Processor -- start 2024-04-02T14:54:12.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.004+0000 7fa338200640 1 -- start start 2024-04-02T14:54:12.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.005+0000 7fa338200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330103f20 0x7fa33019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:12.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.005+0000 7fa338200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa330108ff0 0x7fa33019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:12.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.005+0000 7fa338200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33019eeb0 0x7fa33019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:12.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.005+0000 7fa338200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa330114150 con 0x7fa330108ff0 2024-04-02T14:54:12.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.005+0000 7fa338200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa330113fd0 con 0x7fa33019eeb0 2024-04-02T14:54:12.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.005+0000 7fa338200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301142d0 con 0x7fa330103f20 2024-04-02T14:54:12.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.005+0000 7fa336200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33019eeb0 0x7fa33019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:12.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.005+0000 7fa334e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa330108ff0 0x7fa33019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:12.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.005+0000 7fa336200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33019eeb0 0x7fa33019f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:35474/0 (socket says 172.21.15.62:35474) 2024-04-02T14:54:12.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.005+0000 7fa335800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330103f20 0x7fa33019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:12.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.005+0000 7fa336200640 1 -- 172.21.15.62:0/1386984336 learned_addr learned my addr 172.21.15.62:0/1386984336 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:12.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.006+0000 7fa336200640 1 -- 172.21.15.62:0/1386984336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330103f20 msgr2=0x7fa33019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:12.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.006+0000 7fa336200640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330103f20 0x7fa33019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:12.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.006+0000 7fa336200640 1 -- 172.21.15.62:0/1386984336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa330108ff0 msgr2=0x7fa33019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:12.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.006+0000 7fa336200640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa330108ff0 0x7fa33019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:12.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.006+0000 7fa336200640 1 -- 172.21.15.62:0/1386984336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33019f9a0 con 0x7fa33019eeb0 2024-04-02T14:54:12.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.006+0000 7fa336200640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33019eeb0 0x7fa33019f360 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa32800eb10 tx=0x7fa32800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:12.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.006+0000 7fa32e000640 1 -- 172.21.15.62:0/1386984336 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328004b20 con 0x7fa33019eeb0 2024-04-02T14:54:12.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.006+0000 7fa32e000640 1 -- 172.21.15.62:0/1386984336 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa328004cc0 con 0x7fa33019eeb0 2024-04-02T14:54:12.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.006+0000 7fa338200640 1 -- 172.21.15.62:0/1386984336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33019fc90 con 0x7fa33019eeb0 2024-04-02T14:54:12.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.006+0000 7fa32e000640 1 -- 172.21.15.62:0/1386984336 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3280107f0 con 0x7fa33019eeb0 2024-04-02T14:54:13.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.007+0000 7fa338200640 1 -- 172.21.15.62:0/1386984336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3301a7450 con 0x7fa33019eeb0 2024-04-02T14:54:13.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.008+0000 7fa338200640 1 -- 172.21.15.62:0/1386984336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33004f860 con 0x7fa33019eeb0 2024-04-02T14:54:13.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.010+0000 7fa32e000640 1 -- 172.21.15.62:0/1386984336 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa3280109b0 con 0x7fa33019eeb0 2024-04-02T14:54:13.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.011+0000 7fa32e000640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa3140a3e40 0x7fa3140a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:13.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.011+0000 7fa32e000640 1 -- 172.21.15.62:0/1386984336 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fa3280d0cb0 con 0x7fa33019eeb0 2024-04-02T14:54:13.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.011+0000 7fa335800640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa3140a3e40 0x7fa3140a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:13.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.012+0000 7fa335800640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa3140a3e40 0x7fa3140a6300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa320009bd0 tx=0x7fa320005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:13.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.014+0000 7fa32e000640 1 -- 172.21.15.62:0/1386984336 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa32809d310 con 0x7fa33019eeb0 2024-04-02T14:54:13.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.245+0000 7fa338200640 1 -- 172.21.15.62:0/1386984336 --> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa3301048d0 con 0x7fa3140a3e40 2024-04-02T14:54:13.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.262+0000 7fa32e000640 1 -- 172.21.15.62:0/1386984336 <== mgr.25273 v2:172.21.15.62:6800/110874546 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fa3301048d0 con 0x7fa3140a3e40 2024-04-02T14:54:13.253 INFO:teuthology.orchestra.run.smithi062.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-02T00:00:00", "created": "2024-04-02T14:44:10", "first": "2024-04-02T14:45:00", "last": "2024-04-02T14:54:00", "last_pruned": "2024-04-02T14:54:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-02T14:54:13.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 -- 172.21.15.62:0/1386984336 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa3140a3e40 msgr2=0x7fa3140a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:13.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa3140a3e40 0x7fa3140a6300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa320009bd0 tx=0x7fa320005f90 comp rx=0 tx=0).stop 2024-04-02T14:54:13.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 -- 172.21.15.62:0/1386984336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33019eeb0 msgr2=0x7fa33019f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:13.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33019eeb0 0x7fa33019f360 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa32800eb10 tx=0x7fa32800c2d0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 -- 172.21.15.62:0/1386984336 shutdown_connections 2024-04-02T14:54:13.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fa3140a3e40 0x7fa3140a6300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33019eeb0 0x7fa33019f360 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa330108ff0 0x7fa33019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 --2- 172.21.15.62:0/1386984336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330103f20 0x7fa33019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 -- 172.21.15.62:0/1386984336 >> 172.21.15.62:0/1386984336 conn(0x7fa3300ff780 msgr2=0x7fa3301002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 -- 172.21.15.62:0/1386984336 shutdown_connections 2024-04-02T14:54:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.265+0000 7fa338200640 1 -- 172.21.15.62:0/1386984336 wait complete. 2024-04-02T14:54:13.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-02T14:54:13.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.498+0000 7f33bb600640 1 -- 172.21.15.62:0/1147242253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33b4107ac0 msgr2=0x7f33b4107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:13.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.498+0000 7f33bb600640 1 --2- 172.21.15.62:0/1147242253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33b4107ac0 0x7f33b4107ea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009900 tx=0x7f33a402bc30 comp rx=0 tx=0).stop 2024-04-02T14:54:13.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.498+0000 7f33bb600640 1 -- 172.21.15.62:0/1147242253 shutdown_connections 2024-04-02T14:54:13.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.498+0000 7f33bb600640 1 --2- 172.21.15.62:0/1147242253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33b4102470 0x7f33b410e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.498+0000 7f33bb600640 1 --2- 172.21.15.62:0/1147242253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33b4101ab0 0x7f33b4101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.498+0000 7f33bb600640 1 --2- 172.21.15.62:0/1147242253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33b4107ac0 0x7f33b4107ea0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.498+0000 7f33bb600640 1 -- 172.21.15.62:0/1147242253 >> 172.21.15.62:0/1147242253 conn(0x7f33b40fd5c0 msgr2=0x7f33b40ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.498+0000 7f33bb600640 1 -- 172.21.15.62:0/1147242253 shutdown_connections 2024-04-02T14:54:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.498+0000 7f33bb600640 1 -- 172.21.15.62:0/1147242253 wait complete. 2024-04-02T14:54:13.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.499+0000 7f33bb600640 1 Processor -- start 2024-04-02T14:54:13.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.499+0000 7f33bb600640 1 -- start start 2024-04-02T14:54:13.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.499+0000 7f33bb600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33b4101ab0 0x7f33b419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:13.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.499+0000 7f33bb600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33b4102470 0x7f33b419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:13.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.499+0000 7f33bb600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33b4107ac0 0x7f33b419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:13.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.499+0000 7f33bb600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f33b4111ff0 con 0x7f33b4102470 2024-04-02T14:54:13.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.499+0000 7f33bb600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f33b4111e70 con 0x7f33b4101ab0 2024-04-02T14:54:13.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.499+0000 7f33bb600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f33b4112170 con 0x7f33b4107ac0 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.500+0000 7f33b9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33b4107ac0 0x7f33b419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.500+0000 7f33b3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33b4102470 0x7f33b419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.500+0000 7f33b8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33b4101ab0 0x7f33b419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.500+0000 7f33b9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33b4107ac0 0x7f33b419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:54668/0 (socket says 172.21.15.62:54668) 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.500+0000 7f33b9600640 1 -- 172.21.15.62:0/1985528106 learned_addr learned my addr 172.21.15.62:0/1985528106 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.500+0000 7f33b3e00640 1 -- 172.21.15.62:0/1985528106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33b4107ac0 msgr2=0x7f33b419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.500+0000 7f33b3e00640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33b4107ac0 0x7f33b419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.500+0000 7f33b3e00640 1 -- 172.21.15.62:0/1985528106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33b4101ab0 msgr2=0x7f33b419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.500+0000 7f33b3e00640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33b4101ab0 0x7f33b419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.500+0000 7f33b3e00640 1 -- 172.21.15.62:0/1985528106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b419f5b0 con 0x7f33b4102470 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.500+0000 7f33b9600640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33b4107ac0 0x7f33b419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.501+0000 7f33b3e00640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33b4102470 0x7f33b419ab00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f339c00eb00 tx=0x7f339c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:13.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.501+0000 7f33b1600640 1 -- 172.21.15.62:0/1985528106 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c010430 con 0x7f33b4102470 2024-04-02T14:54:13.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.501+0000 7f33bb600640 1 -- 172.21.15.62:0/1985528106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b419f8a0 con 0x7f33b4102470 2024-04-02T14:54:13.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.501+0000 7f33b1600640 1 -- 172.21.15.62:0/1985528106 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f339c010ad0 con 0x7f33b4102470 2024-04-02T14:54:13.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.501+0000 7f33b1600640 1 -- 172.21.15.62:0/1985528106 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c017c10 con 0x7f33b4102470 2024-04-02T14:54:13.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.501+0000 7f33bb600640 1 -- 172.21.15.62:0/1985528106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b41a7130 con 0x7f33b4102470 2024-04-02T14:54:13.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.502+0000 7f33bb600640 1 -- 172.21.15.62:0/1985528106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b404f860 con 0x7f33b4102470 2024-04-02T14:54:13.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.504+0000 7f33b1600640 1 -- 172.21.15.62:0/1985528106 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f339c010c80 con 0x7f33b4102470 2024-04-02T14:54:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.506+0000 7f33b1600640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33980a3dc0 0x7f33980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.506+0000 7f33b1600640 1 -- 172.21.15.62:0/1985528106 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f339c0d1210 con 0x7f33b4102470 2024-04-02T14:54:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.506+0000 7f33b8c00640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33980a3dc0 0x7f33980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.507+0000 7f33b8c00640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33980a3dc0 0x7f33980a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f33a4006010 tx=0x7f33a4005e40 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:13.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.511+0000 7f33b1600640 1 -- 172.21.15.62:0/1985528106 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f339c09d870 con 0x7f33b4102470 2024-04-02T14:54:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.742+0000 7f33bb600640 1 -- 172.21.15.62:0/1985528106 --> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f33b40fe9b0 con 0x7f33980a3dc0 2024-04-02T14:54:13.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.757+0000 7f33b1600640 1 -- 172.21.15.62:0/1985528106 <== mgr.25273 v2:172.21.15.62:6800/110874546 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f33b40fe9b0 con 0x7f33980a3dc0 2024-04-02T14:54:13.748 INFO:teuthology.orchestra.run.smithi062.stdout:/ 1m 6m3h 2024-04-02T14:54:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.760+0000 7f33bb600640 1 -- 172.21.15.62:0/1985528106 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33980a3dc0 msgr2=0x7f33980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.760+0000 7f33bb600640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33980a3dc0 0x7f33980a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f33a4006010 tx=0x7f33a4005e40 comp rx=0 tx=0).stop 2024-04-02T14:54:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.760+0000 7f33bb600640 1 -- 172.21.15.62:0/1985528106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33b4102470 msgr2=0x7f33b419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.760+0000 7f33bb600640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33b4102470 0x7f33b419ab00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f339c00eb00 tx=0x7f339c00c2d0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.760+0000 7f33bb600640 1 -- 172.21.15.62:0/1985528106 shutdown_connections 2024-04-02T14:54:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.760+0000 7f33bb600640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f33980a3dc0 0x7f33980a6280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.760+0000 7f33bb600640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33b4107ac0 0x7f33b419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.760+0000 7f33bb600640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33b4102470 0x7f33b419ab00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.760+0000 7f33bb600640 1 --2- 172.21.15.62:0/1985528106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33b4101ab0 0x7f33b419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.760+0000 7f33bb600640 1 -- 172.21.15.62:0/1985528106 >> 172.21.15.62:0/1985528106 conn(0x7f33b40fd5c0 msgr2=0x7f33b410d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.760+0000 7f33bb600640 1 -- 172.21.15.62:0/1985528106 shutdown_connections 2024-04-02T14:54:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:13.761+0000 7f33bb600640 1 -- 172.21.15.62:0/1985528106 wait complete. 2024-04-02T14:54:13.769 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-02T14:54:13.781 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-02T14:54:13.781 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-04-02T14:54:13.782 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-02T14:54:13.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:54:14.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.064+0000 7f22b5600640 1 -- 172.21.15.62:0/970748481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b0103f20 msgr2=0x7f22b0104300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.064+0000 7f22b5600640 1 --2- 172.21.15.62:0/970748481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b0103f20 0x7f22b0104300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009860 tx=0x7f22a002bb90 comp rx=0 tx=0).stop 2024-04-02T14:54:14.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.064+0000 7f22b5600640 1 -- 172.21.15.62:0/970748481 shutdown_connections 2024-04-02T14:54:14.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.064+0000 7f22b5600640 1 --2- 172.21.15.62:0/970748481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22b0108ff0 0x7f22b0110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.064+0000 7f22b5600640 1 --2- 172.21.15.62:0/970748481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b01048d0 0x7f22b0108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.064+0000 7f22b5600640 1 --2- 172.21.15.62:0/970748481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b0103f20 0x7f22b0104300 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.064+0000 7f22b5600640 1 -- 172.21.15.62:0/970748481 >> 172.21.15.62:0/970748481 conn(0x7f22b00ff780 msgr2=0x7f22b0101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:14.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.064+0000 7f22b5600640 1 -- 172.21.15.62:0/970748481 shutdown_connections 2024-04-02T14:54:14.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.064+0000 7f22b5600640 1 -- 172.21.15.62:0/970748481 wait complete. 2024-04-02T14:54:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.065+0000 7f22b5600640 1 Processor -- start 2024-04-02T14:54:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.065+0000 7f22b5600640 1 -- start start 2024-04-02T14:54:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.065+0000 7f22b5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b0103f20 0x7f22b019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.065+0000 7f22b5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22b0108ff0 0x7f22b019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.065+0000 7f22b5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b019efc0 0x7f22b019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22b5600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01141e0 con 0x7f22b0108ff0 2024-04-02T14:54:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22b5600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f22b0114060 con 0x7f22b019efc0 2024-04-02T14:54:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22b5600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f22b0114360 con 0x7f22b0103f20 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22ae000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22b0108ff0 0x7f22b019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22ae000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22b0108ff0 0x7f22b019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45468/0 (socket says 172.21.15.62:45468) 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22ae000640 1 -- 172.21.15.62:0/2149070143 learned_addr learned my addr 172.21.15.62:0/2149070143 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22aea00640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b0103f20 0x7f22b019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22af400640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b019efc0 0x7f22b019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22ae000640 1 -- 172.21.15.62:0/2149070143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b0103f20 msgr2=0x7f22b019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22ae000640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b0103f20 0x7f22b019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22ae000640 1 -- 172.21.15.62:0/2149070143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b019efc0 msgr2=0x7f22b019f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22ae000640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b019efc0 0x7f22b019f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22ae000640 1 -- 172.21.15.62:0/2149070143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b019fab0 con 0x7f22b0108ff0 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.066+0000 7f22af400640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b019efc0 0x7f22b019f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.067+0000 7f22ae000640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22b0108ff0 0x7f22b019ac40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f229800ebc0 tx=0x7f229800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.067+0000 7f22a7400640 1 -- 172.21.15.62:0/2149070143 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298004b20 con 0x7f22b0108ff0 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.067+0000 7f22b5600640 1 -- 172.21.15.62:0/2149070143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b019fda0 con 0x7f22b0108ff0 2024-04-02T14:54:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.067+0000 7f22a7400640 1 -- 172.21.15.62:0/2149070143 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2298004cc0 con 0x7f22b0108ff0 2024-04-02T14:54:14.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.067+0000 7f22a7400640 1 -- 172.21.15.62:0/2149070143 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980107f0 con 0x7f22b0108ff0 2024-04-02T14:54:14.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.067+0000 7f22b5600640 1 -- 172.21.15.62:0/2149070143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b01a7590 con 0x7f22b0108ff0 2024-04-02T14:54:14.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.068+0000 7f22b5600640 1 -- 172.21.15.62:0/2149070143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b0068d40 con 0x7f22b0108ff0 2024-04-02T14:54:14.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.071+0000 7f22a7400640 1 -- 172.21.15.62:0/2149070143 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2298005000 con 0x7f22b0108ff0 2024-04-02T14:54:14.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.073+0000 7f22a7400640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f22940a3d20 0x7f22940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:14.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.073+0000 7f22a7400640 1 -- 172.21.15.62:0/2149070143 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f22980d0fe0 con 0x7f22b0108ff0 2024-04-02T14:54:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.073+0000 7f22aea00640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f22940a3d20 0x7f22940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.074+0000 7f22aea00640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f22940a3d20 0x7f22940a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f22a002c1a0 tx=0x7f22a00058d0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.077+0000 7f22a7400640 1 -- 172.21.15.62:0/2149070143 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f229809d640 con 0x7f22b0108ff0 2024-04-02T14:54:14.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:13 smithi062 ceph-mon[32116]: pgmap v317: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:54:14.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:13 smithi113 ceph-mon[36135]: pgmap v317: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:54:14.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:13 smithi204 ceph-mon[35948]: pgmap v317: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:54:14.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.288+0000 7f22b5600640 1 -- 172.21.15.62:0/2149070143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22b01048d0 con 0x7f22b0108ff0 2024-04-02T14:54:14.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.291+0000 7f22a7400640 1 -- 172.21.15.62:0/2149070143 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23047 (secure 0 0 0) 0x7f229809cf40 con 0x7f22b0108ff0 2024-04-02T14:54:14.282 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:54:14.283 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":99,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:54:11.768040+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/129789951":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/79979639":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/566820061":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3400451184":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:54:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 -- 172.21.15.62:0/2149070143 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f22940a3d20 msgr2=0x7f22940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f22940a3d20 0x7f22940a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f22a002c1a0 tx=0x7f22a00058d0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 -- 172.21.15.62:0/2149070143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22b0108ff0 msgr2=0x7f22b019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22b0108ff0 0x7f22b019ac40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f229800ebc0 tx=0x7f229800c2d0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 -- 172.21.15.62:0/2149070143 shutdown_connections 2024-04-02T14:54:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f22940a3d20 0x7f22940a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b019efc0 0x7f22b019f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22b0108ff0 0x7f22b019ac40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 --2- 172.21.15.62:0/2149070143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b0103f20 0x7f22b019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 -- 172.21.15.62:0/2149070143 >> 172.21.15.62:0/2149070143 conn(0x7f22b00ff780 msgr2=0x7f22b0101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:14.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 -- 172.21.15.62:0/2149070143 shutdown_connections 2024-04-02T14:54:14.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.294+0000 7f22b5600640 1 -- 172.21.15.62:0/2149070143 wait complete. 2024-04-02T14:54:14.306 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T14:41:20.030393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T14:41:20.917405+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:54:14.307 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:54:14.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.548+0000 7ffb4de00640 1 -- 172.21.15.62:0/3911117259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb48108b20 msgr2=0x7ffb4810cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.548+0000 7ffb4de00640 1 --2- 172.21.15.62:0/3911117259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb48108b20 0x7ffb4810cb40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009900 tx=0x7ffb3402bc40 comp rx=0 tx=0).stop 2024-04-02T14:54:14.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.548+0000 7ffb4de00640 1 -- 172.21.15.62:0/3911117259 shutdown_connections 2024-04-02T14:54:14.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.548+0000 7ffb4de00640 1 --2- 172.21.15.62:0/3911117259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb4810d220 0x7ffb48114df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.548+0000 7ffb4de00640 1 --2- 172.21.15.62:0/3911117259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb48108b20 0x7ffb4810cb40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.548+0000 7ffb4de00640 1 --2- 172.21.15.62:0/3911117259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb48108170 0x7ffb48108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.548+0000 7ffb4de00640 1 -- 172.21.15.62:0/3911117259 >> 172.21.15.62:0/3911117259 conn(0x7ffb480776d0 msgr2=0x7ffb48079af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:14.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.548+0000 7ffb4de00640 1 -- 172.21.15.62:0/3911117259 shutdown_connections 2024-04-02T14:54:14.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.548+0000 7ffb4de00640 1 -- 172.21.15.62:0/3911117259 wait complete. 2024-04-02T14:54:14.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.549+0000 7ffb4de00640 1 Processor -- start 2024-04-02T14:54:14.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.550+0000 7ffb4de00640 1 -- start start 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.550+0000 7ffb4de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb48108170 0x7ffb481a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.550+0000 7ffb4de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb4810d220 0x7ffb481a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.550+0000 7ffb4de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb4819e690 0x7ffb4819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.550+0000 7ffb4de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffb481183f0 con 0x7ffb48108170 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.550+0000 7ffb4de00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffb48118270 con 0x7ffb4810d220 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.550+0000 7ffb4de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffb48118570 con 0x7ffb4819e690 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.550+0000 7ffb4ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb48108170 0x7ffb481a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.550+0000 7ffb4ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb48108170 0x7ffb481a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45494/0 (socket says 172.21.15.62:45494) 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.550+0000 7ffb4ca00640 1 -- 172.21.15.62:0/2877655937 learned_addr learned my addr 172.21.15.62:0/2877655937 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.551+0000 7ffb47e00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb4810d220 0x7ffb481a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.551+0000 7ffb4d400640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb4819e690 0x7ffb4819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.551+0000 7ffb4ca00640 1 -- 172.21.15.62:0/2877655937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb4819e690 msgr2=0x7ffb4819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.551+0000 7ffb4ca00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb4819e690 0x7ffb4819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.551+0000 7ffb4ca00640 1 -- 172.21.15.62:0/2877655937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb4810d220 msgr2=0x7ffb481a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.551+0000 7ffb4ca00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb4810d220 0x7ffb481a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.551+0000 7ffb4ca00640 1 -- 172.21.15.62:0/2877655937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4819f290 con 0x7ffb48108170 2024-04-02T14:54:14.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.551+0000 7ffb4ca00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb48108170 0x7ffb481a45d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e9f0 tx=0x7ffb3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:14.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.552+0000 7ffb45600640 1 -- 172.21.15.62:0/2877655937 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00cd80 con 0x7ffb48108170 2024-04-02T14:54:14.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.552+0000 7ffb4de00640 1 -- 172.21.15.62:0/2877655937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb481ab370 con 0x7ffb48108170 2024-04-02T14:54:14.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.552+0000 7ffb45600640 1 -- 172.21.15.62:0/2877655937 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffb3c010930 con 0x7ffb48108170 2024-04-02T14:54:14.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.552+0000 7ffb45600640 1 -- 172.21.15.62:0/2877655937 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c018b00 con 0x7ffb48108170 2024-04-02T14:54:14.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.552+0000 7ffb4de00640 1 -- 172.21.15.62:0/2877655937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb481ab770 con 0x7ffb48108170 2024-04-02T14:54:14.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.553+0000 7ffb4de00640 1 -- 172.21.15.62:0/2877655937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb48109620 con 0x7ffb48108170 2024-04-02T14:54:14.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.555+0000 7ffb45600640 1 -- 172.21.15.62:0/2877655937 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ffb3c018ca0 con 0x7ffb48108170 2024-04-02T14:54:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.557+0000 7ffb45600640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7ffb300a3d20 0x7ffb300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.557+0000 7ffb45600640 1 -- 172.21.15.62:0/2877655937 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7ffb3c0d0c70 con 0x7ffb48108170 2024-04-02T14:54:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.557+0000 7ffb47e00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7ffb300a3d20 0x7ffb300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.558+0000 7ffb47e00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7ffb300a3d20 0x7ffb300a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffb3402c180 tx=0x7ffb34000bc0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:14.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.561+0000 7ffb45600640 1 -- 172.21.15.62:0/2877655937 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ffb3c09d2d0 con 0x7ffb48108170 2024-04-02T14:54:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.767+0000 7ffb4de00640 1 -- 172.21.15.62:0/2877655937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffb48068c60 con 0x7ffb48108170 2024-04-02T14:54:14.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.768+0000 7ffb45600640 1 -- 172.21.15.62:0/2877655937 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7ffb3c09cbd0 con 0x7ffb48108170 2024-04-02T14:54:14.759 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:54:14.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 -- 172.21.15.62:0/2877655937 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7ffb300a3d20 msgr2=0x7ffb300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7ffb300a3d20 0x7ffb300a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffb3402c180 tx=0x7ffb34000bc0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 -- 172.21.15.62:0/2877655937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb48108170 msgr2=0x7ffb481a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb48108170 0x7ffb481a45d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e9f0 tx=0x7ffb3c00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 -- 172.21.15.62:0/2877655937 shutdown_connections 2024-04-02T14:54:14.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7ffb300a3d20 0x7ffb300a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb4819e690 0x7ffb4819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb4810d220 0x7ffb481a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 --2- 172.21.15.62:0/2877655937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb48108170 0x7ffb481a45d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 -- 172.21.15.62:0/2877655937 >> 172.21.15.62:0/2877655937 conn(0x7ffb480776d0 msgr2=0x7ffb48079ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:14.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 -- 172.21.15.62:0/2877655937 shutdown_connections 2024-04-02T14:54:14.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:14.770+0000 7ffb4de00640 1 -- 172.21.15.62:0/2877655937 wait complete. 2024-04-02T14:54:14.779 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T14:54:14.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.004+0000 7f1d26800640 1 -- 172.21.15.62:0/3162002926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d20100730 msgr2=0x7f1d20100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.004+0000 7f1d26800640 1 --2- 172.21.15.62:0/3162002926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d20100730 0x7f1d20100b10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009ca0 tx=0x7f1d1402c100 comp rx=0 tx=0).stop 2024-04-02T14:54:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.004+0000 7f1d26800640 1 -- 172.21.15.62:0/3162002926 shutdown_connections 2024-04-02T14:54:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.004+0000 7f1d26800640 1 --2- 172.21.15.62:0/3162002926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d2010e2e0 0x7f1d201106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.004+0000 7f1d26800640 1 --2- 172.21.15.62:0/3162002926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d201010e0 0x7f1d2010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.004+0000 7f1d26800640 1 --2- 172.21.15.62:0/3162002926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d20100730 0x7f1d20100b10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.004+0000 7f1d26800640 1 -- 172.21.15.62:0/3162002926 >> 172.21.15.62:0/3162002926 conn(0x7f1d200fc350 msgr2=0x7f1d200fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.005+0000 7f1d26800640 1 -- 172.21.15.62:0/3162002926 shutdown_connections 2024-04-02T14:54:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.005+0000 7f1d26800640 1 -- 172.21.15.62:0/3162002926 wait complete. 2024-04-02T14:54:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.005+0000 7f1d26800640 1 Processor -- start 2024-04-02T14:54:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.006+0000 7f1d26800640 1 -- start start 2024-04-02T14:54:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.006+0000 7f1d26800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20100730 0x7f1d2019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.006+0000 7f1d26800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d201010e0 0x7f1d2019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.006+0000 7f1d26800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d2010e2e0 0x7f1d2019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.006+0000 7f1d26800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1d201140a0 con 0x7f1d201010e0 2024-04-02T14:54:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.006+0000 7f1d26800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1d20113f20 con 0x7f1d2010e2e0 2024-04-02T14:54:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.006+0000 7f1d26800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1d20114220 con 0x7f1d20100730 2024-04-02T14:54:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.006+0000 7f1d24a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d201010e0 0x7f1d2019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.006+0000 7f1d25400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20100730 0x7f1d2019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.006+0000 7f1d25400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20100730 0x7f1d2019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:54720/0 (socket says 172.21.15.62:54720) 2024-04-02T14:54:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.006+0000 7f1d25400640 1 -- 172.21.15.62:0/529447859 learned_addr learned my addr 172.21.15.62:0/529447859 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.007+0000 7f1d25e00640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d2010e2e0 0x7f1d2019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.007+0000 7f1d25400640 1 -- 172.21.15.62:0/529447859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d2010e2e0 msgr2=0x7f1d2019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.007+0000 7f1d25400640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d2010e2e0 0x7f1d2019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.007+0000 7f1d25400640 1 -- 172.21.15.62:0/529447859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d201010e0 msgr2=0x7f1d2019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.007+0000 7f1d25400640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d201010e0 0x7f1d2019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.007+0000 7f1d25400640 1 -- 172.21.15.62:0/529447859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2019f580 con 0x7f1d20100730 2024-04-02T14:54:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.007+0000 7f1d24a00640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d201010e0 0x7f1d2019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:54:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.007+0000 7f1d25400640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20100730 0x7f1d2019a5e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005e30 tx=0x7f1d140346f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:15.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.008+0000 7f1d1e000640 1 -- 172.21.15.62:0/529447859 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14039040 con 0x7f1d20100730 2024-04-02T14:54:15.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.008+0000 7f1d1e000640 1 -- 172.21.15.62:0/529447859 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1d14044070 con 0x7f1d20100730 2024-04-02T14:54:15.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.008+0000 7f1d1e000640 1 -- 172.21.15.62:0/529447859 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d140044d0 con 0x7f1d20100730 2024-04-02T14:54:15.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.008+0000 7f1d26800640 1 -- 172.21.15.62:0/529447859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2019f810 con 0x7f1d20100730 2024-04-02T14:54:15.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.008+0000 7f1d26800640 1 -- 172.21.15.62:0/529447859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d201a7100 con 0x7f1d20100730 2024-04-02T14:54:15.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.009+0000 7f1d26800640 1 -- 172.21.15.62:0/529447859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d200694c0 con 0x7f1d20100730 2024-04-02T14:54:15.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.011+0000 7f1d1e000640 1 -- 172.21.15.62:0/529447859 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1d14032840 con 0x7f1d20100730 2024-04-02T14:54:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.012+0000 7f1d1e000640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1d080a3da0 0x7f1d080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.012+0000 7f1d1e000640 1 -- 172.21.15.62:0/529447859 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f1d140f2b30 con 0x7f1d20100730 2024-04-02T14:54:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.012+0000 7f1d24a00640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1d080a3da0 0x7f1d080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.013+0000 7f1d24a00640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1d080a3da0 0x7f1d080a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1d20102110 tx=0x7f1d0c005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:15.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.017+0000 7f1d1e000640 1 -- 172.21.15.62:0/529447859 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1d140bf2a0 con 0x7f1d20100730 2024-04-02T14:54:15.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:14 smithi062 ceph-mon[32116]: from='client.26615 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:54:15.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:14 smithi062 ceph-mon[32116]: from='client.16485 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:54:15.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:14 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2149070143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:54:15.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:14 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2877655937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:54:15.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:14 smithi113 ceph-mon[36135]: from='client.26615 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:54:15.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:14 smithi113 ceph-mon[36135]: from='client.16485 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:54:15.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:14 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2149070143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:54:15.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:14 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2877655937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:54:15.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.226+0000 7f1d26800640 1 -- 172.21.15.62:0/529447859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1d20109470 con 0x7f1d20100730 2024-04-02T14:54:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.227+0000 7f1d1e000640 1 -- 172.21.15.62:0/529447859 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f1d140beba0 con 0x7f1d20100730 2024-04-02T14:54:15.218 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-02T14:54:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 -- 172.21.15.62:0/529447859 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1d080a3da0 msgr2=0x7f1d080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1d080a3da0 0x7f1d080a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1d20102110 tx=0x7f1d0c005f90 comp rx=0 tx=0).stop 2024-04-02T14:54:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 -- 172.21.15.62:0/529447859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20100730 msgr2=0x7f1d2019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20100730 0x7f1d2019a5e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005e30 tx=0x7f1d140346f0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 -- 172.21.15.62:0/529447859 shutdown_connections 2024-04-02T14:54:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f1d080a3da0 0x7f1d080a6260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d2010e2e0 0x7f1d2019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d201010e0 0x7f1d2019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 --2- 172.21.15.62:0/529447859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20100730 0x7f1d2019a5e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 -- 172.21.15.62:0/529447859 >> 172.21.15.62:0/529447859 conn(0x7f1d200fc350 msgr2=0x7f1d200fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:15.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 -- 172.21.15.62:0/529447859 shutdown_connections 2024-04-02T14:54:15.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.232+0000 7f1d26800640 1 -- 172.21.15.62:0/529447859 wait complete. 2024-04-02T14:54:15.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T14:54:15.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:14 smithi204 ceph-mon[35948]: from='client.26615 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:54:15.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:14 smithi204 ceph-mon[35948]: from='client.16485 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:54:15.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:14 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2149070143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:54:15.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:14 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2877655937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:54:15.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.482+0000 7fb67e600640 1 -- 172.21.15.62:0/1664661264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678103df0 msgr2=0x7fb6781041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:15.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.482+0000 7fb67e600640 1 --2- 172.21.15.62:0/1664661264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678103df0 0x7fb6781041d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009900 tx=0x7fb66c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:54:15.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.483+0000 7fb67e600640 1 -- 172.21.15.62:0/1664661264 shutdown_connections 2024-04-02T14:54:15.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.483+0000 7fb67e600640 1 --2- 172.21.15.62:0/1664661264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb678108ec0 0x7fb678110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.483+0000 7fb67e600640 1 --2- 172.21.15.62:0/1664661264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6781047a0 0x7fb6781087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.483+0000 7fb67e600640 1 --2- 172.21.15.62:0/1664661264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678103df0 0x7fb6781041d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.483+0000 7fb67e600640 1 -- 172.21.15.62:0/1664661264 >> 172.21.15.62:0/1664661264 conn(0x7fb6780ff630 msgr2=0x7fb678101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:15.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.483+0000 7fb67e600640 1 -- 172.21.15.62:0/1664661264 shutdown_connections 2024-04-02T14:54:15.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.483+0000 7fb67e600640 1 -- 172.21.15.62:0/1664661264 wait complete. 2024-04-02T14:54:15.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.484+0000 7fb67e600640 1 Processor -- start 2024-04-02T14:54:15.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.484+0000 7fb67e600640 1 -- start start 2024-04-02T14:54:15.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.484+0000 7fb67e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678103df0 0x7fb67819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.484+0000 7fb67e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb678108ec0 0x7fb67819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.484+0000 7fb67e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb67819ee60 0x7fb67819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.484+0000 7fb67e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb678114060 con 0x7fb678103df0 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.484+0000 7fb67e600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb678113ee0 con 0x7fb67819ee60 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.484+0000 7fb67e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb6781141e0 con 0x7fb678108ec0 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.485+0000 7fb67d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678103df0 0x7fb67819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.485+0000 7fb67d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678103df0 0x7fb67819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45516/0 (socket says 172.21.15.62:45516) 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.485+0000 7fb67d200640 1 -- 172.21.15.62:0/995348584 learned_addr learned my addr 172.21.15.62:0/995348584 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.485+0000 7fb67dc00640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb67819ee60 0x7fb67819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.485+0000 7fb677e00640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb678108ec0 0x7fb67819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.485+0000 7fb67d200640 1 -- 172.21.15.62:0/995348584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb678108ec0 msgr2=0x7fb67819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:15.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.485+0000 7fb67d200640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb678108ec0 0x7fb67819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.485+0000 7fb67d200640 1 -- 172.21.15.62:0/995348584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb67819ee60 msgr2=0x7fb67819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:15.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.485+0000 7fb67d200640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb67819ee60 0x7fb67819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.485+0000 7fb67d200640 1 -- 172.21.15.62:0/995348584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67819f950 con 0x7fb678103df0 2024-04-02T14:54:15.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.486+0000 7fb67d200640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678103df0 0x7fb67819a5a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb66c005ab0 tx=0x7fb66c02dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:15.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.486+0000 7fb675600640 1 -- 172.21.15.62:0/995348584 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c043020 con 0x7fb678103df0 2024-04-02T14:54:15.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.486+0000 7fb675600640 1 -- 172.21.15.62:0/995348584 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb66c004250 con 0x7fb678103df0 2024-04-02T14:54:15.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.486+0000 7fb67e600640 1 -- 172.21.15.62:0/995348584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67819fbe0 con 0x7fb678103df0 2024-04-02T14:54:15.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.486+0000 7fb675600640 1 -- 172.21.15.62:0/995348584 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c003a10 con 0x7fb678103df0 2024-04-02T14:54:15.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.486+0000 7fb67e600640 1 -- 172.21.15.62:0/995348584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6781a7430 con 0x7fb678103df0 2024-04-02T14:54:15.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.487+0000 7fb67e600640 1 -- 172.21.15.62:0/995348584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb678067f80 con 0x7fb678103df0 2024-04-02T14:54:15.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.489+0000 7fb675600640 1 -- 172.21.15.62:0/995348584 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb66c046020 con 0x7fb678103df0 2024-04-02T14:54:15.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.491+0000 7fb675600640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb6600a3da0 0x7fb6600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:15.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.491+0000 7fb675600640 1 -- 172.21.15.62:0/995348584 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fb66c0f2660 con 0x7fb678103df0 2024-04-02T14:54:15.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.491+0000 7fb677e00640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb6600a3da0 0x7fb6600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:15.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.492+0000 7fb677e00640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb6600a3da0 0x7fb6600a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb678102de0 tx=0x7fb668005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:15.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.495+0000 7fb675600640 1 -- 172.21.15.62:0/995348584 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb66c04d020 con 0x7fb678103df0 2024-04-02T14:54:15.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.702+0000 7fb67e600640 1 -- 172.21.15.62:0/995348584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb67810c0f0 con 0x7fb678103df0 2024-04-02T14:54:15.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.703+0000 7fb675600640 1 -- 172.21.15.62:0/995348584 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7fb66c0bedd0 con 0x7fb678103df0 2024-04-02T14:54:15.694 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 -- 172.21.15.62:0/995348584 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb6600a3da0 msgr2=0x7fb6600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb6600a3da0 0x7fb6600a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb678102de0 tx=0x7fb668005f90 comp rx=0 tx=0).stop 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 -- 172.21.15.62:0/995348584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678103df0 msgr2=0x7fb67819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678103df0 0x7fb67819a5a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb66c005ab0 tx=0x7fb66c02dc90 comp rx=0 tx=0).stop 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 -- 172.21.15.62:0/995348584 shutdown_connections 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fb6600a3da0 0x7fb6600a6260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb67819ee60 0x7fb67819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb678108ec0 0x7fb67819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 --2- 172.21.15.62:0/995348584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678103df0 0x7fb67819a5a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 -- 172.21.15.62:0/995348584 >> 172.21.15.62:0/995348584 conn(0x7fb6780ff630 msgr2=0x7fb678101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 -- 172.21.15.62:0/995348584 shutdown_connections 2024-04-02T14:54:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.706+0000 7fb67e600640 1 -- 172.21.15.62:0/995348584 wait complete. 2024-04-02T14:54:15.715 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-02T14:54:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.944+0000 7eff47200640 1 -- 172.21.15.62:0/3178375598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff48106850 msgr2=0x7eff48068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.944+0000 7eff47200640 1 --2- 172.21.15.62:0/3178375598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff48106850 0x7eff48068c60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7eff38009900 tx=0x7eff3802bc30 comp rx=0 tx=0).stop 2024-04-02T14:54:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.944+0000 7eff47200640 1 -- 172.21.15.62:0/3178375598 shutdown_connections 2024-04-02T14:54:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.944+0000 7eff47200640 1 --2- 172.21.15.62:0/3178375598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff48104350 0x7eff48104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.944+0000 7eff47200640 1 --2- 172.21.15.62:0/3178375598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff48106cc0 0x7eff480691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.944+0000 7eff47200640 1 --2- 172.21.15.62:0/3178375598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff48106850 0x7eff48068c60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.944+0000 7eff47200640 1 -- 172.21.15.62:0/3178375598 >> 172.21.15.62:0/3178375598 conn(0x7eff48076410 msgr2=0x7eff48076820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.945+0000 7eff47200640 1 -- 172.21.15.62:0/3178375598 shutdown_connections 2024-04-02T14:54:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.945+0000 7eff47200640 1 -- 172.21.15.62:0/3178375598 wait complete. 2024-04-02T14:54:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.946+0000 7eff47200640 1 Processor -- start 2024-04-02T14:54:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.946+0000 7eff47200640 1 -- start start 2024-04-02T14:54:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.946+0000 7eff47200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff48104350 0x7eff481a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.946+0000 7eff47200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff48106850 0x7eff481a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff47200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff48106cc0 0x7eff4819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff47200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7eff48118350 con 0x7eff48104350 2024-04-02T14:54:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff47200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7eff481181d0 con 0x7eff48106cc0 2024-04-02T14:54:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff47200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff481184d0 con 0x7eff48106850 2024-04-02T14:54:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff45e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff48104350 0x7eff481a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff45e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff48104350 0x7eff481a4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45528/0 (socket says 172.21.15.62:45528) 2024-04-02T14:54:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff45e00640 1 -- 172.21.15.62:0/3257314881 learned_addr learned my addr 172.21.15.62:0/3257314881 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff46800640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff48106cc0 0x7eff4819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff45400640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff48106850 0x7eff481a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff45e00640 1 -- 172.21.15.62:0/3257314881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff48106850 msgr2=0x7eff481a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff45e00640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff48106850 0x7eff481a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff45e00640 1 -- 172.21.15.62:0/3257314881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff48106cc0 msgr2=0x7eff4819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff45e00640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff48106cc0 0x7eff4819e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.947+0000 7eff45e00640 1 -- 172.21.15.62:0/3257314881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4819ed90 con 0x7eff48104350 2024-04-02T14:54:15.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.948+0000 7eff45e00640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff48104350 0x7eff481a4460 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7eff38005ab0 tx=0x7eff3802dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:15.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.948+0000 7eff3ea00640 1 -- 172.21.15.62:0/3257314881 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38043020 con 0x7eff48104350 2024-04-02T14:54:15.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.948+0000 7eff3ea00640 1 -- 172.21.15.62:0/3257314881 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff38004250 con 0x7eff48104350 2024-04-02T14:54:15.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.948+0000 7eff47200640 1 -- 172.21.15.62:0/3257314881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4819f080 con 0x7eff48104350 2024-04-02T14:54:15.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.948+0000 7eff3ea00640 1 -- 172.21.15.62:0/3257314881 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38003a30 con 0x7eff48104350 2024-04-02T14:54:15.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.948+0000 7eff47200640 1 -- 172.21.15.62:0/3257314881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff481ab320 con 0x7eff48104350 2024-04-02T14:54:15.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.950+0000 7eff47200640 1 -- 172.21.15.62:0/3257314881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff48068810 con 0x7eff48104350 2024-04-02T14:54:15.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.952+0000 7eff3ea00640 1 -- 172.21.15.62:0/3257314881 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7eff38046020 con 0x7eff48104350 2024-04-02T14:54:15.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.953+0000 7eff3ea00640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7eff2c0a3d20 0x7eff2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:15.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.954+0000 7eff3ea00640 1 -- 172.21.15.62:0/3257314881 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7eff380f28f0 con 0x7eff48104350 2024-04-02T14:54:15.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.954+0000 7eff45400640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7eff2c0a3d20 0x7eff2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:15.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.954+0000 7eff45400640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7eff2c0a3d20 0x7eff2c0a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7eff48076ba0 tx=0x7eff30009500 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:15.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:15.957+0000 7eff3ea00640 1 -- 172.21.15.62:0/3257314881 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7eff3804d020 con 0x7eff48104350 2024-04-02T14:54:16.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:15 smithi062 ceph-mon[32116]: pgmap v318: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 238 B/s wr, 12 op/s 2024-04-02T14:54:16.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:15 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/529447859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:54:16.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:15 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/995348584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:54:16.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:15 smithi113 ceph-mon[36135]: pgmap v318: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 238 B/s wr, 12 op/s 2024-04-02T14:54:16.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:15 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/529447859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:54:16.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:15 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/995348584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:54:16.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.221+0000 7eff47200640 1 -- 172.21.15.62:0/3257314881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7eff481ab580 con 0x7eff48104350 2024-04-02T14:54:16.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.251+0000 7eff3ea00640 1 -- 172.21.15.62:0/3257314881 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+746060 (secure 0 0 0) 0x7eff380bf060 con 0x7eff48104350 2024-04-02T14:54:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:54:16.251 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:54:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 27, 2024-04-02T14:54:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: "active_gid": 25273, 2024-04-02T14:54:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: "active_name": "x", 2024-04-02T14:54:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addrs": { 2024-04-02T14:54:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6800", 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 110874546 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6801", 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 110874546 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addr": "172.21.15.62:6801/110874546", 2024-04-02T14:54:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "active_change": "2024-04-02T14:44:00.087635+0000", 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "active_mgr_features": 4540701547738038271, 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "available": true, 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "standbys": [ 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 25267, 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "z", 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540701547738038271, 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-04-02T14:54:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-04-02T14:54:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-04-02T14:54:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T14:54:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:54:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-04-02T14:54:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-04-02T14:54:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-04-02T14:54:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-04-02T14:54:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T14:54:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T14:54:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T14:54:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-04-02T14:54:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-04-02T14:54:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-04-02T14:54:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-04-02T14:54:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-04-02T14:54:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-04-02T14:54:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T14:54:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-04-02T14:54:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-04-02T14:54:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-04-02T14:54:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T14:54:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T14:54:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-04-02T14:54:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-04-02T14:54:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-04-02T14:54:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-04-02T14:54:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-04-02T14:54:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T14:54:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T14:54:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T14:54:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T14:54:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T14:54:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T14:54:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T14:54:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T14:54:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T14:54:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T14:54:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-04-02T14:54:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T14:54:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "[]", 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "{}", 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T14:54:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-04-02T14:54:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-04-02T14:54:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-04-02T14:54:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-04-02T14:54:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-04-02T14:54:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-04-02T14:54:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-04-02T14:54:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T14:54:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-04-02T14:54:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-04-02T14:54:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-04-02T14:54:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-04-02T14:54:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-04-02T14:54:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-04-02T14:54:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-04-02T14:54:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-04-02T14:54:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-04-02T14:54:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-04-02T14:54:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-04-02T14:54:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T14:54:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 25283, 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "y", 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540701547738038271, 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T14:54:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-04-02T14:54:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-04-02T14:54:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-04-02T14:54:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-04-02T14:54:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-04-02T14:54:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-04-02T14:54:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-04-02T14:54:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-04-02T14:54:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-04-02T14:54:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-04-02T14:54:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-04-02T14:54:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-04-02T14:54:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T14:54:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T14:54:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-04-02T14:54:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T14:54:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T14:54:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T14:54:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-04-02T14:54:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T14:54:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T14:54:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T14:54:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-04-02T14:54:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-04-02T14:54:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-04-02T14:54:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-04-02T14:54:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T14:54:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "[]", 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "{}", 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T14:54:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T14:54:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T14:54:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T14:54:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-04-02T14:54:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-04-02T14:54:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.857 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.857 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.857 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.858 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:15 smithi204 ceph-mon[35948]: pgmap v318: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 238 B/s wr, 12 op/s 2024-04-02T14:54:16.858 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:15 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/529447859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:54:16.858 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:15 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/995348584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:54:16.861 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.861 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.861 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.861 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-04-02T14:54:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-04-02T14:54:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-04-02T14:54:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-04-02T14:54:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-04-02T14:54:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-04-02T14:54:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-04-02T14:54:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-04-02T14:54:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-04-02T14:54:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T14:54:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T14:54:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-04-02T14:54:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-04-02T14:54:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-04-02T14:54:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T14:54:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-04-02T14:54:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-04-02T14:54:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-04-02T14:54:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-04-02T14:54:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-04-02T14:54:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-04-02T14:54:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T14:54:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-04-02T14:54:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-04-02T14:54:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-04-02T14:54:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T14:54:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T14:54:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-04-02T14:54:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-04-02T14:54:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-04-02T14:54:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "modules": [ 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard", 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "iostat", 2024-04-02T14:54:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "nfs", 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "restful", 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "snap_schedule" 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-04-02T14:54:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-04-02T14:54:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-04-02T14:54:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-04-02T14:54:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T14:54:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-04-02T14:54:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T14:54:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T14:54:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-04-02T14:54:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-04-02T14:54:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-04-02T14:54:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-04-02T14:54:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-04-02T14:54:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-04-02T14:54:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-04-02T14:54:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-04-02T14:54:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-04-02T14:54:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-04-02T14:54:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-04-02T14:54:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-04-02T14:54:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T14:54:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-04-02T14:54:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-04-02T14:54:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T14:54:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T14:54:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T14:54:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T14:54:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T14:54:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T14:54:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T14:54:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T14:54:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T14:54:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "[]", 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "{}", 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T14:54:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T14:54:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-04-02T14:54:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-04-02T14:54:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-04-02T14:54:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-04-02T14:54:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-04-02T14:54:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-04-02T14:54:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-04-02T14:54:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-04-02T14:54:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-04-02T14:54:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T14:54:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-04-02T14:54:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-04-02T14:54:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T14:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-04-02T14:54:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T14:54:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T14:54:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T14:54:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-04-02T14:54:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-04-02T14:54:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-04-02T14:54:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-04-02T14:54:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.268+0000 7eff47200640 1 -- 172.21.15.62:0/3257314881 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7eff2c0a3d20 msgr2=0x7eff2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.268+0000 7eff47200640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7eff2c0a3d20 0x7eff2c0a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7eff48076ba0 tx=0x7eff30009500 comp rx=0 tx=0).stop 2024-04-02T14:54:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.268+0000 7eff47200640 1 -- 172.21.15.62:0/3257314881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff48104350 msgr2=0x7eff481a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.268+0000 7eff47200640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff48104350 0x7eff481a4460 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7eff38005ab0 tx=0x7eff3802dd00 comp rx=0 tx=0).stop 2024-04-02T14:54:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.268+0000 7eff47200640 1 -- 172.21.15.62:0/3257314881 shutdown_connections 2024-04-02T14:54:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.268+0000 7eff47200640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7eff2c0a3d20 0x7eff2c0a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.268+0000 7eff47200640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff48106cc0 0x7eff4819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.268+0000 7eff47200640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff48106850 0x7eff481a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.268+0000 7eff47200640 1 --2- 172.21.15.62:0/3257314881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff48104350 0x7eff481a4460 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.268+0000 7eff47200640 1 -- 172.21.15.62:0/3257314881 >> 172.21.15.62:0/3257314881 conn(0x7eff48076410 msgr2=0x7eff481017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.269+0000 7eff47200640 1 -- 172.21.15.62:0/3257314881 shutdown_connections 2024-04-02T14:54:17.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:16.269+0000 7eff47200640 1 -- 172.21.15.62:0/3257314881 wait complete. 2024-04-02T14:54:17.405 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:16 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3257314881' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T14:54:17.405 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:16 smithi113 ceph-mon[36135]: pgmap v319: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-02T14:54:17.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:16 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3257314881' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T14:54:17.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:16 smithi204 ceph-mon[35948]: pgmap v319: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-02T14:54:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T14:54:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-04-02T14:54:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-04-02T14:54:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-04-02T14:54:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-04-02T14:54:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-04-02T14:54:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-04-02T14:54:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-04-02T14:54:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-04-02T14:54:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T14:54:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-04-02T14:54:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-04-02T14:54:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-04-02T14:54:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-04-02T14:54:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-04-02T14:54:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-04-02T14:54:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-04-02T14:54:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-04-02T14:54:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-04-02T14:54:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-04-02T14:54:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-04-02T14:54:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-04-02T14:54:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T14:54:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-04-02T14:54:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-04-02T14:54:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-04-02T14:54:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-04-02T14:54:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-04-02T14:54:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "services": { 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard": "https://172.21.15.62:8443/" 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "always_on_modules": { 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "octopus": [ 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-04-02T14:54:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "pacific": [ 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-04-02T14:54:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "quincy": [ 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-04-02T14:54:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "reef": [ 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-04-02T14:54:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "squid": [ 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "last_failure_osd_epoch": 91, 2024-04-02T14:54:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "active_clients": [ 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 3638951972 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "libcephsqlite", 2024-04-02T14:54:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2815840294 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-04-02T14:54:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 1234769948 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 1723792251 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:54:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2566311935 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-02T14:54:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-04-02T14:54:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-04-02T14:54:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 4103435725 2024-04-02T14:54:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:54:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-02T14:54:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:54:17.575 DEBUG:tasks.check_counter:Getting stats from a 2024-04-02T14:54:17.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.a perf dump 2024-04-02T14:54:17.576 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:16 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3257314881' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T14:54:17.576 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:16 smithi062 ceph-mon[32116]: pgmap v319: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-02T14:54:18.072 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:54:19.584 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:19 smithi062 ceph-mon[32116]: pgmap v320: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-02T14:54:19.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:19 smithi204 ceph-mon[35948]: pgmap v320: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-02T14:54:19.814 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:19 smithi113 ceph-mon[36135]: pgmap v320: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-02T14:54:20.759 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:54:20.759 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:20.759 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 469, 2024-04-02T14:54:20.759 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 555, 2024-04-02T14:54:20.759 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1973334, 2024-04-02T14:54:20.759 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 1457421, 2024-04-02T14:54:20.759 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 20, 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 5, 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.184992837, 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.059173274, 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.131072922, 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.019184322, 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 554, 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.084299549, 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000152165 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 23, 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000004094, 2024-04-02T14:54:20.760 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000178 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1930144, 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 460432 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 555, 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 910, 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 245106, 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 666377, 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 17, 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.256332056, 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.154157701, 2024-04-02T14:54:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.113749522, 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.003107371, 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 908, 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.123318794, 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000135813 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1084, 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000546457, 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000504 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 156080, 2024-04-02T14:54:20.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 497824 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2175, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1799, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 841642, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 646418, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 8, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.454299321, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.201990414, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.300320728, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.013903067, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1797, 2024-04-02T14:54:20.763 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.208102225, 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000115805 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3660, 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.002834819, 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000774 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.764 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 277, 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.362193934, 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001307559 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 210, 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.022783147, 2024-04-02T14:54:20.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000108491 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 176, 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 173, 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 173, 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.717548218, 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004147677 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 3, 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-02T14:54:20.766 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 11, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 34, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 4, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 2, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 28, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 28, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 151, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 3, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 381, 2024-04-02T14:54:20.767 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 223, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 10, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 9, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 97, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 9, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 66, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 45, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-04-02T14:54:20.768 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 3720, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 120169321, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 3, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 45, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 1, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 1, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 44, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 2, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 35, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 141, 2024-04-02T14:54:20.769 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 143, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 110, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 17, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 170 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 11, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 2, 2024-04-02T14:54:20.770 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 1, 2024-04-02T14:54:20.771 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-04-02T14:54:20.771 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:54:20.771 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 81, 2024-04-02T14:54:20.773 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 80, 2024-04-02T14:54:20.773 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-04-02T14:54:20.773 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-02T14:54:20.773 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-02T14:54:20.773 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:54:20.773 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:54:20.773 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 9, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 447, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-04-02T14:54:20.774 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 447, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 2, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 2, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 2, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 5029823, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-02T14:54:20.775 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 83, 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.292315543, 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.003521874 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 31, 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 197, 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 166, 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 23, 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 62, 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 39, 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 34, 2024-04-02T14:54:20.776 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 265, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 231, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 99, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 99, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 54656, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 223608 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 339, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 31, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 176, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 17, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 192, 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-02T14:54:20.777 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 36, 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.128550593, 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.003570849 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3, 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.004311455, 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001437151 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.778 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 39, 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.038407005, 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000984795 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.779 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.781 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-02T14:54:20.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.781 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.781 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-02T14:54:20.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.781 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.782 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-02T14:54:20.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.782 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.782 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-02T14:54:20.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.782 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 13, 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.014632261, 2024-04-02T14:54:20.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001125558 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.784 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.046361373, 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.046361373 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-02T14:54:20.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.056131008, 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.056131008 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-02T14:54:20.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-02T14:54:20.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 80, 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.429154523, 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.005364431 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.788 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 3, 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 2, 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 99, 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 99, 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 1958, 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:20.789 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:20.790 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:20.791 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 309519, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 141, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 792, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 9, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:20.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 7936, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 165, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 137217, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 840, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.793 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 350, 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 873462, 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 27, 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 350, 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 350, 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.322962853, 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.006637036 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-02T14:54:20.794 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 350, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 417 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 350, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 66, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 284, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 0, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 10, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 21, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 104, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 27, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-02T14:54:20.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 92, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 45, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 20, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:54:20.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-02T14:54:20.797 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 99, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 11, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 12, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 12, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 73, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 0, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 25 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.798 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 6, 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 6, 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 80, 2024-04-02T14:54:20.799 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 80, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 3195, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 2736527, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 3195, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3195, 2024-04-02T14:54:20.800 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2736527, 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:20.801 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 350, 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 881894, 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 350, 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 881894, 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 350, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 350, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 350, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 350, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:20.803 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 80, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 8432, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 80, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 21, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 8432, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:20.804 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x561dfd6e6500": { 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 447, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 835519, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 447, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 83, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 835519, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:20.805 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:20.806 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:20.806 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:20.806 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:54:21.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:21 smithi204 ceph-mon[35948]: pgmap v321: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-02T14:54:21.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:21 smithi062 ceph-mon[32116]: pgmap v321: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-02T14:54:21.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:21 smithi113 ceph-mon[36135]: pgmap v321: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-02T14:54:21.993 INFO:tasks.check_counter:Daemon mds.a mds.exported=9 2024-04-02T14:54:21.993 INFO:tasks.check_counter:Daemon mds.a mds.imported=9 2024-04-02T14:54:21.993 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=3 2024-04-02T14:54:21.993 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:54:21.993 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:54:21.993 DEBUG:tasks.check_counter:Getting stats from d 2024-04-02T14:54:21.993 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.d perf dump 2024-04-02T14:54:22.478 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:54:23.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:23 smithi062 ceph-mon[32116]: pgmap v322: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-02T14:54:23.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:23 smithi204 ceph-mon[35948]: pgmap v322: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-02T14:54:23.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:23 smithi113 ceph-mon[36135]: pgmap v322: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-02T14:54:24.584 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:24 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:54:24.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:24 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:54:24.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:24 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:54:25.009 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:54:25.009 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1045, 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1009, 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 2164400, 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 383334, 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.279940962, 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.086044002, 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.155400531, 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.057550221, 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1009, 2024-04-02T14:54:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.102691089, 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000101775 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1904672, 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 137520 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 568, 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 732, 2024-04-02T14:54:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 140208, 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 630712, 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 7, 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 5, 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.163114938, 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.066526208, 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.036793800, 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.041762719, 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 729, 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.083568728, 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000114634 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 450656 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 814, 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 816, 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 2029563, 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 258853, 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-04-02T14:54:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.170876097, 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.058586766, 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.051771685, 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.042697135, 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 814, 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.071421456, 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000087741 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3437, 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.198552994, 2024-04-02T14:54:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000057769 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-02T14:54:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 220, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 220, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-04-02T14:54:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 220, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 1, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-02T14:54:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:54:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-02T14:54:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-02T14:54:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 999, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 2, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 2, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-02T14:54:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 6011198, 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 6010599, 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 999 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 17, 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 369, 2024-04-02T14:54:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 352, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 4, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 79, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 75, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 220, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 488, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 268, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 50852, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215416 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-02T14:54:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-02T14:54:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 2, 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 2, 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 0, 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 2, 2024-04-02T14:54:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 1961, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 190716, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 67, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 7936, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 165, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 204362, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 1480, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 2153, 2024-04-02T14:54:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 21, 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 3, 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 2153, 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2153, 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.895969695, 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000880617 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2153, 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2154 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 2153, 2024-04-02T14:54:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 2153, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 1336, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 815, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-04-02T14:54:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:54:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-02T14:54:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 99, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 11, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 5, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 5, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-04-02T14:54:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:54:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 0, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 2423, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 4099448, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 2423, 2024-04-02T14:54:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2423, 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 4099448, 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 2153, 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 1816899, 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2153, 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1816899, 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 2153, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 2153, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2153, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2153, 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-02T14:54:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x559249d585a0": { 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-02T14:54:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:25.055 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:54:25.611 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:25 smithi113 ceph-mon[36135]: pgmap v323: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:54:25.611 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:25 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.611 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:25 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.611 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:25 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:25 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:25 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:25 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:25 smithi204 ceph-mon[35948]: pgmap v323: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:54:25.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:25 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:25 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:25 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:25 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:25 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:25 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:25 smithi062 ceph-mon[32116]: pgmap v323: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:54:25.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:25 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:25 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:25 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:25 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:25 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:25 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:25.967 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-04-02T14:54:25.967 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-04-02T14:54:25.967 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-04-02T14:54:25.967 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:54:25.968 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:54:25.968 DEBUG:tasks.check_counter:Getting stats from g 2024-04-02T14:54:25.968 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.g perf dump 2024-04-02T14:54:27.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: pgmap v324: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.336 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.336 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.336 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.336 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.336 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:27 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.371 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:54:27.500 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: pgmap v324: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:27 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.611 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: pgmap v324: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:27.613 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:27 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:28.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:28 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi204 to 174.1M 2024-04-02T14:54:28.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:28 smithi062 ceph-mon[32116]: Unable to set osd_memory_target on smithi204 to 182606233: error parsing value: Value '182606233' is below minimum 939524096 2024-04-02T14:54:28.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:28 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi113 to 174.1M 2024-04-02T14:54:28.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:28 smithi062 ceph-mon[32116]: Unable to set osd_memory_target on smithi113 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T14:54:28.500 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:28 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi204 to 174.1M 2024-04-02T14:54:28.500 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:28 smithi204 ceph-mon[35948]: Unable to set osd_memory_target on smithi204 to 182606233: error parsing value: Value '182606233' is below minimum 939524096 2024-04-02T14:54:28.500 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:28 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi113 to 174.1M 2024-04-02T14:54:28.500 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:28 smithi204 ceph-mon[35948]: Unable to set osd_memory_target on smithi113 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T14:54:28.611 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:28 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi204 to 174.1M 2024-04-02T14:54:28.611 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:28 smithi113 ceph-mon[36135]: Unable to set osd_memory_target on smithi204 to 182606233: error parsing value: Value '182606233' is below minimum 939524096 2024-04-02T14:54:28.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:28 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi113 to 174.1M 2024-04-02T14:54:28.612 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:28 smithi113 ceph-mon[36135]: Unable to set osd_memory_target on smithi113 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T14:54:29.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:29 smithi062 ceph-mon[32116]: pgmap v325: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:29.500 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:29 smithi204 ceph-mon[35948]: pgmap v325: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:29.611 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:29 smithi113 ceph-mon[36135]: pgmap v325: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:29.758 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:54:29.758 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:29.758 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2611, 2024-04-02T14:54:29.758 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2386, 2024-04-02T14:54:29.758 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 2927989, 2024-04-02T14:54:29.758 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 3464469, 2024-04-02T14:54:29.758 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 12, 2024-04-02T14:54:29.758 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.625899404, 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.257441254, 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.439816226, 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.028681840, 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2386, 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.277706998, 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000116390 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4032, 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.002372053, 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000588 2024-04-02T14:54:29.759 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1927760, 2024-04-02T14:54:29.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 150608 2024-04-02T14:54:29.760 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.760 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:29.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 338, 2024-04-02T14:54:29.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 378, 2024-04-02T14:54:29.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 88252, 2024-04-02T14:54:29.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 317575, 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 14, 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.078777403, 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.028156246, 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.026846968, 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.016119046, 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 375, 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.073413502, 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000195769 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 30, 2024-04-02T14:54:29.761 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000005470, 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000182 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3840, 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 145952 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1284, 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2318, 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 515587, 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 1582328, 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 18, 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-04-02T14:54:29.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.571115415, 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.365737995, 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.272753883, 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.010087195, 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2316, 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.373797349, 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000161397 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2532, 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.001346940, 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000531 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 438240, 2024-04-02T14:54:29.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 1371744 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 382, 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.089293082, 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002851552 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 513, 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.049412079, 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000096319 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 508, 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 506, 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 506, 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.725902803, 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.003410875 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.765 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 2, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 11, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 81, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 4, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 4, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 73, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 73, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 364, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 1, 2024-04-02T14:54:29.766 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 1, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 3, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 983, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 661, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 1, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 14, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 12, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 306, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 12, 2024-04-02T14:54:29.767 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 181, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 181, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 1, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 3720, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 120169321, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 3, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 181, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 4, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 4, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 177, 2024-04-02T14:54:29.768 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 69, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 363, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 496, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 438, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 13, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 482 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 15, 2024-04-02T14:54:29.769 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-04-02T14:54:29.770 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 4, 2024-04-02T14:54:29.770 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-04-02T14:54:29.770 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:54:29.770 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 259, 2024-04-02T14:54:29.770 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 255, 2024-04-02T14:54:29.770 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-04-02T14:54:29.770 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-02T14:54:29.770 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-02T14:54:29.770 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:54:29.770 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:54:29.770 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 12, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 1355, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-04-02T14:54:29.771 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 1355, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 3, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 3, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 2, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 6697650, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 98, 2024-04-02T14:54:29.772 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.511480719, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.005219191 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 57, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 498, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 441, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 28, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 75, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 47, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 81, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 700, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 619, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 1, 2024-04-02T14:54:29.773 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 276, 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 275, 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 57748, 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 223608 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 829, 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 12, 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 508, 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 17, 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 309, 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 78, 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.233220547, 2024-04-02T14:54:29.774 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002990007 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2, 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000638354, 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000319177 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.775 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 144, 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.130771522, 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000908135 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000189224, 2024-04-02T14:54:29.776 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000189224 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.777 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.778 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-02T14:54:29.778 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.778 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.778 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 22, 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.041424418, 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001882928 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4, 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.185928306, 2024-04-02T14:54:29.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.046482076 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 255, 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.133730432, 2024-04-02T14:54:29.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004446001 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 3, 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 2, 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2024-04-02T14:54:29.785 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 348, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 348, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 615, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:29.786 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:29.788 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 334615, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 168, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 792, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 9, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 7936, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 165, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:29.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 242146, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2268, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 679, 2024-04-02T14:54:29.790 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 2575615, 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 27, 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 679, 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 679, 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.445960929, 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.003602298 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 679, 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 743 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 679, 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 205, 2024-04-02T14:54:29.791 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 474, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 0, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 10, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 24, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 122, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 28, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 267, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-02T14:54:29.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 181, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 20, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-02T14:54:29.793 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 99, 2024-04-02T14:54:29.794 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 11, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 12, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 12, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 69, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 0, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 22 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-02T14:54:29.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 6, 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 6, 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 255, 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 255, 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-02T14:54:29.796 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 4229, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 3083601, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 4229, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 4229, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 3083601, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-02T14:54:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 679, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 2602786, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 679, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2602786, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.798 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 679, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 679, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 679, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 679, 2024-04-02T14:54:29.799 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 255, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 27171, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 255, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-02T14:54:29.800 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 24, 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 27171, 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x557022754500": { 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 1355, 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 2503346, 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:29.801 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 1355, 2024-04-02T14:54:29.802 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-02T14:54:29.802 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-02T14:54:29.802 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 98, 2024-04-02T14:54:29.802 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2503346, 2024-04-02T14:54:29.802 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:29.802 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:29.802 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:29.802 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:29.802 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:29.802 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:29.802 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:54:30.814 INFO:tasks.check_counter:Daemon mds.g mds.exported=12 2024-04-02T14:54:30.814 INFO:tasks.check_counter:Daemon mds.g mds.imported=12 2024-04-02T14:54:30.815 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=3 2024-04-02T14:54:30.815 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:54:30.815 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:54:30.815 DEBUG:tasks.check_counter:Getting stats from j 2024-04-02T14:54:30.815 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.j perf dump 2024-04-02T14:54:31.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:30 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:31.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:30 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:31.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:30 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:31.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:30 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:31.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:30 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:31.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:30 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:31.270 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:54:32.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: pgmap v326: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:54:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:54:32.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:31 smithi062 ceph-mon[32116]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:32.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: pgmap v326: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:32.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:54:32.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:54:32.112 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:31 smithi113 ceph-mon[36135]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:32.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: pgmap v326: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:32.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:54:32.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:54:32.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:54:32.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:31 smithi204 ceph-mon[35948]: from='mgr.25273 ' entity='mgr.x' 2024-04-02T14:54:33.028 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:32 smithi062 ceph-mon[32116]: Adjusting osd_memory_target on smithi062 to 176.8M 2024-04-02T14:54:33.028 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:32 smithi062 ceph-mon[32116]: Unable to set osd_memory_target on smithi062 to 185396019: error parsing value: Value '185396019' is below minimum 939524096 2024-04-02T14:54:33.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:32 smithi113 ceph-mon[36135]: Adjusting osd_memory_target on smithi062 to 176.8M 2024-04-02T14:54:33.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:32 smithi113 ceph-mon[36135]: Unable to set osd_memory_target on smithi062 to 185396019: error parsing value: Value '185396019' is below minimum 939524096 2024-04-02T14:54:33.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:32 smithi204 ceph-mon[35948]: Adjusting osd_memory_target on smithi062 to 176.8M 2024-04-02T14:54:33.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:32 smithi204 ceph-mon[35948]: Unable to set osd_memory_target on smithi062 to 185396019: error parsing value: Value '185396019' is below minimum 939524096 2024-04-02T14:54:33.632 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-02T14:54:33.632 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:33.632 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2024-04-02T14:54:33.632 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2024-04-02T14:54:33.632 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2024-04-02T14:54:33.632 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2024-04-02T14:54:33.632 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-04-02T14:54:33.632 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-04-02T14:54:33.632 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.004840268, 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000702135, 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:54:33.633 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2, 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 169, 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 320, 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 185376, 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.037347489, 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.025563797, 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.005729376, 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T14:54:33.634 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 166, 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.037702762, 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000227125 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 185376 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:33.635 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 237, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 223, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1734448, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 136672, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.109757984, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.032777658, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.098489681, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.009904823, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 223, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.024028051, 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000107749 2024-04-02T14:54:33.636 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1734448, 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 136672 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:33.637 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:33.638 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:33.639 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148714, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 30, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:33.640 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-02T14:54:33.641 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 235, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1698562, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 235, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 235, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1698562, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-02T14:54:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-02T14:54:33.643 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-02T14:54:33.643 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:33.643 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-02T14:54:33.643 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-02T14:54:34.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:33 smithi062 ceph-mon[32116]: pgmap v327: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:34.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:33 smithi113 ceph-mon[36135]: pgmap v327: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:34.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:33 smithi204 ceph-mon[35948]: pgmap v327: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:34.591 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-02T14:54:34.591 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-02T14:54:34.592 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-02T14:54:34.592 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-02T14:54:34.592 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-02T14:54:34.592 DEBUG:tasks.check_counter:Getting stats from b 2024-04-02T14:54:34.592 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.b perf dump 2024-04-02T14:54:34.841 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:54:36.063 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:35 smithi113 ceph-mon[36135]: pgmap v328: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:36.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:35 smithi062 ceph-mon[32116]: pgmap v328: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:36.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:35 smithi204 ceph-mon[35948]: pgmap v328: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:36.698 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-02T14:54:36.698 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:36.698 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 13150, 2024-04-02T14:54:36.698 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 13729, 2024-04-02T14:54:36.698 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 6567284, 2024-04-02T14:54:36.698 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 40652792, 2024-04-02T14:54:36.698 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 11, 2024-04-02T14:54:36.698 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 3, 2024-04-02T14:54:36.698 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.898988827, 2024-04-02T14:54:36.698 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.316954176, 2024-04-02T14:54:36.698 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.301361647, 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.264733294, 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 13734, 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 4.816335100, 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000350686 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 7904, 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.001260841, 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000159 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1980992, 2024-04-02T14:54:36.699 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 153184 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 62453, 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 106690, 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 25700957, 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 103541300, 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 29, 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 6, 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 4.578007880, 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 2.046818003, 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 3.002131574, 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.576641649, 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:36.700 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 106694, 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 10.240613962, 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000095981 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 114735, 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.017515559, 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000152 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 17743920, 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 44879776 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 14596, 2024-04-02T14:54:36.701 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 14581, 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 5203536, 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 71617834, 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 74, 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 13, 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 1.189759857, 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.496900944, 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.207734445, 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.427932250, 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 14567, 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 10.253622081, 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000703893 2024-04-02T14:54:36.702 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 338, 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000048467, 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000143 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 98928, 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 350576 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 1, 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-MDSRank": { 2024-04-02T14:54:36.703 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8729, 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 33.276016288, 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.003812122 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-PurgeQueue": { 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 18951, 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 2.079731143, 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000109742 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:36.704 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": { 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "request": 23302, 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "reply": 23243, 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "reply_latency": { 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 23243, 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 60.748881215, 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002613642 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_reply": 0, 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "forward": 59, 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_complete": 3, 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_keys": 1, 2024-04-02T14:54:36.705 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_commit": 2288, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_split": 21, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_merge": 1, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes": 16635, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_top": 9779, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_bottom": 6543, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pin_tail": 313, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pinned": 2438, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_expired": 697, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_with_caps": 3, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": 3, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "subtrees": 14, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse": 23550, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_hit": 23449, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_forward": 0, 2024-04-02T14:54:36.706 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_discover": 0, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_remote_ino": 0, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_lock": 0, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "load_cent": 0, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "q": 0, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "exported": 51, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "exported_inodes": 694, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "imported": 58, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "imported_inodes": 1280, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_dir_fetch": 0, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_backtrace_fetch": 8846, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_peer_discover": 0, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rfiles": 4, 2024-04-02T14:54:36.707 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rbytes": 234, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rsnaps": 6, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_fetch": 8846, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_set_tag": 0, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_inodes": 791, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_base_inodes": 6, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dirfrag_rstats": 785, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_file_inodes": 8055, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_inode_file_caps": 88, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_revoke": 2784, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_grant": 23559, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:54:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps": 18465, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps_dirty": 15343, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_cap_release": 621, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "process_request_cap_release": 12416 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_cache": { 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update": 0, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update_receipt": 0, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_try_discover": 0, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_send_discover": 0, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_handle_discover": 56, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays": 601, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_delayed": 0, 2024-04-02T14:54:36.709 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_created": 10634, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_enqueued": 10033, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_reintegrated": 0, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_migrated": 0, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_processing": 0, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_started": 0, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_completed": 0, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_enqueue_scrub": 6, 2024-04-02T14:54:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_exportdir": 51, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_flush": 0, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragmentdir": 22, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragstats": 0, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_inodestats": 0 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_log": { 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "evlrg": 0, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "evadd": 56875, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "evex": 54377, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "evtrm": 54377, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "ev": 2498, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "evexg": 0, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "evexd": 12529, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "segadd": 84, 2024-04-02T14:54:36.711 INFO:teuthology.orchestra.run.smithi113.stdout: "segex": 80, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "segtrm": 80, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "seg": 4, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "segmjr": 1, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "segexg": 0, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "segexd": 18, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "expos": 134696193, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "wrpos": 140711628, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "rdpos": 0, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "jlat": { 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1939, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 93.258897544, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.048096388 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.712 INFO:teuthology.orchestra.run.smithi113.stdout: "replayed": 0 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_mem": { 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 7921, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "ino+": 18656, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "ino-": 10735, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "dir": 770, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "dir+": 817, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "dir-": 47, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "dn": 16635, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "dn+": 27253, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "dn-": 10618, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "cap": 3, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "cap+": 11344, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "cap-": 11341, 2024-04-02T14:54:36.713 INFO:teuthology.orchestra.run.smithi113.stdout: "rss": 132580, 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "heap": 231800 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_server": { 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_client_request": 26419, 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_server_request": 1495, 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_request": 23302, 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_session": 82, 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_peer_request": 3081, 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "req_create_latency": { 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 10715, 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 21.764057608, 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002031176 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getattr_latency": { 2024-04-02T14:54:36.714 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 150, 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.086198145, 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000574654 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getfilelock_latency": { 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "req_link_latency": { 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookup_latency": { 2024-04-02T14:54:36.715 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 62, 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.022529039, 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000363371 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookuphash_latency": { 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupino_latency": { 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000189325, 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000189325 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupname_latency": { 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.716 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupparent_latency": { 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupsnap_latency": { 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lssnap_latency": { 2024-04-02T14:54:36.717 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 10, 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.003781608, 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000378160 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mkdir_latency": { 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 623, 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.311241073, 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002104720 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mknod_latency": { 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mksnap_latency": { 2024-04-02T14:54:36.718 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 10, 2024-04-02T14:54:36.719 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.017539372, 2024-04-02T14:54:36.719 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.101753937 2024-04-02T14:54:36.719 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.721 INFO:teuthology.orchestra.run.smithi113.stdout: "req_open_latency": { 2024-04-02T14:54:36.721 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000200457, 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000200457 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: "req_readdir_latency": { 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 999, 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.431642192, 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001433075 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rename_latency": { 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2, 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.002849169, 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001424584 2024-04-02T14:54:36.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "req_renamesnap_latency": { 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "req_snapdiff_latency": { 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmdir_latency": { 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 608, 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 11.997830590, 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.019733273 2024-04-02T14:54:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmsnap_latency": { 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4, 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.034617071, 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.008654267 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmxattr_latency": { 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setattr_latency": { 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 24, 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 2.625552897, 2024-04-02T14:54:36.724 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.109398037 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setdirlayout_latency": { 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setfilelock_latency": { 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setlayout_latency": { 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setxattr_latency": { 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8, 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.020636330, 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002579541 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: "req_symlink_latency": { 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.726 INFO:teuthology.orchestra.run.smithi113.stdout: "req_unlink_latency": { 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 10026, 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 20.430016339, 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002037703 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getvxattr_latency": { 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_sessions": { 2024-04-02T14:54:36.727 INFO:teuthology.orchestra.run.smithi113.stdout: "session_count": 2, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "session_add": 5, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "session_remove": 3, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_open": 2, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_stale": 0, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "total_load": 165, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "average_load": 82, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "avg_session_uptime": 605, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "mdthresh_evicted": 0 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:36.728 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:36.729 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:36.730 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 5409893, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 9233, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 1408, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 16, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:36.731 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 7936, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 165, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 31982267, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 509382, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "objecter": { 2024-04-02T14:54:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: "op_active": 0, 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "op_laggy": 0, 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send": 36987, 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send_bytes": 162192769, 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "op_resend": 37, 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "op_reply": 36987, 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "op_latency": { 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 36987, 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 2782.255812279, 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.075222532 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "op_inflight": 0, 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "oplen_avg": { 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 36987, 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 66701 2024-04-02T14:54:36.733 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "op": 36987, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "op_r": 8985, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "op_w": 28002, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "op_rmw": 0, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "op_pg": 0, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_stat": 1493, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_create": 12708, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_read": 129, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_write": 2062, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writefull": 123, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writesame": 0, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_append": 0, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_zero": 0, 2024-04-02T14:54:36.734 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_truncate": 21, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_delete": 10666, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_mapext": 0, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_sparse_read": 0, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_clonerange": 0, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_getxattr": 8856, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_setxattr": 25416, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_rmxattr": 0, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_call": 0, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_watch": 0, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_notify": 0, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls": 0, 2024-04-02T14:54:36.735 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_other": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_active": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_send": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_resend": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_ping": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_active": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_send": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_resend": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_active": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_send": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_resend": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_active": 0, 2024-04-02T14:54:36.736 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_send": 0, 2024-04-02T14:54:36.737 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_resend": 0, 2024-04-02T14:54:36.737 INFO:teuthology.orchestra.run.smithi113.stdout: "command_active": 0, 2024-04-02T14:54:36.737 INFO:teuthology.orchestra.run.smithi113.stdout: "command_send": 0, 2024-04-02T14:54:36.737 INFO:teuthology.orchestra.run.smithi113.stdout: "command_resend": 0, 2024-04-02T14:54:36.737 INFO:teuthology.orchestra.run.smithi113.stdout: "map_epoch": 99, 2024-04-02T14:54:36.737 INFO:teuthology.orchestra.run.smithi113.stdout: "map_full": 0, 2024-04-02T14:54:36.737 INFO:teuthology.orchestra.run.smithi113.stdout: "map_inc": 12, 2024-04-02T14:54:36.737 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_sessions": 12, 2024-04-02T14:54:36.737 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_open": 12, 2024-04-02T14:54:36.737 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_close": 0, 2024-04-02T14:54:36.737 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_laggy": 0, 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_wr": 4429, 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_rd": 8, 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_del": 790 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: "oft": { 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_objs": 1, 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_updates": 0, 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_removes": 0 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: "osd-slow-ops": { 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_ops_count": 0 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.738 INFO:teuthology.orchestra.run.smithi113.stdout: "purge_queue": { 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops": 0, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops_high_water": 40, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing": 0, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_high_water": 6, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed_ops": 10083, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed": 10033, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_item_in_journal": 0 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 90195, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 27231633, 2024-04-02T14:54:36.739 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 90195, 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 90195, 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 27231633, 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_bytes": { 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 36987, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 163259354, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 36987, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 163259354, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_ops": { 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 1024, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 36987, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 36987, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 36987, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 36987, 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:36.742 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 10033, 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 1066637, 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 10033, 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:36.743 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 123, 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 1066637, 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle-0x562d50eb6500": { 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 56875, 2024-04-02T14:54:36.744 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 136517324, 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 56875, 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 1939, 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 136517324, 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:36.745 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-02T14:54:37.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:36 smithi204 ceph-mon[35948]: pgmap v329: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:37.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:36 smithi062 ceph-mon[32116]: pgmap v329: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:37.361 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:36 smithi113 ceph-mon[36135]: pgmap v329: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:37.505 INFO:tasks.check_counter:Daemon mds.b mds.exported=51 2024-04-02T14:54:37.505 INFO:tasks.check_counter:Daemon mds.b mds.imported=58 2024-04-02T14:54:37.505 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-02T14:54:37.505 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-02T14:54:37.506 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-02T14:54:37.506 DEBUG:tasks.check_counter:Getting stats from e 2024-04-02T14:54:37.506 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.e perf dump 2024-04-02T14:54:37.752 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:54:39.398 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 2329, 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 2330, 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 1492776, 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 740311, 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 2, 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.174799858, 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.056469016, 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.044536924, 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.059920718, 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2328, 2024-04-02T14:54:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.072839663, 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000031288 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 8, 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 173, 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 3900, 2024-04-02T14:54:39.400 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 457178, 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 4, 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.010944715, 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.006911982, 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.001969441, 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000050034, 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 171, 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.023895377, 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000139739 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.401 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 456480 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 272, 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 238, 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 1902732, 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 139310, 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 3, 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 2, 2024-04-02T14:54:39.402 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.048729968, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.014382691, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.039883813, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.005541583, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 237, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.013973562, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000058960 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1902464, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 138992 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.403 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 1, 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-MDSRank": { 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 3941, 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.090565190, 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000022980 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-PurgeQueue": { 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-02T14:54:39.404 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": { 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "request": 0, 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "reply": 0, 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "reply_latency": { 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_reply": 0, 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "forward": 0, 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_complete": 0, 2024-04-02T14:54:39.405 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_keys": 0, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_commit": 0, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_split": 0, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_merge": 0, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes": 96, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_top": 0, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_bottom": 96, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pin_tail": 0, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pinned": 96, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_expired": 0, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_with_caps": 0, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": 0, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "subtrees": 1, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse": 0, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_hit": 0, 2024-04-02T14:54:39.406 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_forward": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_discover": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_remote_ino": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_lock": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "load_cent": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "q": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "exported": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "exported_inodes": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "imported": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "imported_inodes": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_dir_fetch": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_peer_discover": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rfiles": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rbytes": 0, 2024-04-02T14:54:39.407 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rsnaps": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_set_tag": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_file_inodes": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps": 0, 2024-04-02T14:54:39.408 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_cap_release": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "process_request_cap_release": 0 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_cache": { 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update_receipt": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_try_discover": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_send_discover": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_handle_discover": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_delayed": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_created": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_enqueued": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_reintegrated": 0, 2024-04-02T14:54:39.409 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_migrated": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_processing": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_started": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_completed": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_exportdir": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_flush": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragstats": 0, 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_inodestats": 0 2024-04-02T14:54:39.410 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_log": { 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "evlrg": 0, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "evadd": 0, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "evex": 0, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "evtrm": 0, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "ev": 447, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "evexg": 0, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "evexd": 0, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "segadd": 0, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "segex": 0, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "segtrm": 0, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "seg": 2, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "segmjr": 2, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "segexg": 0, 2024-04-02T14:54:39.411 INFO:teuthology.orchestra.run.smithi113.stdout: "segexd": 0, 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "expos": 4194304, 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "wrpos": 5029823, 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "rdpos": 5027735, 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "jlat": { 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "replayed": 447 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_mem": { 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 12, 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "ino+": 195, 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "ino-": 183, 2024-04-02T14:54:39.412 INFO:teuthology.orchestra.run.smithi113.stdout: "dir": 3, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "dir+": 60, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "dir-": 57, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "dn": 96, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "dn+": 245, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "dn-": 149, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "cap": 0, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "cap+": 0, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "cap-": 0, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "rss": 51736, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "heap": 215416 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_server": { 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_client_request": 0, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_server_request": 0, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_request": 0, 2024-04-02T14:54:39.413 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_session": 0, 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_peer_request": 0, 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "req_create_latency": { 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getattr_latency": { 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getfilelock_latency": { 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.414 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "req_link_latency": { 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookup_latency": { 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookuphash_latency": { 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.415 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupino_latency": { 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupname_latency": { 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupparent_latency": { 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.416 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupsnap_latency": { 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lssnap_latency": { 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mkdir_latency": { 2024-04-02T14:54:39.417 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.418 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.418 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.418 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.418 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mknod_latency": { 2024-04-02T14:54:39.418 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.418 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.418 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.418 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mksnap_latency": { 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "req_open_latency": { 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "req_readdir_latency": { 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.419 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rename_latency": { 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "req_renamesnap_latency": { 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "req_snapdiff_latency": { 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.420 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmdir_latency": { 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmsnap_latency": { 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmxattr_latency": { 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.421 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setattr_latency": { 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setdirlayout_latency": { 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setfilelock_latency": { 2024-04-02T14:54:39.422 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setlayout_latency": { 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setxattr_latency": { 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "req_symlink_latency": { 2024-04-02T14:54:39.423 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "req_unlink_latency": { 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getvxattr_latency": { 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.424 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_sessions": { 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "session_count": 1, 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "session_add": 3, 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "session_remove": 2, 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_open": 1, 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_stale": 0, 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "total_load": 0, 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "average_load": 0, 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "avg_session_uptime": 2223, 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "mdthresh_evicted": 0 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:39.425 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:39.426 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:39.427 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 182432, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 58, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:39.428 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 7936, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 165, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 103268, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 740, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "objecter": { 2024-04-02T14:54:39.429 INFO:teuthology.orchestra.run.smithi113.stdout: "op_active": 0, 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "op_laggy": 0, 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send": 2331, 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send_bytes": 21, 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "op_resend": 3, 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "op_reply": 2331, 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "op_latency": { 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2331, 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.394969189, 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000598442 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "op_inflight": 0, 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "oplen_avg": { 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2331, 2024-04-02T14:54:39.430 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 2332 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "op": 2331, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "op_r": 2331, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "op_w": 0, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "op_rmw": 0, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "op_pg": 0, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_stat": 1496, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_create": 0, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_read": 833, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_write": 0, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writefull": 0, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writesame": 0, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_append": 0, 2024-04-02T14:54:39.431 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_zero": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_truncate": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_delete": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_mapext": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_sparse_read": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_clonerange": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_getxattr": 1, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_setxattr": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_rmxattr": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_call": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_watch": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_notify": 0, 2024-04-02T14:54:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_other": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_active": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_send": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_resend": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_ping": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_active": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_send": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_resend": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_active": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_send": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_resend": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_active": 0, 2024-04-02T14:54:39.433 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_send": 0, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_resend": 0, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "command_active": 0, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "command_send": 0, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "command_resend": 0, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "map_epoch": 99, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "map_full": 0, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "map_inc": 11, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_sessions": 4, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_open": 4, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_close": 0, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_laggy": 0, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_wr": 0, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_rd": 2, 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_del": 0 2024-04-02T14:54:39.434 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "oft": { 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_objs": 0, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_updates": 0, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_removes": 0 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "osd-slow-ops": { 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_ops_count": 0 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "purge_queue": { 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops": 0, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing": 0, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_high_water": 0, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed_ops": 0, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed": 0, 2024-04-02T14:54:39.435 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_item_in_journal": 0 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 2605, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 3148149, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 2605, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 2605, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 3148149, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.436 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_bytes": { 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 2331, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 835524, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 2331, 2024-04-02T14:54:39.437 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 835524, 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_ops": { 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 1024, 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:39.438 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 2331, 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 2331, 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 2331, 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 2331, 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-02T14:54:39.439 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 0, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 0, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle-0x556d3a8f45a0": { 2024-04-02T14:54:39.440 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 0, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 0, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:39.441 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:39.442 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-02T14:54:39.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:39 smithi204 ceph-mon[35948]: pgmap v330: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:39.752 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:39 smithi113 ceph-mon[36135]: pgmap v330: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:39.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:39 smithi062 ceph-mon[32116]: pgmap v330: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:40.227 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-02T14:54:40.228 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-02T14:54:40.228 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-02T14:54:40.228 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:54:40.228 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:54:40.228 DEBUG:tasks.check_counter:Getting stats from h 2024-04-02T14:54:40.228 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.h perf dump 2024-04-02T14:54:40.468 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:54:41.735 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:41 smithi113 ceph-mon[36135]: pgmap v331: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:41.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:41 smithi204 ceph-mon[35948]: pgmap v331: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:41.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:41 smithi062 ceph-mon[32116]: pgmap v331: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:42.117 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-02T14:54:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 3767, 2024-04-02T14:54:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 3932, 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 1662148, 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 5808240, 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 15, 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 5, 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.274122420, 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.128068484, 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.144328359, 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.019864432, 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 3930, 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.098909691, 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000025167 2024-04-02T14:54:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 6351, 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000993267, 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000156 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 661, 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 756, 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 2010835, 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 646213, 2024-04-02T14:54:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 12, 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 5, 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.081996589, 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.026709369, 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.051291988, 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.012983700, 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 756, 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.059424713, 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000078604 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4, 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000459, 2024-04-02T14:54:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000114 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1918464, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 477264 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 2700, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 4779, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 1086434, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 3164225, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 17, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 6, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.196616644, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.085182192, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.130566180, 2024-04-02T14:54:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.010939656, 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4776, 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.094609595, 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000019809 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 5121, 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000771109, 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000150 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 879936, 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 2416992 2024-04-02T14:54:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 1, 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-MDSRank": { 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 730, 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.979238759, 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001341422 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-PurgeQueue": { 2024-04-02T14:54:42.123 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1176, 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.167470762, 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000142407 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": { 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "request": 1194, 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "reply": 1194, 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "reply_latency": { 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1194, 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 2.005363830, 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001679534 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.124 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_reply": 0, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "forward": 0, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_complete": 0, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_keys": 0, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_commit": 11, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_split": 0, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_merge": 0, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes": 41, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_top": 7, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_bottom": 4, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pin_tail": 30, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pinned": 31, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_expired": 756, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_with_caps": 0, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": 0, 2024-04-02T14:54:42.125 INFO:teuthology.orchestra.run.smithi113.stdout: "subtrees": 4, 2024-04-02T14:54:42.126 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse": 2435, 2024-04-02T14:54:42.126 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_hit": 1538, 2024-04-02T14:54:42.126 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_forward": 0, 2024-04-02T14:54:42.126 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_discover": 27, 2024-04-02T14:54:42.126 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:54:42.126 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_remote_ino": 0, 2024-04-02T14:54:42.126 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_lock": 0, 2024-04-02T14:54:42.126 INFO:teuthology.orchestra.run.smithi113.stdout: "load_cent": 0, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "q": 0, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "exported": 22, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "exported_inodes": 686, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "imported": 22, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "imported_inodes": 351, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_dir_fetch": 0, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_backtrace_fetch": 565, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_peer_discover": 0, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rfiles": 3720, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rbytes": 120169321, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rsnaps": 3, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_fetch": 565, 2024-04-02T14:54:42.127 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_set_tag": 0, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_inodes": 2, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dirfrag_rstats": 2, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_file_inodes": 563, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_revoke": 34, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_grant": 855, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps": 636, 2024-04-02T14:54:42.128 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps_dirty": 612, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_cap_release": 16, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "process_request_cap_release": 1194 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_cache": { 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update": 0, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update_receipt": 0, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_try_discover": 0, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_send_discover": 21, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_handle_discover": 0, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays": 2, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_delayed": 0, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_created": 590, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_enqueued": 588, 2024-04-02T14:54:42.129 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_reintegrated": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_migrated": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_processing": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_started": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_completed": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_exportdir": 22, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_flush": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragstats": 0, 2024-04-02T14:54:42.130 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_inodestats": 0 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_log": { 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "evlrg": 0, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "evadd": 2553, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "evex": 0, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "evtrm": 0, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "ev": 2553, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "evexg": 0, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "evexd": 0, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "segadd": 5, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "segex": 0, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "segtrm": 0, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "seg": 5, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "segmjr": 2, 2024-04-02T14:54:42.131 INFO:teuthology.orchestra.run.smithi113.stdout: "segexg": 0, 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "segexd": 0, 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "expos": 4194304, 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "wrpos": 8759246, 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "rdpos": 0, 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "jlat": { 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 136, 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.138989301, 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.008374921 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "replayed": 0 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_mem": { 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 39, 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "ino+": 1010, 2024-04-02T14:54:42.132 INFO:teuthology.orchestra.run.smithi113.stdout: "ino-": 971, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "dir": 29, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "dir+": 91, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "dir-": 62, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "dn": 42, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "dn+": 1386, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "dn-": 1344, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "cap": 0, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "cap+": 605, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "cap-": 605, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "rss": 60312, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "heap": 223608 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_server": { 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_client_request": 2073, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_server_request": 16, 2024-04-02T14:54:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_request": 1194, 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_session": 12, 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_peer_request": 875, 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "req_create_latency": { 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 239, 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.402626778, 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001684630 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getattr_latency": { 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getfilelock_latency": { 2024-04-02T14:54:42.134 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: "req_link_latency": { 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookup_latency": { 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 328, 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.121932769, 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000371746 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.135 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookuphash_latency": { 2024-04-02T14:54:42.136 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.136 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.136 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.136 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.136 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupino_latency": { 2024-04-02T14:54:42.136 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.136 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.136 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.136 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.136 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupname_latency": { 2024-04-02T14:54:42.136 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.137 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.137 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.137 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.137 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupparent_latency": { 2024-04-02T14:54:42.137 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.137 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.137 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.137 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.137 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupsnap_latency": { 2024-04-02T14:54:42.137 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.138 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.138 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.138 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.138 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lssnap_latency": { 2024-04-02T14:54:42.138 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.138 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.139 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.139 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.139 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mkdir_latency": { 2024-04-02T14:54:42.139 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.140 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.140 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.140 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.140 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mknod_latency": { 2024-04-02T14:54:42.140 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.140 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.140 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.140 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.141 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mksnap_latency": { 2024-04-02T14:54:42.141 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.141 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.141 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.141 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.141 INFO:teuthology.orchestra.run.smithi113.stdout: "req_open_latency": { 2024-04-02T14:54:42.141 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.141 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.141 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.142 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.142 INFO:teuthology.orchestra.run.smithi113.stdout: "req_readdir_latency": { 2024-04-02T14:54:42.142 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 37, 2024-04-02T14:54:42.142 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.023929251, 2024-04-02T14:54:42.142 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000646736 2024-04-02T14:54:42.142 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.142 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rename_latency": { 2024-04-02T14:54:42.142 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: "req_renamesnap_latency": { 2024-04-02T14:54:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: "req_snapdiff_latency": { 2024-04-02T14:54:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmdir_latency": { 2024-04-02T14:54:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2, 2024-04-02T14:54:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.049636893, 2024-04-02T14:54:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.024818446 2024-04-02T14:54:42.145 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.145 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmsnap_latency": { 2024-04-02T14:54:42.145 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.145 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.145 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.145 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.145 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmxattr_latency": { 2024-04-02T14:54:42.145 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.145 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.146 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.146 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.146 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setattr_latency": { 2024-04-02T14:54:42.146 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.146 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.146 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.146 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.146 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setdirlayout_latency": { 2024-04-02T14:54:42.147 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.147 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.147 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.147 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.147 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setfilelock_latency": { 2024-04-02T14:54:42.147 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.147 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.147 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.148 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.148 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setlayout_latency": { 2024-04-02T14:54:42.148 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.148 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.148 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.148 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.148 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setxattr_latency": { 2024-04-02T14:54:42.148 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.148 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.149 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.149 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.149 INFO:teuthology.orchestra.run.smithi113.stdout: "req_symlink_latency": { 2024-04-02T14:54:42.149 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.149 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.149 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.149 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.149 INFO:teuthology.orchestra.run.smithi113.stdout: "req_unlink_latency": { 2024-04-02T14:54:42.149 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 588, 2024-04-02T14:54:42.150 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.407238139, 2024-04-02T14:54:42.150 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002393262 2024-04-02T14:54:42.150 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.150 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:54:42.150 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:54:42.150 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getvxattr_latency": { 2024-04-02T14:54:42.150 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.150 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.151 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.151 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:42.151 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.151 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_sessions": { 2024-04-02T14:54:42.151 INFO:teuthology.orchestra.run.smithi113.stdout: "session_count": 1, 2024-04-02T14:54:42.151 INFO:teuthology.orchestra.run.smithi113.stdout: "session_add": 2, 2024-04-02T14:54:42.151 INFO:teuthology.orchestra.run.smithi113.stdout: "session_remove": 1, 2024-04-02T14:54:42.151 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_open": 1, 2024-04-02T14:54:42.151 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_stale": 0, 2024-04-02T14:54:42.151 INFO:teuthology.orchestra.run.smithi113.stdout: "total_load": 768, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "average_load": 768, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "avg_session_uptime": 2227, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "mdthresh_evicted": 0 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:42.152 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:42.153 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 309603, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 142, 2024-04-02T14:54:42.154 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 792, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 9, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 7936, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 165, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 174493, 2024-04-02T14:54:42.155 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 1403, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "objecter": { 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "op_active": 0, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "op_laggy": 0, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send": 1466, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send_bytes": 4711254, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "op_resend": 29, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "op_reply": 1466, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "op_latency": { 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1466, 2024-04-02T14:54:42.156 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 4.385903537, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002991748 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "op_inflight": 0, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "oplen_avg": { 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1466, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1546 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "op": 1466, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "op_r": 597, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "op_w": 869, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "op_rmw": 0, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "op_pg": 0, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_stat": 0, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_create": 10, 2024-04-02T14:54:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_read": 32, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_write": 168, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writefull": 31, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writesame": 0, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_append": 0, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_zero": 0, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_truncate": 0, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_delete": 601, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_mapext": 0, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_sparse_read": 0, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_clonerange": 0, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_getxattr": 565, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_setxattr": 20, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_rmxattr": 0, 2024-04-02T14:54:42.158 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_call": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_watch": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_notify": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_other": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_active": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_send": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_resend": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_ping": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_active": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_send": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_resend": 0, 2024-04-02T14:54:42.159 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_active": 0, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_send": 0, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_resend": 0, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_active": 0, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_send": 0, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_resend": 0, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "command_active": 0, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "command_send": 0, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "command_resend": 0, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "map_epoch": 99, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "map_full": 0, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "map_inc": 11, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_sessions": 12, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_open": 12, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_close": 0, 2024-04-02T14:54:42.160 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_laggy": 0, 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_wr": 90, 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_rd": 0, 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_del": 29 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: "oft": { 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_objs": 1, 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_updates": 0, 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_removes": 0 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: "osd-slow-ops": { 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_ops_count": 0 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.161 INFO:teuthology.orchestra.run.smithi113.stdout: "purge_queue": { 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops": 0, 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops_high_water": 6, 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing": 0, 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_high_water": 6, 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed_ops": 588, 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed": 588, 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_item_in_journal": 0 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 7124, 2024-04-02T14:54:42.162 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 4002775, 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 7124, 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 7124, 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 4002775, 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_bytes": { 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:42.163 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 1466, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 4773434, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 1466, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 4773434, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.164 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:42.165 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.165 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_ops": { 2024-04-02T14:54:42.165 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:42.165 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 1024, 2024-04-02T14:54:42.165 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:42.165 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:42.165 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:42.165 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:42.165 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:42.165 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 1466, 2024-04-02T14:54:42.165 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 1466, 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 1466, 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 1466, 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 588, 2024-04-02T14:54:42.166 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 62180, 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 588, 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 32, 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 62180, 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle-0x56338385c500": { 2024-04-02T14:54:42.167 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 2553, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 4564942, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 2553, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 136, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 4564942, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:42.168 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:42.169 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:42.169 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:42.169 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:42.169 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-02T14:54:42.915 INFO:tasks.check_counter:Daemon mds.h mds.exported=22 2024-04-02T14:54:42.915 INFO:tasks.check_counter:Daemon mds.h mds.imported=22 2024-04-02T14:54:42.915 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=3 2024-04-02T14:54:42.915 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:54:42.915 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:54:42.915 DEBUG:tasks.check_counter:Getting stats from k 2024-04-02T14:54:42.915 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.k perf dump 2024-04-02T14:54:43.152 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:54:43.567 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:43 smithi113 ceph-mon[36135]: pgmap v332: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:43.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:43 smithi204 ceph-mon[35948]: pgmap v332: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:43.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:43 smithi062 ceph-mon[32116]: pgmap v332: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:44.780 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-02T14:54:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 794, 2024-04-02T14:54:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 796, 2024-04-02T14:54:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 266729, 2024-04-02T14:54:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 252493, 2024-04-02T14:54:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-04-02T14:54:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 2, 2024-04-02T14:54:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.095682795, 2024-04-02T14:54:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.029374972, 2024-04-02T14:54:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.024769492, 2024-04-02T14:54:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.031502122, 2024-04-02T14:54:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 794, 2024-04-02T14:54:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.043114117, 2024-04-02T14:54:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000054299 2024-04-02T14:54:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 1760, 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 1726, 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 6846732, 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 613839, 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 4, 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 3, 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.162469610, 2024-04-02T14:54:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.046219434, 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.069688323, 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.049225891, 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1725, 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.066550144, 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000038579 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1903552, 2024-04-02T14:54:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 141552 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 9, 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 177, 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 4692, 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 459589, 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 7, 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 5, 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.010877605, 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.006784717, 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.002179124, 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000060254, 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 175, 2024-04-02T14:54:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.029225999, 2024-04-02T14:54:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000167005 2024-04-02T14:54:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-02T14:54:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 458624 2024-04-02T14:54:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-02T14:54:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 1, 2024-04-02T14:54:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-02T14:54:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-MDSRank": { 2024-04-02T14:54:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-02T14:54:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-02T14:54:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 3774, 2024-04-02T14:54:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.113057265, 2024-04-02T14:54:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000029956 2024-04-02T14:54:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-PurgeQueue": { 2024-04-02T14:54:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-02T14:54:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-02T14:54:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": { 2024-04-02T14:54:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "request": 0, 2024-04-02T14:54:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "reply": 0, 2024-04-02T14:54:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "reply_latency": { 2024-04-02T14:54:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_reply": 0, 2024-04-02T14:54:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "forward": 0, 2024-04-02T14:54:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_complete": 0, 2024-04-02T14:54:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_keys": 0, 2024-04-02T14:54:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_commit": 0, 2024-04-02T14:54:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_split": 0, 2024-04-02T14:54:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_merge": 0, 2024-04-02T14:54:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes": 610, 2024-04-02T14:54:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_top": 0, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_bottom": 610, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pin_tail": 0, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pinned": 610, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_expired": 0, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_with_caps": 0, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": 0, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "subtrees": 1, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse": 0, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_hit": 0, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_forward": 0, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_discover": 0, 2024-04-02T14:54:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_remote_ino": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_lock": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "load_cent": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "q": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "exported": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "exported_inodes": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "imported": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "imported_inodes": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_dir_fetch": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_peer_discover": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rfiles": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rbytes": 0, 2024-04-02T14:54:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rsnaps": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_set_tag": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_file_inodes": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:54:44.793 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps": 0, 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_cap_release": 0, 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: "process_request_cap_release": 0 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_cache": { 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update": 0, 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update_receipt": 0, 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_try_discover": 0, 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_send_discover": 0, 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_handle_discover": 0, 2024-04-02T14:54:44.794 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_delayed": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_created": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_enqueued": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_reintegrated": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_migrated": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_processing": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_started": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_completed": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:54:44.795 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_exportdir": 0, 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_flush": 0, 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragstats": 0, 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_inodestats": 0 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_log": { 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "evlrg": 0, 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "evadd": 0, 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "evex": 0, 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "evtrm": 0, 2024-04-02T14:54:44.796 INFO:teuthology.orchestra.run.smithi113.stdout: "ev": 2553, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "evexg": 0, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "evexd": 0, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "segadd": 0, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "segex": 0, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "segtrm": 0, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "seg": 5, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "segmjr": 2, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "segexg": 0, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "segexd": 0, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "expos": 4194304, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "wrpos": 8759246, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "rdpos": 8757158, 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "jlat": { 2024-04-02T14:54:44.797 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: "replayed": 2553 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_mem": { 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 17, 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: "ino+": 1016, 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: "ino-": 999, 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: "dir": 7, 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: "dir+": 100, 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: "dir-": 93, 2024-04-02T14:54:44.798 INFO:teuthology.orchestra.run.smithi113.stdout: "dn": 610, 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "dn+": 1369, 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "dn-": 759, 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "cap": 0, 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "cap+": 0, 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "cap-": 0, 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "rss": 49708, 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "heap": 215416 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_server": { 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_client_request": 0, 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_server_request": 0, 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_request": 0, 2024-04-02T14:54:44.799 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_session": 0, 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_peer_request": 0, 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "req_create_latency": { 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getattr_latency": { 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getfilelock_latency": { 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.800 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: "req_link_latency": { 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookup_latency": { 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookuphash_latency": { 2024-04-02T14:54:44.801 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupino_latency": { 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupname_latency": { 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.802 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupparent_latency": { 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupsnap_latency": { 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lssnap_latency": { 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.803 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mkdir_latency": { 2024-04-02T14:54:44.804 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.804 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.804 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.804 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.804 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mknod_latency": { 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mksnap_latency": { 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: "req_open_latency": { 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.805 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: "req_readdir_latency": { 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rename_latency": { 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: "req_renamesnap_latency": { 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.806 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "req_snapdiff_latency": { 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmdir_latency": { 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmsnap_latency": { 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.807 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmxattr_latency": { 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setattr_latency": { 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setdirlayout_latency": { 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.808 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setfilelock_latency": { 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setlayout_latency": { 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setxattr_latency": { 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.809 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: "req_symlink_latency": { 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: "req_unlink_latency": { 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:54:44.810 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getvxattr_latency": { 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_sessions": { 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: "session_count": 1, 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: "session_add": 2, 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: "session_remove": 1, 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_open": 1, 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_stale": 0, 2024-04-02T14:54:44.811 INFO:teuthology.orchestra.run.smithi113.stdout: "total_load": 0, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "average_load": 0, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "avg_session_uptime": 2227, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "mdthresh_evicted": 0 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:44.812 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:44.813 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:44.814 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 198680, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 69, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 0, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 7936, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 165, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:44.815 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 485363, 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 3227, 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "objecter": { 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "op_active": 0, 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "op_laggy": 0, 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send": 2282, 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send_bytes": 21, 2024-04-02T14:54:44.816 INFO:teuthology.orchestra.run.smithi113.stdout: "op_resend": 3, 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "op_reply": 2282, 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "op_latency": { 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2282, 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 2.084953929, 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000913652 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "op_inflight": 0, 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "oplen_avg": { 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2282, 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 2283 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "op": 2282, 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "op_r": 2282, 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "op_w": 0, 2024-04-02T14:54:44.817 INFO:teuthology.orchestra.run.smithi113.stdout: "op_rmw": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "op_pg": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_stat": 1435, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_create": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_read": 845, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_write": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writefull": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writesame": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_append": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_zero": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_truncate": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_delete": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_mapext": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_sparse_read": 0, 2024-04-02T14:54:44.818 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_clonerange": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_getxattr": 1, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_setxattr": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_rmxattr": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_call": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_watch": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_notify": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_other": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_active": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_send": 0, 2024-04-02T14:54:44.819 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_resend": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_ping": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_active": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_send": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_resend": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_active": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_send": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_resend": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_active": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_send": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_resend": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "command_active": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "command_send": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "command_resend": 0, 2024-04-02T14:54:44.820 INFO:teuthology.orchestra.run.smithi113.stdout: "map_epoch": 99, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "map_full": 0, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "map_inc": 11, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_sessions": 6, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_open": 6, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_close": 0, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_laggy": 0, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_wr": 0, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_rd": 2, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_del": 0 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "oft": { 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_objs": 0, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_updates": 0, 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_removes": 0 2024-04-02T14:54:44.821 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "osd-slow-ops": { 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_ops_count": 0 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "purge_queue": { 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops": 0, 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing": 0, 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_high_water": 0, 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed_ops": 0, 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed": 0, 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_item_in_journal": 0 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:44.822 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 2559, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 6870826, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 2559, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 2559, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 6870826, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.823 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_bytes": { 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 2282, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 4564947, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 2282, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 4564947, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.824 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_ops": { 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 1024, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 2282, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 2282, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 2282, 2024-04-02T14:54:44.825 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 2282, 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:44.826 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 0, 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 0, 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle-0x55e29f6d65a0": { 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-02T14:54:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 0, 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 0, 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-02T14:54:44.828 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-02T14:54:45.589 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:45 smithi113 ceph-mon[36135]: pgmap v333: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:45.616 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-02T14:54:45.616 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-02T14:54:45.616 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-02T14:54:45.616 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:54:45.616 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:54:45.616 DEBUG:tasks.check_counter:Getting stats from c 2024-04-02T14:54:45.617 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.c perf dump 2024-04-02T14:54:45.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:45 smithi204 ceph-mon[35948]: pgmap v333: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:45.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:45 smithi062 ceph-mon[32116]: pgmap v333: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:45.869 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:54:47.534 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:47 smithi204 ceph-mon[35948]: pgmap v334: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:47.662 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-02T14:54:47.662 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:47.662 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1217, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1217, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 50716493, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 386192, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 8, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.154325187, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.039506968, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.060001583, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.041711162, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1216, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.058669679, 2024-04-02T14:54:47.663 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000048248 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 882, 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 848, 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 56587344, 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 333520, 2024-04-02T14:54:47.664 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.128016883, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.035274524, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.075035619, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.023901193, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 847, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.039924345, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000047136 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.665 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1902896, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 139792 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 828, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 996, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 31873525, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 801345, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 9, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.101722428, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.037674257, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.033241138, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.021487381, 2024-04-02T14:54:47.666 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 992, 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.063417217, 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000063928 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 539312 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-04-02T14:54:47.667 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4278, 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.284741599, 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000066559 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-02T14:54:47.668 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.669 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.669 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.669 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:47.669 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.669 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-04-02T14:54:47.669 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-04-02T14:54:47.669 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-04-02T14:54:47.669 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-04-02T14:54:47.670 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.670 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.670 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.670 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 26495, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 16113, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 10382, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 3475, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-04-02T14:54:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 2, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-04-02T14:54:47.672 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 4, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 234, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-04-02T14:54:47.673 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-04-02T14:54:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-04-02T14:54:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 56875, 2024-04-02T14:54:47.676 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 4, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 17, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 134696193, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 140711628, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 140711516, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 56875 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 7917, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 18644, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 10727, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 766, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 812, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 46, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 26495, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 27245, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 750, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-04-02T14:54:47.678 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 98228, 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215416 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-04-02T14:54:47.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.680 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.680 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.680 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-04-02T14:54:47.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.680 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.680 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.682 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.683 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.684 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.684 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.684 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-04-02T14:54:47.684 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.684 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-04-02T14:54:47.685 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.686 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.687 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-04-02T14:54:47.688 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.689 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.690 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 5, 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 3, 2024-04-02T14:54:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 4, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 3864, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:47.692 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:47.693 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-04-02T14:54:47.694 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 545058, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 8573, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 7936, 2024-04-02T14:54:47.695 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 165, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 38415715, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 488376, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-04-02T14:54:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2646, 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 26, 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 4, 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2646, 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2646, 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 3.138955844, 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001186302 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2646, 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2647 2024-04-02T14:54:47.697 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2646, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2646, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1683, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 960, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-04-02T14:54:47.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 2, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-04-02T14:54:47.699 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-04-02T14:54:47.700 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 99, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 11, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 10, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 10, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.701 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-04-02T14:54:47.702 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2923, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 138897342, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2923, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2923, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 138897342, 2024-04-02T14:54:47.703 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2646, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 136531126, 2024-04-02T14:54:47.704 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2646, 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 136531126, 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:47.705 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2646, 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2646, 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2646, 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2646, 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:47.706 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:47.707 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55758e6ca5a0": { 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:47.708 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:47.709 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:47.709 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:47.709 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-02T14:54:47.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:47 smithi062 ceph-mon[32116]: pgmap v334: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:47.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:47 smithi113 ceph-mon[36135]: pgmap v334: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:48.373 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-02T14:54:48.374 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-02T14:54:48.374 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-02T14:54:48.374 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:54:48.374 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:54:48.374 DEBUG:tasks.check_counter:Getting stats from f 2024-04-02T14:54:48.374 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.f perf dump 2024-04-02T14:54:48.623 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:54:49.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:49 smithi204 ceph-mon[35948]: pgmap v335: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:49.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:49 smithi062 ceph-mon[32116]: pgmap v335: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:49.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:49 smithi113 ceph-mon[36135]: pgmap v335: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:50.311 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-02T14:54:50.312 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:50.312 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 843, 2024-04-02T14:54:50.312 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 845, 2024-04-02T14:54:50.312 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2724163, 2024-04-02T14:54:50.312 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 268138, 2024-04-02T14:54:50.312 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-04-02T14:54:50.312 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-04-02T14:54:50.312 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.088894697, 2024-04-02T14:54:50.312 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.023926769, 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.027230837, 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.029107153, 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 843, 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.033128727, 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000039298 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:54:50.313 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 921, 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 887, 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2062438, 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 346341, 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.120057960, 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.032255506, 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.058225490, 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.034402240, 2024-04-02T14:54:50.314 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 886, 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.035317516, 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000039861 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1903008, 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 140272 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:50.315 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 803, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 969, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 271136, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 713334, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.107519782, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.036195511, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.027192699, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.032267030, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 967, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.070072843, 2024-04-02T14:54:50.316 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000072464 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 459888 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-04-02T14:54:50.317 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-04-02T14:54:50.318 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.318 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-04-02T14:54:50.318 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-02T14:54:50.318 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-02T14:54:50.318 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3876, 2024-04-02T14:54:50.318 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.118628267, 2024-04-02T14:54:50.318 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000030605 2024-04-02T14:54:50.318 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:50.318 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.318 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-04-02T14:54:50.318 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.319 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 317, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 317, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 317, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-04-02T14:54:50.320 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-04-02T14:54:50.321 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:54:50.322 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-04-02T14:54:50.323 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-04-02T14:54:50.324 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-04-02T14:54:50.325 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 1355, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 3, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 6697650, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 6695562, 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-04-02T14:54:50.326 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 1355 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 37, 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 500, 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 463, 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 7, 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 80, 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 73, 2024-04-02T14:54:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 317, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 685, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 368, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 49832, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215416 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-04-02T14:54:50.328 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.329 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.330 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-04-02T14:54:50.333 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.333 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.333 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.333 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.333 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-04-02T14:54:50.333 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.333 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.334 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.334 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.334 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-04-02T14:54:50.334 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.334 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.334 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.334 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.334 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-04-02T14:54:50.334 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.334 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.334 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.335 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.336 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.337 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.338 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.339 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 3, 2024-04-02T14:54:50.340 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 5478, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:50.341 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:50.342 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 191465, 2024-04-02T14:54:50.343 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 89, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 7936, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 165, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 324965, 2024-04-02T14:54:50.344 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2680, 2024-04-02T14:54:50.345 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:50.345 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-04-02T14:54:50.345 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:50.345 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-04-02T14:54:50.345 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.345 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-04-02T14:54:50.345 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-04-02T14:54:50.345 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-04-02T14:54:50.345 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2288, 2024-04-02T14:54:50.345 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-04-02T14:54:50.345 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2288, 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2288, 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 3.414951808, 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001492548 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2288, 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2289 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.346 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2288, 2024-04-02T14:54:50.347 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2288, 2024-04-02T14:54:50.347 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-04-02T14:54:50.347 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-04-02T14:54:50.347 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-04-02T14:54:50.347 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1443, 2024-04-02T14:54:50.347 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-04-02T14:54:50.347 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 843, 2024-04-02T14:54:50.347 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-04-02T14:54:50.347 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-04-02T14:54:50.347 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-04-02T14:54:50.347 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:54:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-04-02T14:54:50.349 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 99, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 11, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 5, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 5, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-04-02T14:54:50.350 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-04-02T14:54:50.351 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2563, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 4810154, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2563, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2563, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 4810154, 2024-04-02T14:54:50.352 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2288, 2024-04-02T14:54:50.353 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2503351, 2024-04-02T14:54:50.354 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2288, 2024-04-02T14:54:50.354 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2503351, 2024-04-02T14:54:50.354 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:50.354 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.354 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.354 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.354 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:50.354 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.354 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-04-02T14:54:50.354 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2288, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2288, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2288, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2288, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.355 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:50.356 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x5573e75945a0": { 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-02T14:54:50.357 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-02T14:54:50.358 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:50.358 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:50.358 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:50.358 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:50.358 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:50.358 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:50.358 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-02T14:54:51.028 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-02T14:54:51.028 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-02T14:54:51.028 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-02T14:54:51.028 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:54:51.028 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:54:51.029 DEBUG:tasks.check_counter:Getting stats from i 2024-04-02T14:54:51.029 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.i perf dump 2024-04-02T14:54:51.274 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:54:51.611 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:51 smithi113 ceph-mon[36135]: pgmap v336: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:51.719 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:51 smithi204 ceph-mon[35948]: pgmap v336: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:51.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:51 smithi062 ceph-mon[32116]: pgmap v336: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:52.963 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-02T14:54:52.963 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:52.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2727, 2024-04-02T14:54:52.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2399, 2024-04-02T14:54:52.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1087160, 2024-04-02T14:54:52.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 2932858, 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 15, 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.193222991, 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.075965348, 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.124897814, 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.013237259, 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2398, 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.080421329, 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000033536 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4381, 2024-04-02T14:54:52.964 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000700794, 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000159 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1250, 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1916, 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2293223, 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 1085482, 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 11, 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.127176546, 2024-04-02T14:54:52.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.048399276, 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.093556320, 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.007826796, 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1915, 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.055288771, 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000028871 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1813, 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000284019, 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000156 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.966 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2269856, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 965776 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 160, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 342, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 39221, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 682889, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 18, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.025263217, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.010959639, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.008495897, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.002939861, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 338, 2024-04-02T14:54:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.040941280, 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000121128 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 36, 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000008945, 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000248 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2048, 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 544448 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-04-02T14:54:52.968 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 361, 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.258163274, 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000715133 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 440, 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.017723615, 2024-04-02T14:54:52.969 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000040280 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 399, 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 399, 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 399, 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.503776244, 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001262597 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-04-02T14:54:52.970 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 20, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 1, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 42, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 4, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 3, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 35, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 35, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 264, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 3, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 790, 2024-04-02T14:54:52.971 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 505, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 11, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 17, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 212, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 10, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 117, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 117, 2024-04-02T14:54:52.972 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 3720, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 120169321, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 3, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 117, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 2, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 2, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 115, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 83, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 302, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:54:52.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 359, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 281, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 14, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 388 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 11, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 2, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:54:52.974 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 197, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 195, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 17, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 1, 2024-04-02T14:54:52.975 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 999, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 999, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 2, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-04-02T14:54:52.976 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 6011198, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 90, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.462124849, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.005134720 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 35, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 369, 2024-04-02T14:54:52.977 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 334, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 23, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 93, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 70, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 42, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 501, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 459, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 207, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 207, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 55312, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 223608 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-04-02T14:54:52.978 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 681, 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 10, 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 399, 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 16, 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 282, 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 80, 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.087278369, 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001090979 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-04-02T14:54:52.979 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 96, 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.032487875, 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000338415 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.982 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-04-02T14:54:52.982 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.982 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.982 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.982 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.982 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-04-02T14:54:52.982 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.982 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-04-02T14:54:52.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 26, 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.020369899, 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000783457 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-04-02T14:54:52.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-04-02T14:54:52.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.024646908, 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.012323454 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.986 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.987 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-04-02T14:54:52.987 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.987 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.987 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.987 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.987 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-04-02T14:54:52.987 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.987 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.987 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.987 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.987 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.988 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 195, 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.338993193, 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001738426 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.989 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 2, 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 0, 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 2, 2024-04-02T14:54:52.990 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 245, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 122, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 589, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:52.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:52.992 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:52.993 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 298552, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 125, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 352, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 4, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:52.994 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 7936, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 165, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 150114, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 1100, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-04-02T14:54:52.995 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 562, 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 2017857, 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 28, 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 562, 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 562, 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.691962299, 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.004789968 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 562, 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 648 2024-04-02T14:54:52.996 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 562, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 139, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 423, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 22, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 112, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 26, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-04-02T14:54:52.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 1, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 207, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 117, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-04-02T14:54:52.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-04-02T14:54:52.999 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 99, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 11, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 12, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 12, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 104, 2024-04-02T14:54:53.000 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 29 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-04-02T14:54:53.001 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 6, 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 6, 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 195, 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 195, 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 4133, 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 2990238, 2024-04-02T14:54:53.002 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 4133, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 4133, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2990238, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 562, 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2038472, 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 562, 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2038472, 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-04-02T14:54:53.004 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 562, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 562, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 562, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 562, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:53.005 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 195, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 20615, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 195, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 22, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 20615, 2024-04-02T14:54:53.006 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55dca4cb2500": { 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 999, 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1816894, 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 999, 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-02T14:54:53.007 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-02T14:54:53.008 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 90, 2024-04-02T14:54:53.008 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1816894, 2024-04-02T14:54:53.008 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:53.008 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:53.008 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:53.008 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:53.008 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:53.008 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:53.008 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-02T14:54:53.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:53 smithi204 ceph-mon[35948]: pgmap v337: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:53.749 INFO:tasks.check_counter:Daemon mds.i mds.exported=17 2024-04-02T14:54:53.749 INFO:tasks.check_counter:Daemon mds.i mds.imported=10 2024-04-02T14:54:53.749 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=3 2024-04-02T14:54:53.749 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:54:53.749 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:54:53.749 DEBUG:tasks.check_counter:Getting stats from l 2024-04-02T14:54:53.749 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 -- ceph daemon mds.l perf dump 2024-04-02T14:54:53.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:53 smithi062 ceph-mon[32116]: pgmap v337: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:53.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:53 smithi113 ceph-mon[36135]: pgmap v337: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:53.990 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:54:55.608 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:55 smithi204 ceph-mon[35948]: pgmap v338: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:55.735 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 4, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 4, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 3024, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 448, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 0, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.001277565, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.000036281, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.000489702, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000662632, 2024-04-02T14:54:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000220877 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2, 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 175, 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 320, 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 186432, 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-04-02T14:54:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.012343763, 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.008431715, 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.001409771, 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 172, 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.021543077, 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000125250 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 186432 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 241, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 229, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1733600, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 141344, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 2, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.050364103, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.014627861, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.039655625, 2024-04-02T14:54:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.005857465, 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 228, 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.013462765, 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000059047 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1733600, 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 141344 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-04-02T14:54:55.740 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 0, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:54:55.741 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:54:55.742 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 148732, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 31, 2024-04-02T14:54:55.743 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:54:55.744 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-04-02T14:54:55.744 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-04-02T14:54:55.744 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-04-02T14:54:55.744 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:54:55.744 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-04-02T14:54:55.744 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:54:55.745 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-04-02T14:54:55.745 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 0, 2024-04-02T14:54:55.745 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 0, 2024-04-02T14:54:55.745 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:54:55.745 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:54:55.745 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-04-02T14:54:55.745 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-04-02T14:54:55.745 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 928, 2024-04-02T14:54:55.745 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 243, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1699602, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 243, 2024-04-02T14:54:55.746 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-02T14:54:55.747 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-02T14:54:55.747 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 243, 2024-04-02T14:54:55.747 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1699602, 2024-04-02T14:54:55.747 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-02T14:54:55.747 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-02T14:54:55.747 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-02T14:54:55.747 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-02T14:54:55.747 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:55.747 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-02T14:54:55.747 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-02T14:54:55.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:55 smithi062 ceph-mon[32116]: pgmap v338: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:55.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:55 smithi113 ceph-mon[36135]: pgmap v338: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:56.419 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-02T14:54:56.419 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-02T14:54:56.419 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-02T14:54:56.419 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-02T14:54:56.419 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-02T14:54:56.419 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-02T14:54:56.432 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-02T14:54:56.432 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:54:56.432 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T14:54:56.491 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:54:56.491 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T14:54:56.545 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:54:56.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.776+0000 7f02ffa00640 1 -- 172.21.15.62:0/472748114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02f8102450 msgr2=0x7f02f810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:56.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.776+0000 7f02ffa00640 1 --2- 172.21.15.62:0/472748114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02f8102450 0x7f02f810e960 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009ca0 tx=0x7f02f002c100 comp rx=0 tx=0).stop 2024-04-02T14:54:56.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.776+0000 7f02ffa00640 1 -- 172.21.15.62:0/472748114 shutdown_connections 2024-04-02T14:54:56.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.776+0000 7f02ffa00640 1 --2- 172.21.15.62:0/472748114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02f8102450 0x7f02f810e960 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:56.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.776+0000 7f02ffa00640 1 --2- 172.21.15.62:0/472748114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f8101a90 0x7f02f8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:56.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.776+0000 7f02ffa00640 1 --2- 172.21.15.62:0/472748114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8107aa0 0x7f02f8107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:56.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.776+0000 7f02ffa00640 1 -- 172.21.15.62:0/472748114 >> 172.21.15.62:0/472748114 conn(0x7f02f80fd5a0 msgr2=0x7f02f80ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:56.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.776+0000 7f02ffa00640 1 -- 172.21.15.62:0/472748114 shutdown_connections 2024-04-02T14:54:56.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.776+0000 7f02ffa00640 1 -- 172.21.15.62:0/472748114 wait complete. 2024-04-02T14:54:56.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.777+0000 7f02ffa00640 1 Processor -- start 2024-04-02T14:54:56.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.777+0000 7f02ffa00640 1 -- start start 2024-04-02T14:54:56.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.778+0000 7f02ffa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8101a90 0x7f02f819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:56.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.778+0000 7f02ffa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f8102450 0x7f02f819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:56.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.778+0000 7f02ffa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02f8107aa0 0x7f02f819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:56.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.778+0000 7f02ffa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f02f8111fd0 con 0x7f02f8101a90 2024-04-02T14:54:56.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.778+0000 7f02ffa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f02f8111e50 con 0x7f02f8107aa0 2024-04-02T14:54:56.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.778+0000 7f02ffa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02f8112150 con 0x7f02f8102450 2024-04-02T14:54:56.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.778+0000 7f02f7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f8102450 0x7f02f819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:56.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.778+0000 7f02f7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f8102450 0x7f02f819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.62:59822/0 (socket says 172.21.15.62:59822) 2024-04-02T14:54:56.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.778+0000 7f02f7e00640 1 -- 172.21.15.62:0/3044250903 learned_addr learned my addr 172.21.15.62:0/3044250903 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.778+0000 7f02fda00640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02f8107aa0 0x7f02f819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.778+0000 7f02fd000640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8101a90 0x7f02f819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.779+0000 7f02f7e00640 1 -- 172.21.15.62:0/3044250903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02f8107aa0 msgr2=0x7f02f819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.779+0000 7f02f7e00640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02f8107aa0 0x7f02f819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.779+0000 7f02f7e00640 1 -- 172.21.15.62:0/3044250903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8101a90 msgr2=0x7f02f819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.779+0000 7f02f7e00640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8101a90 0x7f02f819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.779+0000 7f02f7e00640 1 -- 172.21.15.62:0/3044250903 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f819f610 con 0x7f02f8102450 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.779+0000 7f02f7e00640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f8102450 0x7f02f819ab60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9f0 tx=0x7f02e400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.779+0000 7f02f5600640 1 -- 172.21.15.62:0/3044250903 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e400cd80 con 0x7f02f8102450 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.779+0000 7f02f5600640 1 -- 172.21.15.62:0/3044250903 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f02e4005520 con 0x7f02f8102450 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.779+0000 7f02f5600640 1 -- 172.21.15.62:0/3044250903 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4010690 con 0x7f02f8102450 2024-04-02T14:54:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.779+0000 7f02ffa00640 1 -- 172.21.15.62:0/3044250903 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f819f900 con 0x7f02f8102450 2024-04-02T14:54:56.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.780+0000 7f02ffa00640 1 -- 172.21.15.62:0/3044250903 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f81a7190 con 0x7f02f8102450 2024-04-02T14:54:56.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.781+0000 7f02ffa00640 1 -- 172.21.15.62:0/3044250903 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f8068d40 con 0x7f02f8102450 2024-04-02T14:54:56.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.783+0000 7f02f5600640 1 -- 172.21.15.62:0/3044250903 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f02e4004f70 con 0x7f02f8102450 2024-04-02T14:54:56.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.784+0000 7f02f5600640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f02e00a3d20 0x7f02e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:56.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.784+0000 7f02f5600640 1 -- 172.21.15.62:0/3044250903 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f02e40d0c60 con 0x7f02f8102450 2024-04-02T14:54:56.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.784+0000 7f02fd000640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f02e00a3d20 0x7f02e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:56.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.785+0000 7f02fd000640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f02e00a3d20 0x7f02e00a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009ae0 tx=0x7f02ec005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:56.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.788+0000 7f02f5600640 1 -- 172.21.15.62:0/3044250903 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f02e409d2c0 con 0x7f02f8102450 2024-04-02T14:54:56.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.996+0000 7f02ffa00640 1 -- 172.21.15.62:0/3044250903 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02f81a7430 con 0x7f02f8102450 2024-04-02T14:54:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:56.998+0000 7f02f5600640 1 -- 172.21.15.62:0/3044250903 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23047 (secure 0 0 0) 0x7f02e409cbc0 con 0x7f02f8102450 2024-04-02T14:54:56.989 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:54:56.989 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":99,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:54:11.768040+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/129789951":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/79979639":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/566820061":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3400451184":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:54:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.000+0000 7f02ffa00640 1 -- 172.21.15.62:0/3044250903 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f02e00a3d20 msgr2=0x7f02e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.000+0000 7f02ffa00640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f02e00a3d20 0x7f02e00a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009ae0 tx=0x7f02ec005f90 comp rx=0 tx=0).stop 2024-04-02T14:54:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.000+0000 7f02ffa00640 1 -- 172.21.15.62:0/3044250903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f8102450 msgr2=0x7f02f819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.000+0000 7f02ffa00640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f8102450 0x7f02f819ab60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9f0 tx=0x7f02e400edf0 comp rx=0 tx=0).stop 2024-04-02T14:54:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.000+0000 7f02ffa00640 1 -- 172.21.15.62:0/3044250903 shutdown_connections 2024-04-02T14:54:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.000+0000 7f02ffa00640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f02e00a3d20 0x7f02e00a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.001+0000 7f02ffa00640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02f8107aa0 0x7f02f819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.001+0000 7f02ffa00640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f8102450 0x7f02f819ab60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.001+0000 7f02ffa00640 1 --2- 172.21.15.62:0/3044250903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8101a90 0x7f02f819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.001+0000 7f02ffa00640 1 -- 172.21.15.62:0/3044250903 >> 172.21.15.62:0/3044250903 conn(0x7f02f80fd5a0 msgr2=0x7f02f80fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.001+0000 7f02ffa00640 1 -- 172.21.15.62:0/3044250903 shutdown_connections 2024-04-02T14:54:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.001+0000 7f02ffa00640 1 -- 172.21.15.62:0/3044250903 wait complete. 2024-04-02T14:54:57.011 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T14:36:11.376053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T14:41:20.030393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T14:41:20.917405+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T14:54:57.011 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:54:57.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.235+0000 7f874b200640 1 -- 172.21.15.62:0/1957424499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f874c107890 msgr2=0x7f874c107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:57.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.235+0000 7f874b200640 1 --2- 172.21.15.62:0/1957424499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f874c107890 0x7f874c107c70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f873c009860 tx=0x7f873c02bb70 comp rx=0 tx=0).stop 2024-04-02T14:54:57.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.235+0000 7f874b200640 1 -- 172.21.15.62:0/1957424499 shutdown_connections 2024-04-02T14:54:57.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.235+0000 7f874b200640 1 --2- 172.21.15.62:0/1957424499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c102240 0x7f874c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.235+0000 7f874b200640 1 --2- 172.21.15.62:0/1957424499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f874c101880 0x7f874c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.235+0000 7f874b200640 1 --2- 172.21.15.62:0/1957424499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f874c107890 0x7f874c107c70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.235+0000 7f874b200640 1 -- 172.21.15.62:0/1957424499 >> 172.21.15.62:0/1957424499 conn(0x7f874c0fd390 msgr2=0x7f874c0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:57.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.235+0000 7f874b200640 1 -- 172.21.15.62:0/1957424499 shutdown_connections 2024-04-02T14:54:57.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.235+0000 7f874b200640 1 -- 172.21.15.62:0/1957424499 wait complete. 2024-04-02T14:54:57.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.236+0000 7f874b200640 1 Processor -- start 2024-04-02T14:54:57.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.237+0000 7f874b200640 1 -- start start 2024-04-02T14:54:57.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.237+0000 7f874b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f874c101880 0x7f874c073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:57.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.237+0000 7f874b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f874c102240 0x7f874c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:57.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.237+0000 7f874b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c107890 0x7f874c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:57.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.237+0000 7f874b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f874c111dc0 con 0x7f874c102240 2024-04-02T14:54:57.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.237+0000 7f874b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f874c111c40 con 0x7f874c101880 2024-04-02T14:54:57.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.237+0000 7f874b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f874c111f40 con 0x7f874c107890 2024-04-02T14:54:57.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.237+0000 7f8749e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f874c101880 0x7f874c073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:57.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.237+0000 7f8749400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f874c102240 0x7f874c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:57.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.237+0000 7f8749e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f874c101880 0x7f874c073e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:54970/0 (socket says 172.21.15.62:54970) 2024-04-02T14:54:57.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.237+0000 7f8749e00640 1 -- 172.21.15.62:0/3566695434 learned_addr learned my addr 172.21.15.62:0/3566695434 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:57.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.238+0000 7f874a800640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c107890 0x7f874c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:57.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.238+0000 7f8749e00640 1 -- 172.21.15.62:0/3566695434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c107890 msgr2=0x7f874c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:57.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.238+0000 7f8749e00640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c107890 0x7f874c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.238+0000 7f8749e00640 1 -- 172.21.15.62:0/3566695434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f874c102240 msgr2=0x7f874c074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:57.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.238+0000 7f8749e00640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f874c102240 0x7f874c074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.238+0000 7f8749e00640 1 -- 172.21.15.62:0/3566695434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874c1a66a0 con 0x7f874c101880 2024-04-02T14:54:57.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.238+0000 7f8749e00640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f874c101880 0x7f874c073e00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f873c005e00 tx=0x7f873c0045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:57.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.238+0000 7f8742a00640 1 -- 172.21.15.62:0/3566695434 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c041020 con 0x7f874c101880 2024-04-02T14:54:57.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.239+0000 7f8742a00640 1 -- 172.21.15.62:0/3566695434 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f873c034940 con 0x7f874c101880 2024-04-02T14:54:57.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.239+0000 7f8742a00640 1 -- 172.21.15.62:0/3566695434 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c039040 con 0x7f874c101880 2024-04-02T14:54:57.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.239+0000 7f874b200640 1 -- 172.21.15.62:0/3566695434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874c1a6930 con 0x7f874c101880 2024-04-02T14:54:57.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.239+0000 7f874b200640 1 -- 172.21.15.62:0/3566695434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f874c1a6dd0 con 0x7f874c101880 2024-04-02T14:54:57.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.240+0000 7f874b200640 1 -- 172.21.15.62:0/3566695434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f874c068810 con 0x7f874c101880 2024-04-02T14:54:57.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.242+0000 7f8742a00640 1 -- 172.21.15.62:0/3566695434 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f873c044020 con 0x7f874c101880 2024-04-02T14:54:57.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.244+0000 7f8742a00640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87300a3da0 0x7f87300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:57.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.244+0000 7f8742a00640 1 -- 172.21.15.62:0/3566695434 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f873c0f22e0 con 0x7f874c101880 2024-04-02T14:54:57.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.244+0000 7f8749400640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87300a3da0 0x7f87300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:57.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.244+0000 7f8749400640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87300a3da0 0x7f87300a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f874c075890 tx=0x7f8734009260 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:57.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.246+0000 7f8742a00640 1 -- 172.21.15.62:0/3566695434 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f873c04b020 con 0x7f874c101880 2024-04-02T14:54:57.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.450+0000 7f874b200640 1 -- 172.21.15.62:0/3566695434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f874c1a7100 con 0x7f874c101880 2024-04-02T14:54:57.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.451+0000 7f8742a00640 1 -- 172.21.15.62:0/3566695434 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f873c0bead0 con 0x7f874c101880 2024-04-02T14:54:57.442 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-02T14:54:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 -- 172.21.15.62:0/3566695434 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87300a3da0 msgr2=0x7f87300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87300a3da0 0x7f87300a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f874c075890 tx=0x7f8734009260 comp rx=0 tx=0).stop 2024-04-02T14:54:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 -- 172.21.15.62:0/3566695434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f874c101880 msgr2=0x7f874c073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f874c101880 0x7f874c073e00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f873c005e00 tx=0x7f873c0045b0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 -- 172.21.15.62:0/3566695434 shutdown_connections 2024-04-02T14:54:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f87300a3da0 0x7f87300a6260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c107890 0x7f874c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f874c102240 0x7f874c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 --2- 172.21.15.62:0/3566695434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f874c101880 0x7f874c073e00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 -- 172.21.15.62:0/3566695434 >> 172.21.15.62:0/3566695434 conn(0x7f874c0fd390 msgr2=0x7f874c0fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:57.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 -- 172.21.15.62:0/3566695434 shutdown_connections 2024-04-02T14:54:57.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.454+0000 7f874b200640 1 -- 172.21.15.62:0/3566695434 wait complete. 2024-04-02T14:54:57.463 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T14:54:57.584 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:57 smithi062 ceph-mon[32116]: pgmap v339: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:57.584 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:57 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3044250903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:54:57.584 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:57 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3566695434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:54:57.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.686+0000 7f65bb200640 1 -- 172.21.15.62:0/1643973375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65bc1010e0 msgr2=0x7f65bc10dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:57.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.686+0000 7f65bb200640 1 --2- 172.21.15.62:0/1643973375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65bc1010e0 0x7f65bc10dda0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009900 tx=0x7f65a402bc50 comp rx=0 tx=0).stop 2024-04-02T14:54:57.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.686+0000 7f65bb200640 1 -- 172.21.15.62:0/1643973375 shutdown_connections 2024-04-02T14:54:57.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.686+0000 7f65bb200640 1 --2- 172.21.15.62:0/1643973375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65bc10e2e0 0x7f65bc1106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.686+0000 7f65bb200640 1 --2- 172.21.15.62:0/1643973375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65bc1010e0 0x7f65bc10dda0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.686+0000 7f65bb200640 1 --2- 172.21.15.62:0/1643973375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc100730 0x7f65bc100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.686+0000 7f65bb200640 1 -- 172.21.15.62:0/1643973375 >> 172.21.15.62:0/1643973375 conn(0x7f65bc0fc350 msgr2=0x7f65bc0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:57.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.686+0000 7f65bb200640 1 -- 172.21.15.62:0/1643973375 shutdown_connections 2024-04-02T14:54:57.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.686+0000 7f65bb200640 1 -- 172.21.15.62:0/1643973375 wait complete. 2024-04-02T14:54:57.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.687+0000 7f65bb200640 1 Processor -- start 2024-04-02T14:54:57.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.687+0000 7f65bb200640 1 -- start start 2024-04-02T14:54:57.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.688+0000 7f65bb200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65bc100730 0x7f65bc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:57.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.688+0000 7f65bb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65bc1010e0 0x7f65bc19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:57.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.688+0000 7f65bb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc10e2e0 0x7f65bc19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:57.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.688+0000 7f65bb200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1140a0 con 0x7f65bc1010e0 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.688+0000 7f65bb200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc113f20 con 0x7f65bc100730 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.688+0000 7f65bb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc114220 con 0x7f65bc10e2e0 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.688+0000 7f65b9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65bc1010e0 0x7f65bc19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.688+0000 7f65ba800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc10e2e0 0x7f65bc19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.688+0000 7f65b9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65bc1010e0 0x7f65bc19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37396/0 (socket says 172.21.15.62:37396) 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.688+0000 7f65b9400640 1 -- 172.21.15.62:0/1877033845 learned_addr learned my addr 172.21.15.62:0/1877033845 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.688+0000 7f65b9e00640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65bc100730 0x7f65bc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.689+0000 7f65b9400640 1 -- 172.21.15.62:0/1877033845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc10e2e0 msgr2=0x7f65bc19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.689+0000 7f65b9400640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc10e2e0 0x7f65bc19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.689+0000 7f65b9400640 1 -- 172.21.15.62:0/1877033845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65bc100730 msgr2=0x7f65bc19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.689+0000 7f65b9400640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65bc100730 0x7f65bc19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.689+0000 7f65b9400640 1 -- 172.21.15.62:0/1877033845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc19f600 con 0x7f65bc1010e0 2024-04-02T14:54:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.689+0000 7f65b9400640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65bc1010e0 0x7f65bc19aba0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f65a4005940 tx=0x7f65a402dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:57.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.689+0000 7f65b2a00640 1 -- 172.21.15.62:0/1877033845 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4043020 con 0x7f65bc1010e0 2024-04-02T14:54:57.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.689+0000 7f65b2a00640 1 -- 172.21.15.62:0/1877033845 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f65a4004250 con 0x7f65bc1010e0 2024-04-02T14:54:57.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.689+0000 7f65b2a00640 1 -- 172.21.15.62:0/1877033845 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4003b30 con 0x7f65bc1010e0 2024-04-02T14:54:57.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.689+0000 7f65bb200640 1 -- 172.21.15.62:0/1877033845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc19f890 con 0x7f65bc1010e0 2024-04-02T14:54:57.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.689+0000 7f65bb200640 1 -- 172.21.15.62:0/1877033845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc1a7180 con 0x7f65bc1010e0 2024-04-02T14:54:57.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.690+0000 7f65bb200640 1 -- 172.21.15.62:0/1877033845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65bc068810 con 0x7f65bc1010e0 2024-04-02T14:54:57.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.692+0000 7f65b2a00640 1 -- 172.21.15.62:0/1877033845 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f65a4046020 con 0x7f65bc1010e0 2024-04-02T14:54:57.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.694+0000 7f65b2a00640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f65a00a3da0 0x7f65a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:57.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.695+0000 7f65b2a00640 1 -- 172.21.15.62:0/1877033845 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f65a40f2790 con 0x7f65bc1010e0 2024-04-02T14:54:57.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.695+0000 7f65b9e00640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f65a00a3da0 0x7f65a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:57.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.695+0000 7f65b9e00640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f65a00a3da0 0x7f65a00a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f65ac005fd0 tx=0x7f65ac009680 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:57.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.700+0000 7f65b2a00640 1 -- 172.21.15.62:0/1877033845 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f65a404d020 con 0x7f65bc1010e0 2024-04-02T14:54:57.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:57 smithi204 ceph-mon[35948]: pgmap v339: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:57.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:57 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3044250903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:54:57.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:57 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3566695434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:54:57.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:57 smithi113 ceph-mon[36135]: pgmap v339: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:54:57.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:57 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3044250903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:54:57.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:57 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3566695434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:54:57.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.907+0000 7f65bb200640 1 -- 172.21.15.62:0/1877033845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f65bc109470 con 0x7f65bc1010e0 2024-04-02T14:54:57.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.908+0000 7f65b2a00640 1 -- 172.21.15.62:0/1877033845 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f65a40bef00 con 0x7f65bc1010e0 2024-04-02T14:54:57.899 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-02T14:54:57.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 -- 172.21.15.62:0/1877033845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f65a00a3da0 msgr2=0x7f65a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:57.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f65a00a3da0 0x7f65a00a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f65ac005fd0 tx=0x7f65ac009680 comp rx=0 tx=0).stop 2024-04-02T14:54:57.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 -- 172.21.15.62:0/1877033845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65bc1010e0 msgr2=0x7f65bc19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:57.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65bc1010e0 0x7f65bc19aba0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f65a4005940 tx=0x7f65a402dd70 comp rx=0 tx=0).stop 2024-04-02T14:54:57.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 -- 172.21.15.62:0/1877033845 shutdown_connections 2024-04-02T14:54:57.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f65a00a3da0 0x7f65a00a6260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc10e2e0 0x7f65bc19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65bc1010e0 0x7f65bc19aba0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 --2- 172.21.15.62:0/1877033845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65bc100730 0x7f65bc19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:57.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 -- 172.21.15.62:0/1877033845 >> 172.21.15.62:0/1877033845 conn(0x7f65bc0fc350 msgr2=0x7f65bc0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:57.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 -- 172.21.15.62:0/1877033845 shutdown_connections 2024-04-02T14:54:57.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:57.911+0000 7f65bb200640 1 -- 172.21.15.62:0/1877033845 wait complete. 2024-04-02T14:54:57.920 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T14:54:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.152+0000 7f3d95c00640 1 -- 172.21.15.62:0/1739043115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d901082b0 msgr2=0x7f3d90108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.152+0000 7f3d95c00640 1 --2- 172.21.15.62:0/1739043115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d901082b0 0x7f3d90108690 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009900 tx=0x7f3d8002bc40 comp rx=0 tx=0).stop 2024-04-02T14:54:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.152+0000 7f3d95c00640 1 -- 172.21.15.62:0/1739043115 shutdown_connections 2024-04-02T14:54:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.152+0000 7f3d95c00640 1 --2- 172.21.15.62:0/1739043115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9010d300 0x7f3d90114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.152+0000 7f3d95c00640 1 --2- 172.21.15.62:0/1739043115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d90108c60 0x7f3d9010cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.152+0000 7f3d95c00640 1 --2- 172.21.15.62:0/1739043115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d901082b0 0x7f3d90108690 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.152+0000 7f3d95c00640 1 -- 172.21.15.62:0/1739043115 >> 172.21.15.62:0/1739043115 conn(0x7f3d90077920 msgr2=0x7f3d90106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.152+0000 7f3d95c00640 1 -- 172.21.15.62:0/1739043115 shutdown_connections 2024-04-02T14:54:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.152+0000 7f3d95c00640 1 -- 172.21.15.62:0/1739043115 wait complete. 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.153+0000 7f3d95c00640 1 Processor -- start 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.153+0000 7f3d95c00640 1 -- start start 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d95c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d901082b0 0x7f3d901a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d95c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9010d300 0x7f3d901a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d95c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d9019e790 0x7f3d9019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d95c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901184f0 con 0x7f3d901082b0 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d95c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90118370 con 0x7f3d9010d300 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d95c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90118670 con 0x7f3d9019e790 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d901082b0 0x7f3d901a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d901082b0 0x7f3d901a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37414/0 (socket says 172.21.15.62:37414) 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d8fe00640 1 -- 172.21.15.62:0/2984642870 learned_addr learned my addr 172.21.15.62:0/2984642870 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d87e00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9010d300 0x7f3d901a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d95200640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d9019e790 0x7f3d9019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d8fe00640 1 -- 172.21.15.62:0/2984642870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d9019e790 msgr2=0x7f3d9019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d8fe00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d9019e790 0x7f3d9019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d8fe00640 1 -- 172.21.15.62:0/2984642870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9010d300 msgr2=0x7f3d901a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d8fe00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9010d300 0x7f3d901a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.154+0000 7f3d8fe00640 1 -- 172.21.15.62:0/2984642870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9019f320 con 0x7f3d901082b0 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.155+0000 7f3d8fe00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d901082b0 0x7f3d901a4620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3d80000c00 tx=0x7f3d8002dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.155+0000 7f3d8d600640 1 -- 172.21.15.62:0/2984642870 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80043020 con 0x7f3d901082b0 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.155+0000 7f3d8d600640 1 -- 172.21.15.62:0/2984642870 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3d80004250 con 0x7f3d901082b0 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.155+0000 7f3d95c00640 1 -- 172.21.15.62:0/2984642870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9019f5b0 con 0x7f3d901082b0 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.155+0000 7f3d8d600640 1 -- 172.21.15.62:0/2984642870 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d800039f0 con 0x7f3d901082b0 2024-04-02T14:54:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.155+0000 7f3d95c00640 1 -- 172.21.15.62:0/2984642870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d901ab790 con 0x7f3d901082b0 2024-04-02T14:54:58.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.156+0000 7f3d95c00640 1 -- 172.21.15.62:0/2984642870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d90109700 con 0x7f3d901082b0 2024-04-02T14:54:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.158+0000 7f3d8d600640 1 -- 172.21.15.62:0/2984642870 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3d80046020 con 0x7f3d901082b0 2024-04-02T14:54:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.160+0000 7f3d8d600640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3d740a3d20 0x7f3d740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.160+0000 7f3d8d600640 1 -- 172.21.15.62:0/2984642870 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f3d800f2640 con 0x7f3d901082b0 2024-04-02T14:54:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.160+0000 7f3d87e00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3d740a3d20 0x7f3d740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.160+0000 7f3d87e00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3d740a3d20 0x7f3d740a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d9010e6a0 tx=0x7f3d78005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:58.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.164+0000 7f3d8d600640 1 -- 172.21.15.62:0/2984642870 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3d8004d020 con 0x7f3d901082b0 2024-04-02T14:54:58.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.369+0000 7f3d95c00640 1 -- 172.21.15.62:0/2984642870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3d90110500 con 0x7f3d901082b0 2024-04-02T14:54:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.370+0000 7f3d8d600640 1 -- 172.21.15.62:0/2984642870 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f3d800bedb0 con 0x7f3d901082b0 2024-04-02T14:54:58.361 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-02T14:54:58.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 -- 172.21.15.62:0/2984642870 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3d740a3d20 msgr2=0x7f3d740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:58.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3d740a3d20 0x7f3d740a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d9010e6a0 tx=0x7f3d78005f90 comp rx=0 tx=0).stop 2024-04-02T14:54:58.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 -- 172.21.15.62:0/2984642870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d901082b0 msgr2=0x7f3d901a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:58.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d901082b0 0x7f3d901a4620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3d80000c00 tx=0x7f3d8002dc20 comp rx=0 tx=0).stop 2024-04-02T14:54:58.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 -- 172.21.15.62:0/2984642870 shutdown_connections 2024-04-02T14:54:58.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f3d740a3d20 0x7f3d740a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d9019e790 0x7f3d9019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9010d300 0x7f3d901a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 --2- 172.21.15.62:0/2984642870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d901082b0 0x7f3d901a4620 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 -- 172.21.15.62:0/2984642870 >> 172.21.15.62:0/2984642870 conn(0x7f3d90077920 msgr2=0x7f3d90077f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:58.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 -- 172.21.15.62:0/2984642870 shutdown_connections 2024-04-02T14:54:58.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.373+0000 7f3d95c00640 1 -- 172.21.15.62:0/2984642870 wait complete. 2024-04-02T14:54:58.383 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T14:54:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.630+0000 7f35df800640 1 -- 172.21.15.62:0/870359241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d8107ac0 msgr2=0x7f35d8107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.630+0000 7f35df800640 1 --2- 172.21.15.62:0/870359241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d8107ac0 0x7f35d8107ea0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009cd0 tx=0x7f35c802c0e0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.630+0000 7f35df800640 1 -- 172.21.15.62:0/870359241 shutdown_connections 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.630+0000 7f35df800640 1 --2- 172.21.15.62:0/870359241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d806a9f0 0x7f35d810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.630+0000 7f35df800640 1 --2- 172.21.15.62:0/870359241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d806a070 0x7f35d806a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.630+0000 7f35df800640 1 --2- 172.21.15.62:0/870359241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d8107ac0 0x7f35d8107ea0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.630+0000 7f35df800640 1 -- 172.21.15.62:0/870359241 >> 172.21.15.62:0/870359241 conn(0x7f35d80779e0 msgr2=0x7f35d8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.630+0000 7f35df800640 1 -- 172.21.15.62:0/870359241 shutdown_connections 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.631+0000 7f35df800640 1 -- 172.21.15.62:0/870359241 wait complete. 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.631+0000 7f35df800640 1 Processor -- start 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35df800640 1 -- start start 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35df800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d806a070 0x7f35d81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35df800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d806a9f0 0x7f35d8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35df800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d8107ac0 0x7f35d819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35df800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8111fe0 con 0x7f35d806a9f0 2024-04-02T14:54:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35df800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8111e60 con 0x7f35d806a070 2024-04-02T14:54:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35df800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8112160 con 0x7f35d8107ac0 2024-04-02T14:54:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d806a9f0 0x7f35d8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35dce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d806a070 0x7f35d81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d806a9f0 0x7f35d8198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37432/0 (socket says 172.21.15.62:37432) 2024-04-02T14:54:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35d7e00640 1 -- 172.21.15.62:0/306564165 learned_addr learned my addr 172.21.15.62:0/306564165 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:54:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.632+0000 7f35dd800640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d8107ac0 0x7f35d819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.633+0000 7f35dce00640 1 -- 172.21.15.62:0/306564165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d8107ac0 msgr2=0x7f35d819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.633+0000 7f35dce00640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d8107ac0 0x7f35d819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.633+0000 7f35dce00640 1 -- 172.21.15.62:0/306564165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d806a9f0 msgr2=0x7f35d8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.633+0000 7f35dce00640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d806a9f0 0x7f35d8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.633+0000 7f35dce00640 1 -- 172.21.15.62:0/306564165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d819d4d0 con 0x7f35d806a070 2024-04-02T14:54:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.633+0000 7f35dce00640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d806a070 0x7f35d81984e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009bf0 tx=0x7f35c8034440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.633+0000 7f35d5600640 1 -- 172.21.15.62:0/306564165 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8039040 con 0x7f35d806a070 2024-04-02T14:54:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.633+0000 7f35d5600640 1 -- 172.21.15.62:0/306564165 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f35c8044070 con 0x7f35d806a070 2024-04-02T14:54:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.633+0000 7f35df800640 1 -- 172.21.15.62:0/306564165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d819d760 con 0x7f35d806a070 2024-04-02T14:54:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.633+0000 7f35d5600640 1 -- 172.21.15.62:0/306564165 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c80042a0 con 0x7f35d806a070 2024-04-02T14:54:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.634+0000 7f35df800640 1 -- 172.21.15.62:0/306564165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d81a5050 con 0x7f35d806a070 2024-04-02T14:54:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.635+0000 7f35df800640 1 -- 172.21.15.62:0/306564165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d8069920 con 0x7f35d806a070 2024-04-02T14:54:58.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.637+0000 7f35d5600640 1 -- 172.21.15.62:0/306564165 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f35c8003970 con 0x7f35d806a070 2024-04-02T14:54:58.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.638+0000 7f35d5600640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f35bc0a3d20 0x7f35bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:54:58.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.638+0000 7f35d5600640 1 -- 172.21.15.62:0/306564165 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f35c80f3120 con 0x7f35d806a070 2024-04-02T14:54:58.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.638+0000 7f35d7e00640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f35bc0a3d20 0x7f35bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:54:58.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.639+0000 7f35d7e00640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f35bc0a3d20 0x7f35bc0a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f35d8100890 tx=0x7f35c0005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:54:58.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.641+0000 7f35d5600640 1 -- 172.21.15.62:0/306564165 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f35c80bf890 con 0x7f35d806a070 2024-04-02T14:54:58.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:58 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1877033845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:54:58.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:58 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2984642870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:54:58.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:58 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1877033845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:54:58.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:58 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2984642870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:54:58.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.859+0000 7f35df800640 1 -- 172.21.15.62:0/306564165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f35d8068d40 con 0x7f35d806a070 2024-04-02T14:54:58.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.860+0000 7f35d5600640 1 -- 172.21.15.62:0/306564165 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v99) ==== 81+0+942 (secure 0 0 0) 0x7f35c80bf190 con 0x7f35d806a070 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/129789951 2024-04-03T14:44:00.087305+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/1995176952 2024-04-03T14:44:00.087305+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/79979639 2024-04-03T14:44:00.087305+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/1356430694 2024-04-03T14:33:19.211822+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/1995176952 2024-04-03T14:44:00.087305+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/1356430694 2024-04-03T14:33:19.211822+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2679742291 2024-04-03T14:33:19.211822+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3147101239 2024-04-03T14:32:43.373728+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3831753172 2024-04-03T14:33:19.211822+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2106382100 2024-04-03T14:32:43.373728+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/3623171244 2024-04-03T14:32:43.373728+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/3623171244 2024-04-03T14:32:43.373728+0000 2024-04-02T14:54:58.851 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/566820061 2024-04-03T14:44:00.087305+0000 2024-04-02T14:54:58.852 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3400451184 2024-04-03T14:44:00.087305+0000 2024-04-02T14:54:58.852 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3824719077 2024-04-03T14:32:43.373728+0000 2024-04-02T14:54:58.852 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2038939412 2024-04-03T14:33:19.211822+0000 2024-04-02T14:54:58.852 INFO:teuthology.orchestra.run.smithi062.stderr:listed 16 entries 2024-04-02T14:54:58.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.862+0000 7f35df800640 1 -- 172.21.15.62:0/306564165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f35bc0a3d20 msgr2=0x7f35bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:58.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.862+0000 7f35df800640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f35bc0a3d20 0x7f35bc0a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f35d8100890 tx=0x7f35c0005f90 comp rx=0 tx=0).stop 2024-04-02T14:54:58.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.862+0000 7f35df800640 1 -- 172.21.15.62:0/306564165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d806a070 msgr2=0x7f35d81984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:54:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.862+0000 7f35df800640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d806a070 0x7f35d81984e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009bf0 tx=0x7f35c8034440 comp rx=0 tx=0).stop 2024-04-02T14:54:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.862+0000 7f35df800640 1 -- 172.21.15.62:0/306564165 shutdown_connections 2024-04-02T14:54:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.862+0000 7f35df800640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f35bc0a3d20 0x7f35bc0a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.862+0000 7f35df800640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d8107ac0 0x7f35d819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.862+0000 7f35df800640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d806a9f0 0x7f35d8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.862+0000 7f35df800640 1 --2- 172.21.15.62:0/306564165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d806a070 0x7f35d81984e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:54:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.862+0000 7f35df800640 1 -- 172.21.15.62:0/306564165 >> 172.21.15.62:0/306564165 conn(0x7f35d80779e0 msgr2=0x7f35d80fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:54:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.863+0000 7f35df800640 1 -- 172.21.15.62:0/306564165 shutdown_connections 2024-04-02T14:54:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:54:58.863+0000 7f35df800640 1 -- 172.21.15.62:0/306564165 wait complete. 2024-04-02T14:54:58.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:58 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1877033845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:54:58.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:58 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2984642870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:54:58.874 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-02T14:54:58.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:54:58.874 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-02T14:54:59.080 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-02T14:54:59.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:54:59.081 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-02T14:54:59.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:54:59.172 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-02T14:54:59.255 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-02T14:54:59.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:54:59.255 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T14:54:59.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:54:59.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-04-02T14:54:59.518 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-02T14:54:59.518 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-02T14:54:59.518 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T14:54:59.557 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053434,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/c052c3c06f6d2c82c52d3bd9c514a70bf15ea0decff39484d58586c48ec52abc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/GK3MTVEZDF4TFIDJSY3MGQIANK:/var/lib/containers/storage/overlay/l/6AKK2TYZDRYOVN7INVOGCDHXKH,upperdir=/var/lib/containers/storage/overlay/c052c3c06f6d2c82c52d3bd9c514a70bf15ea0decff39484d58586c48ec52abc/diff,workdir=/var/lib/containers/storage/overlay/c052c3c06f6d2c82c52d3bd9c514a70bf15ea0decff39484d58586c48ec52abc/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e8936dbf200836879e6187803c6ca1bb91966aacbd8435fc30f805d2c0ef76be/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/GK3MTVEZDF4TFIDJSY3MGQIANK:/var/lib/containers/storage/overlay/l/6AKK2TYZDRYOVN7INVOGCDHXKH,upperdir=/var/lib/containers/storage/overlay/e8936dbf200836879e6187803c6ca1bb91966aacbd8435fc30f805d2c0ef76be/diff,workdir=/var/lib/containers/storage/overlay/e8936dbf200836879e6187803c6ca1bb91966aacbd8435fc30f805d2c0ef76be/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/d556ca80c48be3cfe3b9541157eb936bae48b256018659c98450b99017ec293d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/GK3MTVEZDF4TFIDJSY3MGQIANK:/var/lib/containers/storage/overlay/l/6AKK2TYZDRYOVN7INVOGCDHXKH,upperdir=/var/lib/containers/storage/overlay/d556ca80c48be3cfe3b9541157eb936bae48b256018659c98450b99017ec293d/diff,workdir=/var/lib/containers/storage/overlay/d556ca80c48be3cfe3b9541157eb936bae48b256018659c98450b99017ec293d/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/812b795bbba669faf9de21233bb00733fceeb954a1b81ebcd3f5a09a1bb6bf10/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/GK3MTVEZDF4TFIDJSY3MGQIANK:/var/lib/containers/storage/overlay/l/6AKK2TYZDRYOVN7INVOGCDHXKH,upperdir=/var/lib/containers/storage/overlay/812b795bbba669faf9de21233bb00733fceeb954a1b81ebcd3f5a09a1bb6bf10/diff,workdir=/var/lib/containers/storage/overlay/812b795bbba669faf9de21233bb00733fceeb954a1b81ebcd3f5a09a1bb6bf10/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/70a0ca0f8a0d03591d3960929a063b80e240eb873a38bbe008ff9340d091b6ce/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/GK3MTVEZDF4TFIDJSY3MGQIANK:/var/lib/containers/storage/overlay/l/6AKK2TYZDRYOVN7INVOGCDHXKH,upperdir=/var/lib/containers/storage/overlay/70a0ca0f8a0d03591d3960929a063b80e240eb873a38bbe008ff9340d091b6ce/diff,workdir=/var/lib/containers/storage/overlay/70a0ca0f8a0d03591d3960929a063b80e240eb873a38bbe008ff9340d091b6ce/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e61ccf6cced005cc2c3e11dd2d20e25858955cba9dabeba97a85875a91685289/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/GK3MTVEZDF4TFIDJSY3MGQIANK:/var/lib/containers/storage/overlay/l/6AKK2TYZDRYOVN7INVOGCDHXKH,upperdir=/var/lib/containers/storage/overlay/e61ccf6cced005cc2c3e11dd2d20e25858955cba9dabeba97a85875a91685289/diff,workdir=/var/lib/containers/storage/overlay/e61ccf6cced005cc2c3e11dd2d20e25858955cba9dabeba97a85875a91685289/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/dd5411f801a3018991c7e49b52b83d0d34da7f48b6e37249728b306d9d96bf93/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/GK3MTVEZDF4TFIDJSY3MGQIANK:/var/lib/containers/storage/overlay/l/6AKK2TYZDRYOVN7INVOGCDHXKH,upperdir=/var/lib/containers/storage/overlay/dd5411f801a3018991c7e49b52b83d0d34da7f48b6e37249728b306d9d96bf93/diff,workdir=/var/lib/containers/storage/overlay/dd5411f801a3018991c7e49b52b83d0d34da7f48b6e37249728b306d9d96bf93/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ee1afd8e5495291c9bea19eb21343947f5f25c1818f410d4c0e2ae454161a701/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/GK3MTVEZDF4TFIDJSY3MGQIANK:/var/lib/containers/storage/overlay/l/6AKK2TYZDRYOVN7INVOGCDHXKH,upperdir=/var/lib/containers/storage/overlay/ee1afd8e5495291c9bea19eb21343947f5f25c1818f410d4c0e2ae454161a701/diff,workdir=/var/lib/containers/storage/overlay/ee1afd8e5495291c9bea19eb21343947f5f25c1818f410d4c0e2ae454161a701/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/89abce7330476a62e72f25ad742c668baa1e5e839ddec0b695544f99e02ca5d1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/GK3MTVEZDF4TFIDJSY3MGQIANK:/var/lib/containers/storage/overlay/l/6AKK2TYZDRYOVN7INVOGCDHXKH,upperdir=/var/lib/containers/storage/overlay/89abce7330476a62e72f25ad742c668baa1e5e839ddec0b695544f99e02ca5d1/diff,workdir=/var/lib/containers/storage/overlay/89abce7330476a62e72f25ad742c668baa1e5e839ddec0b695544f99e02ca5d1/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/74f8fe7df3f7708cfa28faeb18c3b6dd9b204853bb66f6defb5a7168b6ce29bc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/GK3MTVEZDF4TFIDJSY3MGQIANK:/var/lib/containers/storage/overlay/l/6AKK2TYZDRYOVN7INVOGCDHXKH,upperdir=/var/lib/containers/storage/overlay/74f8fe7df3f7708cfa28faeb18c3b6dd9b204853bb66f6defb5a7168b6ce29bc/diff,workdir=/var/lib/containers/storage/overlay/74f8fe7df3f7708cfa28faeb18c3b6dd9b204853bb66f6defb5a7168b6ce29bc/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 2024-04-02T14:54:59.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:54:59.558 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-02T14:54:59.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:54:59.641 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-04-02T14:54:59.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:59 smithi204 ceph-mon[35948]: pgmap v340: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:59.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:54:59 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/306564165' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T14:54:59.762 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:59 smithi062 ceph-mon[32116]: pgmap v340: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:59.762 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:54:59 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/306564165' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T14:54:59.781 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-04-02T14:54:59.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:54:59.787 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-02T14:54:59.799 INFO:tasks.cephadm:Teardown begin 2024-04-02T14:54:59.800 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-02T14:54:59.800 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:54:59.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:59 smithi113 ceph-mon[36135]: pgmap v340: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:54:59.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:54:59 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/306564165' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T14:55:00.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.040+0000 7fe085e00640 1 -- 172.21.15.62:0/1152172338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0800747d0 msgr2=0x7fe08010dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:00.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.040+0000 7fe085e00640 1 --2- 172.21.15.62:0/1152172338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0800747d0 0x7fe08010dc40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009900 tx=0x7fe06c02bcc0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.040+0000 7fe085e00640 1 -- 172.21.15.62:0/1152172338 shutdown_connections 2024-04-02T14:55:00.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.040+0000 7fe085e00640 1 --2- 172.21.15.62:0/1152172338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe08010e180 0x7fe080110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.040+0000 7fe085e00640 1 --2- 172.21.15.62:0/1152172338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0800747d0 0x7fe08010dc40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.040+0000 7fe085e00640 1 --2- 172.21.15.62:0/1152172338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe080073eb0 0x7fe080074290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.040+0000 7fe085e00640 1 -- 172.21.15.62:0/1152172338 >> 172.21.15.62:0/1152172338 conn(0x7fe0800fd3d0 msgr2=0x7fe0800ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:00.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.040+0000 7fe085e00640 1 -- 172.21.15.62:0/1152172338 shutdown_connections 2024-04-02T14:55:00.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.040+0000 7fe085e00640 1 -- 172.21.15.62:0/1152172338 wait complete. 2024-04-02T14:55:00.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.041+0000 7fe085e00640 1 Processor -- start 2024-04-02T14:55:00.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.041+0000 7fe085e00640 1 -- start start 2024-04-02T14:55:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe085e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe080073eb0 0x7fe08019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe085e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0800747d0 0x7fe08019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe085e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe08010e180 0x7fe08019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe085e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe080113fa0 con 0x7fe0800747d0 2024-04-02T14:55:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe085e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe080113e20 con 0x7fe080073eb0 2024-04-02T14:55:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe085e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe080114120 con 0x7fe08010e180 2024-04-02T14:55:00.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe07fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0800747d0 0x7fe08019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe07fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0800747d0 0x7fe08019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37446/0 (socket says 172.21.15.62:37446) 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe07fe00640 1 -- 172.21.15.62:0/2410854273 learned_addr learned my addr 172.21.15.62:0/2410854273 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe084a00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe080073eb0 0x7fe08019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe085400640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe08010e180 0x7fe08019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe07fe00640 1 -- 172.21.15.62:0/2410854273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe08010e180 msgr2=0x7fe08019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe07fe00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe08010e180 0x7fe08019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe07fe00640 1 -- 172.21.15.62:0/2410854273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe080073eb0 msgr2=0x7fe08019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe07fe00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe080073eb0 0x7fe08019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe07fe00640 1 -- 172.21.15.62:0/2410854273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08019f4c0 con 0x7fe0800747d0 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.042+0000 7fe084a00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe080073eb0 0x7fe08019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.043+0000 7fe07fe00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0800747d0 0x7fe08019aa60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe06c005ec0 tx=0x7fe06c0049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.043+0000 7fe07d600640 1 -- 172.21.15.62:0/2410854273 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c043020 con 0x7fe0800747d0 2024-04-02T14:55:00.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.043+0000 7fe07d600640 1 -- 172.21.15.62:0/2410854273 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe06c004250 con 0x7fe0800747d0 2024-04-02T14:55:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.043+0000 7fe085e00640 1 -- 172.21.15.62:0/2410854273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08019f750 con 0x7fe0800747d0 2024-04-02T14:55:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.043+0000 7fe07d600640 1 -- 172.21.15.62:0/2410854273 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c003b50 con 0x7fe0800747d0 2024-04-02T14:55:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.043+0000 7fe085e00640 1 -- 172.21.15.62:0/2410854273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0801a7040 con 0x7fe0800747d0 2024-04-02T14:55:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.044+0000 7fe085e00640 1 -- 172.21.15.62:0/2410854273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe080068c60 con 0x7fe0800747d0 2024-04-02T14:55:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.046+0000 7fe07d600640 1 -- 172.21.15.62:0/2410854273 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe06c046020 con 0x7fe0800747d0 2024-04-02T14:55:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.048+0000 7fe07d600640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0680a3da0 0x7fe0680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:00.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.048+0000 7fe07d600640 1 -- 172.21.15.62:0/2410854273 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fe06c0f2760 con 0x7fe0800747d0 2024-04-02T14:55:00.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.048+0000 7fe084a00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0680a3da0 0x7fe0680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:00.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.049+0000 7fe084a00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0680a3da0 0x7fe0680a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe0740095f0 tx=0x7fe074005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:00.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.052+0000 7fe07d600640 1 -- 172.21.15.62:0/2410854273 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe06c04d020 con 0x7fe0800747d0 2024-04-02T14:55:00.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.307+0000 7fe085e00640 1 -- 172.21.15.62:0/2410854273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe08019b820 con 0x7fe0800747d0 2024-04-02T14:55:00.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.308+0000 7fe07d600640 1 -- 172.21.15.62:0/2410854273 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11938 (secure 0 0 0) 0x7fe06c0beed0 con 0x7fe0800747d0 2024-04-02T14:55:00.299 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:55:00.299 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:49:56.750365+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":93,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":91,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":91,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24565,24553,24533,24584,24535]},"id":1}]} 2024-04-02T14:55:00.300 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 49 2024-04-02T14:55:00.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 -- 172.21.15.62:0/2410854273 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0680a3da0 msgr2=0x7fe0680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:00.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0680a3da0 0x7fe0680a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe0740095f0 tx=0x7fe074005f90 comp rx=0 tx=0).stop 2024-04-02T14:55:00.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 -- 172.21.15.62:0/2410854273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0800747d0 msgr2=0x7fe08019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:00.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0800747d0 0x7fe08019aa60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe06c005ec0 tx=0x7fe06c0049e0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 -- 172.21.15.62:0/2410854273 shutdown_connections 2024-04-02T14:55:00.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fe0680a3da0 0x7fe0680a6260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe08010e180 0x7fe08019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0800747d0 0x7fe08019aa60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 --2- 172.21.15.62:0/2410854273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe080073eb0 0x7fe08019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 -- 172.21.15.62:0/2410854273 >> 172.21.15.62:0/2410854273 conn(0x7fe0800fd3d0 msgr2=0x7fe0800fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:00.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 -- 172.21.15.62:0/2410854273 shutdown_connections 2024-04-02T14:55:00.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.311+0000 7fe085e00640 1 -- 172.21.15.62:0/2410854273 wait complete. 2024-04-02T14:55:00.324 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:55:00.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.535+0000 7fce96400640 1 -- 172.21.15.62:0/362048570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce90107aa0 msgr2=0x7fce90107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:00.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.535+0000 7fce96400640 1 --2- 172.21.15.62:0/362048570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce90107aa0 0x7fce90107e80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fce80009930 tx=0x7fce8002bc20 comp rx=0 tx=0).stop 2024-04-02T14:55:00.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.535+0000 7fce96400640 1 -- 172.21.15.62:0/362048570 shutdown_connections 2024-04-02T14:55:00.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.535+0000 7fce96400640 1 --2- 172.21.15.62:0/362048570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce90102450 0x7fce9010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.535+0000 7fce96400640 1 --2- 172.21.15.62:0/362048570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90101a90 0x7fce90101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.535+0000 7fce96400640 1 --2- 172.21.15.62:0/362048570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce90107aa0 0x7fce90107e80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.535+0000 7fce96400640 1 -- 172.21.15.62:0/362048570 >> 172.21.15.62:0/362048570 conn(0x7fce900fd5a0 msgr2=0x7fce900ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:00.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.535+0000 7fce96400640 1 -- 172.21.15.62:0/362048570 shutdown_connections 2024-04-02T14:55:00.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.535+0000 7fce96400640 1 -- 172.21.15.62:0/362048570 wait complete. 2024-04-02T14:55:00.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.536+0000 7fce96400640 1 Processor -- start 2024-04-02T14:55:00.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.536+0000 7fce96400640 1 -- start start 2024-04-02T14:55:00.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.537+0000 7fce96400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce90101a90 0x7fce9019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:00.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.537+0000 7fce96400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90102450 0x7fce9019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:00.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.537+0000 7fce96400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce90107aa0 0x7fce9019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.537+0000 7fce96400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fce90111fd0 con 0x7fce90101a90 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.537+0000 7fce96400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fce90111e50 con 0x7fce90107aa0 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.537+0000 7fce96400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce90112150 con 0x7fce90102450 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.537+0000 7fce8f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce90101a90 0x7fce9019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.537+0000 7fce8f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce90101a90 0x7fce9019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37474/0 (socket says 172.21.15.62:37474) 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.537+0000 7fce8f400640 1 -- 172.21.15.62:0/4112772924 learned_addr learned my addr 172.21.15.62:0/4112772924 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.537+0000 7fce8fe00640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce90107aa0 0x7fce9019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.537+0000 7fce8ea00640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90102450 0x7fce9019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.538+0000 7fce8f400640 1 -- 172.21.15.62:0/4112772924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90102450 msgr2=0x7fce9019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.538+0000 7fce8f400640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90102450 0x7fce9019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.538+0000 7fce8f400640 1 -- 172.21.15.62:0/4112772924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce90107aa0 msgr2=0x7fce9019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.538+0000 7fce8f400640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce90107aa0 0x7fce9019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.538+0000 7fce8f400640 1 -- 172.21.15.62:0/4112772924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9019f6c0 con 0x7fce90101a90 2024-04-02T14:55:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.538+0000 7fce8fe00640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce90107aa0 0x7fce9019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:55:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.538+0000 7fce8f400640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce90101a90 0x7fce9019a6d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fce80009900 tx=0x7fce80004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.538+0000 7fce87e00640 1 -- 172.21.15.62:0/4112772924 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80043020 con 0x7fce90101a90 2024-04-02T14:55:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.538+0000 7fce87e00640 1 -- 172.21.15.62:0/4112772924 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fce80004e50 con 0x7fce90101a90 2024-04-02T14:55:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.538+0000 7fce96400640 1 -- 172.21.15.62:0/4112772924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9019f950 con 0x7fce90101a90 2024-04-02T14:55:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.538+0000 7fce87e00640 1 -- 172.21.15.62:0/4112772924 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80003950 con 0x7fce90101a90 2024-04-02T14:55:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.539+0000 7fce96400640 1 -- 172.21.15.62:0/4112772924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce901a7240 con 0x7fce90101a90 2024-04-02T14:55:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.540+0000 7fce96400640 1 -- 172.21.15.62:0/4112772924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce90068d40 con 0x7fce90101a90 2024-04-02T14:55:00.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.542+0000 7fce87e00640 1 -- 172.21.15.62:0/4112772924 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fce80046020 con 0x7fce90101a90 2024-04-02T14:55:00.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.543+0000 7fce87e00640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fce740a3da0 0x7fce740a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:00.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.543+0000 7fce87e00640 1 -- 172.21.15.62:0/4112772924 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fce800f2540 con 0x7fce90101a90 2024-04-02T14:55:00.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.544+0000 7fce8ea00640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fce740a3da0 0x7fce740a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:00.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.544+0000 7fce8ea00640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fce740a3da0 0x7fce740a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce90100fd0 tx=0x7fce7c005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:00.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.547+0000 7fce87e00640 1 -- 172.21.15.62:0/4112772924 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fce8004d020 con 0x7fce90101a90 2024-04-02T14:55:00.584 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:00 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/2410854273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:55:00.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:00 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/2410854273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:55:00.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.795+0000 7fce96400640 1 -- 172.21.15.62:0/4112772924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce901a74d0 con 0x7fce90101a90 2024-04-02T14:55:00.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.796+0000 7fce87e00640 1 -- 172.21.15.62:0/4112772924 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11938 (secure 0 0 0) 0x7fce800bedd0 con 0x7fce90101a90 2024-04-02T14:55:00.787 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:55:00.788 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1178681921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1178681921},{"type":"v1","addr":"172.21.15.62:6835","nonce":1178681921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3512482618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3512482618},{"type":"v1","addr":"172.21.15.204:6833","nonce":3512482618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T14:41:22.009309+0000","modified":"2024-04-02T14:49:56.750365+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24553,"mds_3":24535,"mds_4":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/1862883972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1862883972},{"type":"v1","addr":"172.21.15.113:6833","nonce":1862883972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3805165269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3805165269},{"type":"v1","addr":"172.21.15.62:6837","nonce":3805165269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3756932584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3756932584},{"type":"v1","addr":"172.21.15.204:6835","nonce":3756932584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/360641878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":360641878},{"type":"v1","addr":"172.21.15.113:6835","nonce":360641878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3537379638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3537379638},{"type":"v1","addr":"172.21.15.204:6837","nonce":3537379638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":93,"addr":"172.21.15.113:6837/1122486361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1122486361},{"type":"v1","addr":"172.21.15.113:6837","nonce":1122486361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.204:6839/631749651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":631749651},{"type":"v1","addr":"172.21.15.204:6839","nonce":631749651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":91,"addr":"172.21.15.62:6839/3188264140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3188264140},{"type":"v1","addr":"172.21.15.62:6839","nonce":3188264140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":91,"addr":"172.21.15.62:6841/450176372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":450176372},{"type":"v1","addr":"172.21.15.62:6841","nonce":450176372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6839/1791174899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1791174899},{"type":"v1","addr":"172.21.15.113:6839","nonce":1791174899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24565,24553,24533,24584,24535]},"id":1}]} 2024-04-02T14:55:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 49 2024-04-02T14:55:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.799+0000 7fce96400640 1 -- 172.21.15.62:0/4112772924 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fce740a3da0 msgr2=0x7fce740a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.799+0000 7fce96400640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fce740a3da0 0x7fce740a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce90100fd0 tx=0x7fce7c005f90 comp rx=0 tx=0).stop 2024-04-02T14:55:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.799+0000 7fce96400640 1 -- 172.21.15.62:0/4112772924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce90101a90 msgr2=0x7fce9019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.799+0000 7fce96400640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce90101a90 0x7fce9019a6d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fce80009900 tx=0x7fce80004c30 comp rx=0 tx=0).stop 2024-04-02T14:55:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.799+0000 7fce96400640 1 -- 172.21.15.62:0/4112772924 shutdown_connections 2024-04-02T14:55:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.799+0000 7fce96400640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fce740a3da0 0x7fce740a6260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.799+0000 7fce96400640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce90107aa0 0x7fce9019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.799+0000 7fce96400640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90102450 0x7fce9019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.799+0000 7fce96400640 1 --2- 172.21.15.62:0/4112772924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce90101a90 0x7fce9019a6d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.799+0000 7fce96400640 1 -- 172.21.15.62:0/4112772924 >> 172.21.15.62:0/4112772924 conn(0x7fce900fd5a0 msgr2=0x7fce9010d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.800+0000 7fce96400640 1 -- 172.21.15.62:0/4112772924 shutdown_connections 2024-04-02T14:55:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:00.800+0000 7fce96400640 1 -- 172.21.15.62:0/4112772924 wait complete. 2024-04-02T14:55:00.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:55:00.825 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:00 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/2410854273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:55:01.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.041+0000 7f09e7000640 1 -- 172.21.15.62:0/595341329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e0109b70 msgr2=0x7f09e0109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.041+0000 7f09e7000640 1 --2- 172.21.15.62:0/595341329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e0109b70 0x7f09e0109f50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009900 tx=0x7f09d402bc30 comp rx=0 tx=0).stop 2024-04-02T14:55:01.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.041+0000 7f09e7000640 1 -- 172.21.15.62:0/595341329 shutdown_connections 2024-04-02T14:55:01.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.041+0000 7f09e7000640 1 --2- 172.21.15.62:0/595341329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e0104490 0x7f09e01109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.041+0000 7f09e7000640 1 --2- 172.21.15.62:0/595341329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0103ad0 0x7f09e0103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.041+0000 7f09e7000640 1 --2- 172.21.15.62:0/595341329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e0109b70 0x7f09e0109f50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.041+0000 7f09e7000640 1 -- 172.21.15.62:0/595341329 >> 172.21.15.62:0/595341329 conn(0x7f09e00ff5a0 msgr2=0x7f09e01019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:01.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.041+0000 7f09e7000640 1 -- 172.21.15.62:0/595341329 shutdown_connections 2024-04-02T14:55:01.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.042+0000 7f09e7000640 1 -- 172.21.15.62:0/595341329 wait complete. 2024-04-02T14:55:01.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.042+0000 7f09e7000640 1 Processor -- start 2024-04-02T14:55:01.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e7000640 1 -- start start 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e7000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e0103ad0 0x7f09e019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e7000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e0104490 0x7f09e019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e7000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0109b70 0x7f09e019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e7000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0114010 con 0x7f09e0109b70 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e7000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0113e90 con 0x7f09e0103ad0 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e7000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0114190 con 0x7f09e0104490 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e5c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e0103ad0 0x7f09e019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e6600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0109b70 0x7f09e019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e5c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e0103ad0 0x7f09e019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55048/0 (socket says 172.21.15.62:55048) 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e5c00640 1 -- 172.21.15.62:0/1915974369 learned_addr learned my addr 172.21.15.62:0/1915974369 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.043+0000 7f09e5200640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e0104490 0x7f09e019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.044+0000 7f09e5c00640 1 -- 172.21.15.62:0/1915974369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e0104490 msgr2=0x7f09e019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.044+0000 7f09e5c00640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e0104490 0x7f09e019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.044+0000 7f09e5c00640 1 -- 172.21.15.62:0/1915974369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0109b70 msgr2=0x7f09e019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.044+0000 7f09e5c00640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0109b70 0x7f09e019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.044+0000 7f09e5c00640 1 -- 172.21.15.62:0/1915974369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e019f480 con 0x7f09e0103ad0 2024-04-02T14:55:01.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.044+0000 7f09e6600640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0109b70 0x7f09e019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:55:01.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.044+0000 7f09e5c00640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e0103ad0 0x7f09e019a490 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f09d4005ab0 tx=0x7f09d402dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:01.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.044+0000 7f09de000640 1 -- 172.21.15.62:0/1915974369 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4043020 con 0x7f09e0103ad0 2024-04-02T14:55:01.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.044+0000 7f09de000640 1 -- 172.21.15.62:0/1915974369 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f09d4004e50 con 0x7f09e0103ad0 2024-04-02T14:55:01.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.045+0000 7f09de000640 1 -- 172.21.15.62:0/1915974369 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4003970 con 0x7f09e0103ad0 2024-04-02T14:55:01.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.045+0000 7f09e7000640 1 -- 172.21.15.62:0/1915974369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e019f710 con 0x7f09e0103ad0 2024-04-02T14:55:01.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.045+0000 7f09e7000640 1 -- 172.21.15.62:0/1915974369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e01a7000 con 0x7f09e0103ad0 2024-04-02T14:55:01.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.046+0000 7f09e7000640 1 -- 172.21.15.62:0/1915974369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e0068c60 con 0x7f09e0103ad0 2024-04-02T14:55:01.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.048+0000 7f09de000640 1 -- 172.21.15.62:0/1915974369 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f09d4046020 con 0x7f09e0103ad0 2024-04-02T14:55:01.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.049+0000 7f09de000640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09c80a3da0 0x7f09c80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.050+0000 7f09de000640 1 -- 172.21.15.62:0/1915974369 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f09d40f2cc0 con 0x7f09e0103ad0 2024-04-02T14:55:01.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.050+0000 7f09e5200640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09c80a3da0 0x7f09c80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.050+0000 7f09e5200640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09c80a3da0 0x7f09c80a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0095f0 tx=0x7f09cc005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:01.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.052+0000 7f09de000640 1 -- 172.21.15.62:0/1915974369 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f09d404d020 con 0x7f09e0103ad0 2024-04-02T14:55:01.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.255+0000 7f09e7000640 1 -- 172.21.15.62:0/1915974369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09e010c040 con 0x7f09e0103ad0 2024-04-02T14:55:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.257+0000 7f09de000640 1 -- 172.21.15.62:0/1915974369 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23047 (secure 0 0 0) 0x7f09d40bf430 con 0x7f09e0103ad0 2024-04-02T14:55:01.248 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-02T14:55:01.248 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":99,"fsid":"9e86b65e-f0fd-11ee-b647-cb9ed24678a4","created":"2024-04-02T14:31:58.064401+0000","modified":"2024-04-02T14:54:11.768040+0000","last_up_change":"2024-04-02T14:40:24.208983+0000","last_in_change":"2024-04-02T14:40:04.615076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T14:36:11.376053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T14:41:20.030393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T14:41:20.917405+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fd83a35-8d86-4932-adfb-5185155bec3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6803","nonce":2100120191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6805","nonce":2100120191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6809","nonce":2100120191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2100120191},{"type":"v1","addr":"172.21.15.62:6807","nonce":2100120191}]},"public_addr":"172.21.15.62:6803/2100120191","cluster_addr":"172.21.15.62:6805/2100120191","heartbeat_back_addr":"172.21.15.62:6809/2100120191","heartbeat_front_addr":"172.21.15.62:6807/2100120191","state":["exists","up"]},{"osd":1,"uuid":"801d3b5d-05f4-4c9e-8ff4-46d8ca9d22b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6801","nonce":124870295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6803","nonce":124870295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6807","nonce":124870295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":124870295},{"type":"v1","addr":"172.21.15.113:6805","nonce":124870295}]},"public_addr":"172.21.15.113:6801/124870295","cluster_addr":"172.21.15.113:6803/124870295","heartbeat_back_addr":"172.21.15.113:6807/124870295","heartbeat_front_addr":"172.21.15.113:6805/124870295","state":["exists","up"]},{"osd":2,"uuid":"26dd6e9a-9371-40b8-bdb7-909b89f97877","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6801","nonce":3439625144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6803","nonce":3439625144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6807","nonce":3439625144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3439625144},{"type":"v1","addr":"172.21.15.204:6805","nonce":3439625144}]},"public_addr":"172.21.15.204:6801/3439625144","cluster_addr":"172.21.15.204:6803/3439625144","heartbeat_back_addr":"172.21.15.204:6807/3439625144","heartbeat_front_addr":"172.21.15.204:6805/3439625144","state":["exists","up"]},{"osd":3,"uuid":"8017fd90-25ff-460d-8273-c49a205ae472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6811","nonce":263141868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6813","nonce":263141868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6817","nonce":263141868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":263141868},{"type":"v1","addr":"172.21.15.62:6815","nonce":263141868}]},"public_addr":"172.21.15.62:6811/263141868","cluster_addr":"172.21.15.62:6813/263141868","heartbeat_back_addr":"172.21.15.62:6817/263141868","heartbeat_front_addr":"172.21.15.62:6815/263141868","state":["exists","up"]},{"osd":4,"uuid":"5b899f2e-b8bc-4dce-991e-e222816cd0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6809","nonce":3019628927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6811","nonce":3019628927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6815","nonce":3019628927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3019628927},{"type":"v1","addr":"172.21.15.113:6813","nonce":3019628927}]},"public_addr":"172.21.15.113:6809/3019628927","cluster_addr":"172.21.15.113:6811/3019628927","heartbeat_back_addr":"172.21.15.113:6815/3019628927","heartbeat_front_addr":"172.21.15.113:6813/3019628927","state":["exists","up"]},{"osd":5,"uuid":"65e6ae99-58a3-4ca6-b090-62db34e993b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6809","nonce":730785574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6811","nonce":730785574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6815","nonce":730785574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":730785574},{"type":"v1","addr":"172.21.15.204:6813","nonce":730785574}]},"public_addr":"172.21.15.204:6809/730785574","cluster_addr":"172.21.15.204:6811/730785574","heartbeat_back_addr":"172.21.15.204:6815/730785574","heartbeat_front_addr":"172.21.15.204:6813/730785574","state":["exists","up"]},{"osd":6,"uuid":"6dfddadf-9e19-4690-aa90-b9bf9044e6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6819","nonce":2905631892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6821","nonce":2905631892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6825","nonce":2905631892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2905631892},{"type":"v1","addr":"172.21.15.62:6823","nonce":2905631892}]},"public_addr":"172.21.15.62:6819/2905631892","cluster_addr":"172.21.15.62:6821/2905631892","heartbeat_back_addr":"172.21.15.62:6825/2905631892","heartbeat_front_addr":"172.21.15.62:6823/2905631892","state":["exists","up"]},{"osd":7,"uuid":"3097ff07-ddef-4070-88b8-d3002833f1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6817","nonce":3862122396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6819","nonce":3862122396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6823","nonce":3862122396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3862122396},{"type":"v1","addr":"172.21.15.113:6821","nonce":3862122396}]},"public_addr":"172.21.15.113:6817/3862122396","cluster_addr":"172.21.15.113:6819/3862122396","heartbeat_back_addr":"172.21.15.113:6823/3862122396","heartbeat_front_addr":"172.21.15.113:6821/3862122396","state":["exists","up"]},{"osd":8,"uuid":"12ef880a-ec50-4a20-9edd-f25fcf4ac28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6817","nonce":499713626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6819","nonce":499713626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6823","nonce":499713626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":499713626},{"type":"v1","addr":"172.21.15.204:6821","nonce":499713626}]},"public_addr":"172.21.15.204:6817/499713626","cluster_addr":"172.21.15.204:6819/499713626","heartbeat_back_addr":"172.21.15.204:6823/499713626","heartbeat_front_addr":"172.21.15.204:6821/499713626","state":["exists","up"]},{"osd":9,"uuid":"2f2258db-f7e1-432b-be11-a2299b46c30f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6827","nonce":465866320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6829","nonce":465866320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6833","nonce":465866320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":465866320},{"type":"v1","addr":"172.21.15.62:6831","nonce":465866320}]},"public_addr":"172.21.15.62:6827/465866320","cluster_addr":"172.21.15.62:6829/465866320","heartbeat_back_addr":"172.21.15.62:6833/465866320","heartbeat_front_addr":"172.21.15.62:6831/465866320","state":["exists","up"]},{"osd":10,"uuid":"1ccb9dfd-397e-4996-a37d-1161181a34cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6825","nonce":4261348940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6827","nonce":4261348940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6831","nonce":4261348940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":4261348940},{"type":"v1","addr":"172.21.15.113:6829","nonce":4261348940}]},"public_addr":"172.21.15.113:6825/4261348940","cluster_addr":"172.21.15.113:6827/4261348940","heartbeat_back_addr":"172.21.15.113:6831/4261348940","heartbeat_front_addr":"172.21.15.113:6829/4261348940","state":["exists","up"]},{"osd":11,"uuid":"d6f10568-eebc-4be6-92eb-942cc4afa1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6825","nonce":1914333038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6827","nonce":1914333038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6831","nonce":1914333038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1914333038},{"type":"v1","addr":"172.21.15.204:6829","nonce":1914333038}]},"public_addr":"172.21.15.204:6825/1914333038","cluster_addr":"172.21.15.204:6827/1914333038","heartbeat_back_addr":"172.21.15.204:6831/1914333038","heartbeat_front_addr":"172.21.15.204:6829/1914333038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:16.603833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:35:42.487726+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:06.982708+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:36:40.201181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:06.588638+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:37:30.940281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:05.715104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:31.750445+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:38:56.545416+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:30.913708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:39:57.643493+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T14:40:22.555368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/129789951":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/79979639":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6801/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:6800/1995176952":"2024-04-03T14:44:00.087305+0000","172.21.15.62:6800/1356430694":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2679742291":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/3147101239":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/3831753172":"2024-04-03T14:33:19.211822+0000","172.21.15.62:0/2106382100":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6801/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:6800/3623171244":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/566820061":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3400451184":"2024-04-03T14:44:00.087305+0000","172.21.15.62:0/3824719077":"2024-04-03T14:32:43.373728+0000","172.21.15.62:0/2038939412":"2024-04-03T14:33:19.211822+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:55:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.260+0000 7f09e7000640 1 -- 172.21.15.62:0/1915974369 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09c80a3da0 msgr2=0x7f09c80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.260+0000 7f09e7000640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09c80a3da0 0x7f09c80a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0095f0 tx=0x7f09cc005f90 comp rx=0 tx=0).stop 2024-04-02T14:55:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.260+0000 7f09e7000640 1 -- 172.21.15.62:0/1915974369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e0103ad0 msgr2=0x7f09e019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.260+0000 7f09e7000640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e0103ad0 0x7f09e019a490 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f09d4005ab0 tx=0x7f09d402dc00 comp rx=0 tx=0).stop 2024-04-02T14:55:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.260+0000 7f09e7000640 1 -- 172.21.15.62:0/1915974369 shutdown_connections 2024-04-02T14:55:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.260+0000 7f09e7000640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f09c80a3da0 0x7f09c80a6260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.260+0000 7f09e7000640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0109b70 0x7f09e019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.260+0000 7f09e7000640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e0104490 0x7f09e019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.260+0000 7f09e7000640 1 --2- 172.21.15.62:0/1915974369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e0103ad0 0x7f09e019a490 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.260+0000 7f09e7000640 1 -- 172.21.15.62:0/1915974369 >> 172.21.15.62:0/1915974369 conn(0x7f09e00ff5a0 msgr2=0x7f09e00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.261+0000 7f09e7000640 1 -- 172.21.15.62:0/1915974369 shutdown_connections 2024-04-02T14:55:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.261+0000 7f09e7000640 1 -- 172.21.15.62:0/1915974369 wait complete. 2024-04-02T14:55:01.269 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-02T14:55:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.488+0000 7f0c17200640 1 -- 172.21.15.62:0/2746293005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10108ec0 msgr2=0x7f0c10110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.488+0000 7f0c17200640 1 --2- 172.21.15.62:0/2746293005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10108ec0 0x7f0c10110a90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009ca0 tx=0x7f0c0c02c100 comp rx=0 tx=0).stop 2024-04-02T14:55:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.488+0000 7f0c17200640 1 -- 172.21.15.62:0/2746293005 shutdown_connections 2024-04-02T14:55:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.488+0000 7f0c17200640 1 --2- 172.21.15.62:0/2746293005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10108ec0 0x7f0c10110a90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.488+0000 7f0c17200640 1 --2- 172.21.15.62:0/2746293005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c101047a0 0x7f0c101087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.488+0000 7f0c17200640 1 --2- 172.21.15.62:0/2746293005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10103df0 0x7f0c101041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.488+0000 7f0c17200640 1 -- 172.21.15.62:0/2746293005 >> 172.21.15.62:0/2746293005 conn(0x7f0c100ff630 msgr2=0x7f0c10101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.489+0000 7f0c17200640 1 -- 172.21.15.62:0/2746293005 shutdown_connections 2024-04-02T14:55:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.489+0000 7f0c17200640 1 -- 172.21.15.62:0/2746293005 wait complete. 2024-04-02T14:55:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.490+0000 7f0c17200640 1 Processor -- start 2024-04-02T14:55:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.490+0000 7f0c17200640 1 -- start start 2024-04-02T14:55:01.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.490+0000 7f0c17200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c10103df0 0x7f0c1019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c17200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10108ec0 0x7f0c1019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c17200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c1019edc0 0x7f0c1019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c17200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0c101140b0 con 0x7f0c10103df0 2024-04-02T14:55:01.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c17200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10113f30 con 0x7f0c1019edc0 2024-04-02T14:55:01.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c17200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10114230 con 0x7f0c10108ec0 2024-04-02T14:55:01.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c16800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c1019edc0 0x7f0c1019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c15e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c10103df0 0x7f0c1019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c16800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c1019edc0 0x7f0c1019f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:55076/0 (socket says 172.21.15.62:55076) 2024-04-02T14:55:01.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c16800640 1 -- 172.21.15.62:0/3330556370 learned_addr learned my addr 172.21.15.62:0/3330556370 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:55:01.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c15400640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10108ec0 0x7f0c1019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c16800640 1 -- 172.21.15.62:0/3330556370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10108ec0 msgr2=0x7f0c1019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c16800640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10108ec0 0x7f0c1019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c16800640 1 -- 172.21.15.62:0/3330556370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c10103df0 msgr2=0x7f0c1019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c16800640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c10103df0 0x7f0c1019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.491+0000 7f0c16800640 1 -- 172.21.15.62:0/3330556370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1019f8b0 con 0x7f0c1019edc0 2024-04-02T14:55:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.492+0000 7f0c16800640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c1019edc0 0x7f0c1019f270 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c02c710 tx=0x7f0c0c0346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.492+0000 7f0c0aa00640 1 -- 172.21.15.62:0/3330556370 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c039040 con 0x7f0c1019edc0 2024-04-02T14:55:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.492+0000 7f0c17200640 1 -- 172.21.15.62:0/3330556370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1019fb40 con 0x7f0c1019edc0 2024-04-02T14:55:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.492+0000 7f0c0aa00640 1 -- 172.21.15.62:0/3330556370 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0c0c044070 con 0x7f0c1019edc0 2024-04-02T14:55:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.492+0000 7f0c0aa00640 1 -- 172.21.15.62:0/3330556370 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c004440 con 0x7f0c1019edc0 2024-04-02T14:55:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.492+0000 7f0c17200640 1 -- 172.21.15.62:0/3330556370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c101a7360 con 0x7f0c1019edc0 2024-04-02T14:55:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.493+0000 7f0c17200640 1 -- 172.21.15.62:0/3330556370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c10068c60 con 0x7f0c1019edc0 2024-04-02T14:55:01.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.496+0000 7f0c0aa00640 1 -- 172.21.15.62:0/3330556370 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0c0c032840 con 0x7f0c1019edc0 2024-04-02T14:55:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.497+0000 7f0c0aa00640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0bf80a3d20 0x7f0bf80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.497+0000 7f0c0aa00640 1 -- 172.21.15.62:0/3330556370 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f0c0c0f2fe0 con 0x7f0c1019edc0 2024-04-02T14:55:01.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.497+0000 7f0c15e00640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0bf80a3d20 0x7f0bf80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.498+0000 7f0c15e00640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0bf80a3d20 0x7f0bf80a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009bd0 tx=0x7f0c04005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:01.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.499+0000 7f0c0aa00640 1 -- 172.21.15.62:0/3330556370 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0c0c04e020 con 0x7f0c1019edc0 2024-04-02T14:55:01.584 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-mon[32116]: pgmap v341: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:55:01.584 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/4112772924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:55:01.584 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1915974369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:55:01.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.746+0000 7f0c17200640 1 -- 172.21.15.62:0/3330556370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0c1010c150 con 0x7f0c1019edc0 2024-04-02T14:55:01.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.760+0000 7f0c0aa00640 1 -- 172.21.15.62:0/3330556370 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) ==== 161+0+0 (secure 0 0 0) 0x7f0c0c0bf750 con 0x7f0c1019edc0 2024-04-02T14:55:01.752 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-02T14:55:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 -- 172.21.15.62:0/3330556370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0bf80a3d20 msgr2=0x7f0bf80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0bf80a3d20 0x7f0bf80a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009bd0 tx=0x7f0c04005f90 comp rx=0 tx=0).stop 2024-04-02T14:55:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 -- 172.21.15.62:0/3330556370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c1019edc0 msgr2=0x7f0c1019f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c1019edc0 0x7f0c1019f270 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c02c710 tx=0x7f0c0c0346f0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 -- 172.21.15.62:0/3330556370 shutdown_connections 2024-04-02T14:55:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f0bf80a3d20 0x7f0bf80a61e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c1019edc0 0x7f0c1019f270 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10108ec0 0x7f0c1019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 --2- 172.21.15.62:0/3330556370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c10103df0 0x7f0c1019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 -- 172.21.15.62:0/3330556370 >> 172.21.15.62:0/3330556370 conn(0x7f0c100ff630 msgr2=0x7f0c10100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:01.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 -- 172.21.15.62:0/3330556370 shutdown_connections 2024-04-02T14:55:01.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.766+0000 7f0c17200640 1 -- 172.21.15.62:0/3330556370 wait complete. 2024-04-02T14:55:01.786 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-mon[36135]: pgmap v341: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:55:01.787 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/4112772924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:55:01.787 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1915974369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:55:01.789 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-02T14:55:01.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-mon[35948]: pgmap v341: 97 pgs: 97 active+clean; 18 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:55:01.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/4112772924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:55:01.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1915974369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:55:01.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.993+0000 7f76cf200640 1 -- 172.21.15.62:0/2577705712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c810d3e0 msgr2=0x7f76c8114fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.993+0000 7f76cf200640 1 --2- 172.21.15.62:0/2577705712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c810d3e0 0x7f76c8114fb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009be0 tx=0x7f76c002bef0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.993+0000 7f76cf200640 1 -- 172.21.15.62:0/2577705712 shutdown_connections 2024-04-02T14:55:01.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.993+0000 7f76cf200640 1 --2- 172.21.15.62:0/2577705712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c810d3e0 0x7f76c8114fb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.993+0000 7f76cf200640 1 --2- 172.21.15.62:0/2577705712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c8108cc0 0x7f76c810cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.993+0000 7f76cf200640 1 --2- 172.21.15.62:0/2577705712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c8108310 0x7f76c81086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.993+0000 7f76cf200640 1 -- 172.21.15.62:0/2577705712 >> 172.21.15.62:0/2577705712 conn(0x7f76c8076040 msgr2=0x7f76c8078460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:01.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.993+0000 7f76cf200640 1 -- 172.21.15.62:0/2577705712 shutdown_connections 2024-04-02T14:55:01.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.993+0000 7f76cf200640 1 -- 172.21.15.62:0/2577705712 wait complete. 2024-04-02T14:55:01.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.994+0000 7f76cf200640 1 Processor -- start 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.994+0000 7f76cf200640 1 -- start start 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.994+0000 7f76cf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c8108310 0x7f76c81a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.994+0000 7f76cf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c810d3e0 0x7f76c81a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.994+0000 7f76cf200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c819e860 0x7f76c819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.994+0000 7f76cf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8118600 con 0x7f76c8108310 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.994+0000 7f76cf200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8118480 con 0x7f76c819e860 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.994+0000 7f76cf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8118780 con 0x7f76c810d3e0 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.994+0000 7f76c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c8108310 0x7f76c81a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.995+0000 7f76c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c8108310 0x7f76c81a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37580/0 (socket says 172.21.15.62:37580) 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.995+0000 7f76c7e00640 1 -- 172.21.15.62:0/1256422621 learned_addr learned my addr 172.21.15.62:0/1256422621 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.995+0000 7f76bf400640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c810d3e0 0x7f76c81a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.995+0000 7f76cd200640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c819e860 0x7f76c819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.995+0000 7f76c7e00640 1 -- 172.21.15.62:0/1256422621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c810d3e0 msgr2=0x7f76c81a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.995+0000 7f76c7e00640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c810d3e0 0x7f76c81a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.995+0000 7f76c7e00640 1 -- 172.21.15.62:0/1256422621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c819e860 msgr2=0x7f76c819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.995+0000 7f76c7e00640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c819e860 0x7f76c819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.995+0000 7f76c7e00640 1 -- 172.21.15.62:0/1256422621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c819f380 con 0x7f76c8108310 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.995+0000 7f76c7e00640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c8108310 0x7f76c81a46b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f76b800e9c0 tx=0x7f76b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.995+0000 7f76c5600640 1 -- 172.21.15.62:0/1256422621 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b800ccc0 con 0x7f76c8108310 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.996+0000 7f76cf200640 1 -- 172.21.15.62:0/1256422621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c819f670 con 0x7f76c8108310 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.996+0000 7f76c5600640 1 -- 172.21.15.62:0/1256422621 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f76b8005520 con 0x7f76c8108310 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.996+0000 7f76c5600640 1 -- 172.21.15.62:0/1256422621 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8010690 con 0x7f76c8108310 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.996+0000 7f76cf200640 1 -- 172.21.15.62:0/1256422621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c81ab910 con 0x7f76c8108310 2024-04-02T14:55:01.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.997+0000 7f76cf200640 1 -- 172.21.15.62:0/1256422621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c8068d40 con 0x7f76c8108310 2024-04-02T14:55:01.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:01.999+0000 7f76c5600640 1 -- 172.21.15.62:0/1256422621 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f76b80046c0 con 0x7f76c8108310 2024-04-02T14:55:01.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.000+0000 7f76c5600640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f76ac0a3d20 0x7f76ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:01.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.000+0000 7f76c5600640 1 -- 172.21.15.62:0/1256422621 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8870+0+0 (secure 0 0 0) 0x7f76b80d0c40 con 0x7f76c8108310 2024-04-02T14:55:01.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.000+0000 7f76bf400640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f76ac0a3d20 0x7f76ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:01.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.001+0000 7f76bf400640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f76ac0a3d20 0x7f76ac0a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f76c8107360 tx=0x7f76b000b390 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:01.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.004+0000 7f76c5600640 1 -- 172.21.15.62:0/1256422621 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f76b809d120 con 0x7f76c8108310 2024-04-02T14:55:02.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[32112]: 2024-04-02T14:55:01.749+0000 7f7744a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T14:55:02.084 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: ignoring --setuser ceph since I am not root 2024-04-02T14:55:02.084 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: ignoring --setgroup ceph since I am not root 2024-04-02T14:55:02.084 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-mds[72320]: -- 172.21.15.62:0/1080064184 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d4c3404e0 con 0x564d4c317000 2024-04-02T14:55:02.084 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-mds[72320]: -- 172.21.15.62:0/1080064184 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d4c343a40 con 0x564d4c317000 2024-04-02T14:55:02.085 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: starting mds.a at 2024-04-02T14:55:02.085 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d[71341]: ignoring --setuser ceph since I am not root 2024-04-02T14:55:02.085 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d[71341]: ignoring --setgroup ceph since I am not root 2024-04-02T14:55:02.085 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-mds[71345]: -- 172.21.15.62:0/2928385788 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55557874fa40 con 0x555578722c00 2024-04-02T14:55:02.085 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d[71341]: starting mds.d at 2024-04-02T14:55:02.085 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-g[71943]: ignoring --setuser ceph since I am not root 2024-04-02T14:55:02.085 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-g[71943]: ignoring --setgroup ceph since I am not root 2024-04-02T14:55:02.085 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-mds[71950]: -- 172.21.15.62:0/4265794258 <== mon.2 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd7d0fda40 con 0x55bd7d0d0800 2024-04-02T14:55:02.086 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:55:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-g[71943]: starting mds.g at 2024-04-02T14:55:02.111 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b[67157]: ignoring --setuser ceph since I am not root 2024-04-02T14:55:02.111 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b[67157]: ignoring --setgroup ceph since I am not root 2024-04-02T14:55:02.111 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-mds[67162]: -- 172.21.15.113:0/2481798965 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561507b0ba40 con 0x561507adec00 2024-04-02T14:55:02.111 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b[67157]: starting mds.b at 2024-04-02T14:55:02.112 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e[66537]: ignoring --setuser ceph since I am not root 2024-04-02T14:55:02.112 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e[66537]: ignoring --setgroup ceph since I am not root 2024-04-02T14:55:02.112 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-mds[66541]: -- 172.21.15.113:0/1897633952 <== mon.2 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564ce5599a40 con 0x564ce556c800 2024-04-02T14:55:02.112 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e[66537]: starting mds.e at 2024-04-02T14:55:02.112 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: ignoring --setuser ceph since I am not root 2024-04-02T14:55:02.112 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: ignoring --setgroup ceph since I am not root 2024-04-02T14:55:02.113 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-mds[66850]: -- 172.21.15.113:0/4004643493 <== mon.2 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55909849ba40 con 0x55909846ec00 2024-04-02T14:55:02.113 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: starting mds.h at 2024-04-02T14:55:02.113 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k[66212]: ignoring --setuser ceph since I am not root 2024-04-02T14:55:02.113 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k[66212]: ignoring --setgroup ceph since I am not root 2024-04-02T14:55:02.113 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-mds[66216]: -- 172.21.15.113:0/1024404752 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5650602cda40 con 0x5650602a0c00 2024-04-02T14:55:02.113 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:55:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k[66212]: starting mds.k at 2024-04-02T14:55:02.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.249+0000 7f76cf200640 1 -- 172.21.15.62:0/1256422621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f76c8108d50 con 0x7f76c8108310 2024-04-02T14:55:02.250 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c[67310]: ignoring --setuser ceph since I am not root 2024-04-02T14:55:02.250 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c[67310]: ignoring --setgroup ceph since I am not root 2024-04-02T14:55:02.250 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-mds[67314]: -- 172.21.15.204:0/935715163 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561147851a40 con 0x561147824c00 2024-04-02T14:55:02.251 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c[67310]: starting mds.c at 2024-04-02T14:55:02.251 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i[67624]: ignoring --setuser ceph since I am not root 2024-04-02T14:55:02.251 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i[67624]: ignoring --setgroup ceph since I am not root 2024-04-02T14:55:02.251 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-mds[67633]: -- 172.21.15.204:0/1794164976 <== mon.2 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dbb5115a40 con 0x55dbb50e9000 2024-04-02T14:55:02.251 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i[67624]: starting mds.i at 2024-04-02T14:55:02.251 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f[67002]: ignoring --setuser ceph since I am not root 2024-04-02T14:55:02.251 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f[67002]: ignoring --setgroup ceph since I am not root 2024-04-02T14:55:02.251 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-mds[67006]: -- 172.21.15.204:0/1565189719 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a59a2d7a40 con 0x55a59a2aa800 2024-04-02T14:55:02.251 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:55:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f[67002]: starting mds.f at 2024-04-02T14:55:02.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.762+0000 7f76c5600640 1 -- 172.21.15.62:0/1256422621 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) ==== 104+0+0 (secure 0 0 0) 0x7f76b809ca20 con 0x7f76c8108310 2024-04-02T14:55:02.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 -- 172.21.15.62:0/1256422621 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f76ac0a3d20 msgr2=0x7f76ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:02.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f76ac0a3d20 0x7f76ac0a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f76c8107360 tx=0x7f76b000b390 comp rx=0 tx=0).stop 2024-04-02T14:55:02.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 -- 172.21.15.62:0/1256422621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c8108310 msgr2=0x7f76c81a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:02.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c8108310 0x7f76c81a46b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f76b800e9c0 tx=0x7f76b800ee80 comp rx=0 tx=0).stop 2024-04-02T14:55:02.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 -- 172.21.15.62:0/1256422621 shutdown_connections 2024-04-02T14:55:02.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f76ac0a3d20 0x7f76ac0a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:02.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c819e860 0x7f76c819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:02.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c810d3e0 0x7f76c81a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:02.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 --2- 172.21.15.62:0/1256422621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c8108310 0x7f76c81a46b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:02.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 -- 172.21.15.62:0/1256422621 >> 172.21.15.62:0/1256422621 conn(0x7f76c8076040 msgr2=0x7f76c8076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:02.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 -- 172.21.15.62:0/1256422621 shutdown_connections 2024-04-02T14:55:02.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.766+0000 7f76cf200640 1 -- 172.21.15.62:0/1256422621 wait complete. 2024-04-02T14:55:02.779 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:02 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/3330556370' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T14:55:02.779 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:02 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T14:55:02.779 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:02 smithi062 ceph-mon[32116]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T14:55:02.779 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:02 smithi062 ceph-mon[32116]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T14:55:02.779 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:02 smithi062 ceph-mon[32116]: Health check cleared: MDS_DAMAGE (was: 3 MDSs report damaged metadata) 2024-04-02T14:55:02.780 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:02 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T14:55:02.780 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:02 smithi062 ceph-mon[32116]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T14:55:02.780 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:02 smithi062 ceph-mon[32116]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T14:55:02.780 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:02 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1256422621' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:55:02.782 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-02T14:55:02.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:02 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/3330556370' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T14:55:02.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:02 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T14:55:02.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:02 smithi113 ceph-mon[36135]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T14:55:02.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:02 smithi113 ceph-mon[36135]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T14:55:02.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:02 smithi113 ceph-mon[36135]: Health check cleared: MDS_DAMAGE (was: 3 MDSs report damaged metadata) 2024-04-02T14:55:02.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:02 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T14:55:02.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:02 smithi113 ceph-mon[36135]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T14:55:02.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:02 smithi113 ceph-mon[36135]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T14:55:02.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:02 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1256422621' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:55:02.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.998+0000 7f88cfe00640 1 -- 172.21.15.62:0/2378933227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88c8101a80 msgr2=0x7f88c8101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:02.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.998+0000 7f88cfe00640 1 --2- 172.21.15.62:0/2378933227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88c8101a80 0x7f88c8101f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009930 tx=0x7f88b402bc30 comp rx=0 tx=0).stop 2024-04-02T14:55:02.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.998+0000 7f88cfe00640 1 -- 172.21.15.62:0/2378933227 shutdown_connections 2024-04-02T14:55:02.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.998+0000 7f88cfe00640 1 --2- 172.21.15.62:0/2378933227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88c8102440 0x7f88c810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:02.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.998+0000 7f88cfe00640 1 --2- 172.21.15.62:0/2378933227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88c8101a80 0x7f88c8101f00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:02.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.998+0000 7f88cfe00640 1 --2- 172.21.15.62:0/2378933227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8107a90 0x7f88c8107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:02.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.999+0000 7f88cfe00640 1 -- 172.21.15.62:0/2378933227 >> 172.21.15.62:0/2378933227 conn(0x7f88c80fd590 msgr2=0x7f88c80ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:02.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.999+0000 7f88cfe00640 1 -- 172.21.15.62:0/2378933227 shutdown_connections 2024-04-02T14:55:02.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:02.999+0000 7f88cfe00640 1 -- 172.21.15.62:0/2378933227 wait complete. 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.000+0000 7f88cfe00640 1 Processor -- start 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.000+0000 7f88cfe00640 1 -- start start 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.000+0000 7f88cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101a80 0x7f88c8073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.000+0000 7f88cfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88c8102440 0x7f88c8074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.000+0000 7f88cfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88c8107a90 0x7f88c8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.000+0000 7f88cfe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8111f40 con 0x7f88c8101a80 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.000+0000 7f88cfe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8111dc0 con 0x7f88c8102440 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.000+0000 7f88cfe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81120c0 con 0x7f88c8107a90 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.000+0000 7f88cd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101a80 0x7f88c8073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.001+0000 7f88cd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101a80 0x7f88c8073e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45910/0 (socket says 172.21.15.62:45910) 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.001+0000 7f88cca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88c8102440 0x7f88c8074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:02.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.001+0000 7f88cd400640 1 -- 172.21.15.62:0/19616150 learned_addr learned my addr 172.21.15.62:0/19616150 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:55:02.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.001+0000 7f88cde00640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88c8107a90 0x7f88c8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:02.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.001+0000 7f88cd400640 1 -- 172.21.15.62:0/19616150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88c8107a90 msgr2=0x7f88c8074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:02.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.001+0000 7f88cd400640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88c8107a90 0x7f88c8074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:02.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.001+0000 7f88cd400640 1 -- 172.21.15.62:0/19616150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88c8102440 msgr2=0x7f88c8074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:02.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.001+0000 7f88cd400640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88c8102440 0x7f88c8074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:02.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.001+0000 7f88cd400640 1 -- 172.21.15.62:0/19616150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c81a69e0 con 0x7f88c8101a80 2024-04-02T14:55:02.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.001+0000 7f88cca00640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88c8102440 0x7f88c8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:55:02.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.001+0000 7f88cd400640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101a80 0x7f88c8073e00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00e9f0 tx=0x7f88bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:02.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.002+0000 7f88c6000640 1 -- 172.21.15.62:0/19616150 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc00cd80 con 0x7f88c8101a80 2024-04-02T14:55:02.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.002+0000 7f88c6000640 1 -- 172.21.15.62:0/19616150 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f88bc010930 con 0x7f88c8101a80 2024-04-02T14:55:02.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.002+0000 7f88cfe00640 1 -- 172.21.15.62:0/19616150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c81a6cd0 con 0x7f88c8101a80 2024-04-02T14:55:02.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.002+0000 7f88c6000640 1 -- 172.21.15.62:0/19616150 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc018b00 con 0x7f88c8101a80 2024-04-02T14:55:02.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.002+0000 7f88cfe00640 1 -- 172.21.15.62:0/19616150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c81a71a0 con 0x7f88c8101a80 2024-04-02T14:55:02.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.003+0000 7f88cfe00640 1 -- 172.21.15.62:0/19616150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c80688f0 con 0x7f88c8101a80 2024-04-02T14:55:02.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.005+0000 7f88c6000640 1 -- 172.21.15.62:0/19616150 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f88bc018ca0 con 0x7f88c8101a80 2024-04-02T14:55:02.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.006+0000 7f88c6000640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88b00a3d20 0x7f88b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:02.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.007+0000 7f88c6000640 1 -- 172.21.15.62:0/19616150 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8870+0+0 (secure 0 0 0) 0x7f88bc0d0c70 con 0x7f88c8101a80 2024-04-02T14:55:02.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.007+0000 7f88cca00640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88b00a3d20 0x7f88b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:02.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.008+0000 7f88cca00640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88b00a3d20 0x7f88b00a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005ab0 tx=0x7f88b4005fd0 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:03.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:02 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/3330556370' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T14:55:03.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:02 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T14:55:03.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:02 smithi204 ceph-mon[35948]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T14:55:03.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:02 smithi204 ceph-mon[35948]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T14:55:03.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:02 smithi204 ceph-mon[35948]: Health check cleared: MDS_DAMAGE (was: 3 MDSs report damaged metadata) 2024-04-02T14:55:03.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:02 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T14:55:03.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:02 smithi204 ceph-mon[35948]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T14:55:03.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:02 smithi204 ceph-mon[35948]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T14:55:03.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:02 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1256422621' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:55:03.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.011+0000 7f88c6000640 1 -- 172.21.15.62:0/19616150 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f88bc09cb60 con 0x7f88c8101a80 2024-04-02T14:55:03.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.217+0000 7f88cfe00640 1 -- 172.21.15.62:0/19616150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f88c802d370 con 0x7f88c8101a80 2024-04-02T14:55:03.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.765+0000 7f88c6000640 1 -- 172.21.15.62:0/19616150 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7f88bc0a0e60 con 0x7f88c8101a80 2024-04-02T14:55:03.756 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-02T14:55:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 -- 172.21.15.62:0/19616150 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88b00a3d20 msgr2=0x7f88b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88b00a3d20 0x7f88b00a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005ab0 tx=0x7f88b4005fd0 comp rx=0 tx=0).stop 2024-04-02T14:55:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 -- 172.21.15.62:0/19616150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101a80 msgr2=0x7f88c8073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101a80 0x7f88c8073e00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00e9f0 tx=0x7f88bc00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:55:03.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 -- 172.21.15.62:0/19616150 shutdown_connections 2024-04-02T14:55:03.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7f88b00a3d20 0x7f88b00a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:03.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88c8107a90 0x7f88c8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:03.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88c8102440 0x7f88c8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:03.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 --2- 172.21.15.62:0/19616150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101a80 0x7f88c8073e00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:03.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 -- 172.21.15.62:0/19616150 >> 172.21.15.62:0/19616150 conn(0x7f88c80fd590 msgr2=0x7f88c80fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:03.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 -- 172.21.15.62:0/19616150 shutdown_connections 2024-04-02T14:55:03.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:03.769+0000 7f88cfe00640 1 -- 172.21.15.62:0/19616150 wait complete. 2024-04-02T14:55:03.786 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: pgmap v343: 97 pgs: 97 active+clean; 18 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-04-02T14:55:03.786 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T14:55:03.786 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T14:55:03.786 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: Cluster is now healthy 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1256422621' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.113:6833/3001185020,v1:172.21.15.113:6835/3001185020] up:boot 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.113:6832/1916919525,v1:172.21.15.113:6834/1916919525] up:boot 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.204:6837/3553081605,v1:172.21.15.204:6839/3553081605] up:boot 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.204:6834/1062377545,v1:172.21.15.204:6835/1062377545] up:boot 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.113:6836/3682554648,v1:172.21.15.113:6837/3682554648] up:boot 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.204:6836/3511825326,v1:172.21.15.204:6838/3511825326] up:boot 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.113:6838/559347450,v1:172.21.15.113:6839/559347450] up:boot 2024-04-02T14:55:03.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.62:6836/954718497,v1:172.21.15.62:6837/954718497] up:boot 2024-04-02T14:55:03.788 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.62:6838/1687562235,v1:172.21.15.62:6839/1687562235] up:boot 2024-04-02T14:55:03.788 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: mds.? [v2:172.21.15.62:6840/448832068,v1:172.21.15.62:6841/448832068] up:boot 2024-04-02T14:55:03.788 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: fsmap 12 up:standby 2024-04-02T14:55:03.788 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T14:55:03.788 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:55:03.788 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:55:03.788 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:55:03.788 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:55:03.788 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:03 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/19616150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:55:03.789 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-02T14:55:03.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: pgmap v343: 97 pgs: 97 active+clean; 18 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-04-02T14:55:03.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T14:55:03.861 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: Cluster is now healthy 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1256422621' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.113:6833/3001185020,v1:172.21.15.113:6835/3001185020] up:boot 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.113:6832/1916919525,v1:172.21.15.113:6834/1916919525] up:boot 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.204:6837/3553081605,v1:172.21.15.204:6839/3553081605] up:boot 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.204:6834/1062377545,v1:172.21.15.204:6835/1062377545] up:boot 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.113:6836/3682554648,v1:172.21.15.113:6837/3682554648] up:boot 2024-04-02T14:55:03.862 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.204:6836/3511825326,v1:172.21.15.204:6838/3511825326] up:boot 2024-04-02T14:55:03.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.113:6838/559347450,v1:172.21.15.113:6839/559347450] up:boot 2024-04-02T14:55:03.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.62:6836/954718497,v1:172.21.15.62:6837/954718497] up:boot 2024-04-02T14:55:03.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.62:6838/1687562235,v1:172.21.15.62:6839/1687562235] up:boot 2024-04-02T14:55:03.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: mds.? [v2:172.21.15.62:6840/448832068,v1:172.21.15.62:6841/448832068] up:boot 2024-04-02T14:55:03.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: fsmap 12 up:standby 2024-04-02T14:55:03.863 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T14:55:03.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:55:03.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:55:03.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:55:03.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:55:03.864 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:03 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/19616150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:55:04.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: pgmap v343: 97 pgs: 97 active+clean; 18 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-04-02T14:55:04.000 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: Cluster is now healthy 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1256422621' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.113:6833/3001185020,v1:172.21.15.113:6835/3001185020] up:boot 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.113:6832/1916919525,v1:172.21.15.113:6834/1916919525] up:boot 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.204:6837/3553081605,v1:172.21.15.204:6839/3553081605] up:boot 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.204:6834/1062377545,v1:172.21.15.204:6835/1062377545] up:boot 2024-04-02T14:55:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.113:6836/3682554648,v1:172.21.15.113:6837/3682554648] up:boot 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.204:6836/3511825326,v1:172.21.15.204:6838/3511825326] up:boot 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.113:6838/559347450,v1:172.21.15.113:6839/559347450] up:boot 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.62:6836/954718497,v1:172.21.15.62:6837/954718497] up:boot 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.62:6838/1687562235,v1:172.21.15.62:6839/1687562235] up:boot 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: mds.? [v2:172.21.15.62:6840/448832068,v1:172.21.15.62:6841/448832068] up:boot 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: fsmap 12 up:standby 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='mgr.25273 172.21.15.62:0/3455889157' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:55:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:03 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/19616150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:55:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.025+0000 7fbb9dc00640 1 -- 172.21.15.62:0/2879811800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb98103f10 msgr2=0x7fbb981042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.025+0000 7fbb9dc00640 1 --2- 172.21.15.62:0/2879811800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb98103f10 0x7fbb981042f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009900 tx=0x7fbb8802bc30 comp rx=0 tx=0).stop 2024-04-02T14:55:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.025+0000 7fbb9dc00640 1 -- 172.21.15.62:0/2879811800 shutdown_connections 2024-04-02T14:55:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.025+0000 7fbb9dc00640 1 --2- 172.21.15.62:0/2879811800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb98108fe0 0x7fbb98110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.025+0000 7fbb9dc00640 1 --2- 172.21.15.62:0/2879811800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb981048c0 0x7fbb98108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.025+0000 7fbb9dc00640 1 --2- 172.21.15.62:0/2879811800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb98103f10 0x7fbb981042f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.025+0000 7fbb9dc00640 1 -- 172.21.15.62:0/2879811800 >> 172.21.15.62:0/2879811800 conn(0x7fbb980ff7b0 msgr2=0x7fbb98101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.025+0000 7fbb9dc00640 1 -- 172.21.15.62:0/2879811800 shutdown_connections 2024-04-02T14:55:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.025+0000 7fbb9dc00640 1 -- 172.21.15.62:0/2879811800 wait complete. 2024-04-02T14:55:04.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.026+0000 7fbb9dc00640 1 Processor -- start 2024-04-02T14:55:04.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb9dc00640 1 -- start start 2024-04-02T14:55:04.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb9dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb98103f10 0x7fbb9819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:04.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb9dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb98108fe0 0x7fbb9819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:04.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb9dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb9819efa0 0x7fbb9819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb9dc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98114180 con 0x7fbb98108fe0 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb9dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98114000 con 0x7fbb98103f10 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb9dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98114300 con 0x7fbb9819efa0 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb96a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb98103f10 0x7fbb9819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb96a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb98103f10 0x7fbb9819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.62:49840/0 (socket says 172.21.15.62:49840) 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb96a00640 1 -- 172.21.15.62:0/1574626542 learned_addr learned my addr 172.21.15.62:0/1574626542 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb8fe00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb98108fe0 0x7fbb9819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.027+0000 7fbb97400640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb9819efa0 0x7fbb9819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.028+0000 7fbb96a00640 1 -- 172.21.15.62:0/1574626542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb9819efa0 msgr2=0x7fbb9819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.028+0000 7fbb96a00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb9819efa0 0x7fbb9819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.028+0000 7fbb96a00640 1 -- 172.21.15.62:0/1574626542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb98108fe0 msgr2=0x7fbb9819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.028+0000 7fbb96a00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb98108fe0 0x7fbb9819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.028+0000 7fbb96a00640 1 -- 172.21.15.62:0/1574626542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9819fa90 con 0x7fbb98103f10 2024-04-02T14:55:04.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.028+0000 7fbb96a00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb98103f10 0x7fbb9819a6e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbb88005ab0 tx=0x7fbb8802dd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:04.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.028+0000 7fbb8f400640 1 -- 172.21.15.62:0/1574626542 <== mon.2 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88043020 con 0x7fbb98103f10 2024-04-02T14:55:04.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.029+0000 7fbb8f400640 1 -- 172.21.15.62:0/1574626542 <== mon.2 v2:172.21.15.113:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb88004250 con 0x7fbb98103f10 2024-04-02T14:55:04.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.029+0000 7fbb9dc00640 1 -- 172.21.15.62:0/1574626542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9819fd20 con 0x7fbb98103f10 2024-04-02T14:55:04.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.029+0000 7fbb8f400640 1 -- 172.21.15.62:0/1574626542 <== mon.2 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88003ac0 con 0x7fbb98103f10 2024-04-02T14:55:04.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.029+0000 7fbb9dc00640 1 -- 172.21.15.62:0/1574626542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb981a7540 con 0x7fbb98103f10 2024-04-02T14:55:04.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.030+0000 7fbb9dc00640 1 -- 172.21.15.62:0/1574626542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb98068d40 con 0x7fbb98103f10 2024-04-02T14:55:04.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.033+0000 7fbb8f400640 1 -- 172.21.15.62:0/1574626542 <== mon.2 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbb88046020 con 0x7fbb98103f10 2024-04-02T14:55:04.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.034+0000 7fbb8f400640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb7c0a3d20 0x7fbb7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:55:04.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.034+0000 7fbb8f400640 1 -- 172.21.15.62:0/1574626542 <== mon.2 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8345+0+0 (secure 0 0 0) 0x7fbb880f2e60 con 0x7fbb98103f10 2024-04-02T14:55:04.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.035+0000 7fbb8fe00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb7c0a3d20 0x7fbb7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:55:04.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.035+0000 7fbb8fe00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb7c0a3d20 0x7fbb7c0a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbb9810a590 tx=0x7fbb80005f90 comp rx=0 tx=0).ready entity=mgr.25273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:55:04.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.036+0000 7fbb8f400640 1 -- 172.21.15.62:0/1574626542 <== mon.2 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbb8804d020 con 0x7fbb98103f10 2024-04-02T14:55:04.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.243+0000 7fbb9dc00640 1 -- 172.21.15.62:0/1574626542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbb9819ba30 con 0x7fbb98103f10 2024-04-02T14:55:04.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.780+0000 7fbb8f400640 1 -- 172.21.15.62:0/1574626542 <== mon.2 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7fbb9819ba30 con 0x7fbb98103f10 2024-04-02T14:55:04.772 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-02T14:55:04.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.786+0000 7fbb9dc00640 1 -- 172.21.15.62:0/1574626542 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb7c0a3d20 msgr2=0x7fbb7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:04.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.787+0000 7fbb9dc00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb7c0a3d20 0x7fbb7c0a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbb9810a590 tx=0x7fbb80005f90 comp rx=0 tx=0).stop 2024-04-02T14:55:04.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.787+0000 7fbb9dc00640 1 -- 172.21.15.62:0/1574626542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb98103f10 msgr2=0x7fbb9819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:55:04.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.787+0000 7fbb9dc00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb98103f10 0x7fbb9819a6e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbb88005ab0 tx=0x7fbb8802dd00 comp rx=0 tx=0).stop 2024-04-02T14:55:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.788+0000 7fbb9dc00640 1 -- 172.21.15.62:0/1574626542 shutdown_connections 2024-04-02T14:55:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.788+0000 7fbb9dc00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.62:6800/110874546,v1:172.21.15.62:6801/110874546] conn(0x7fbb7c0a3d20 0x7fbb7c0a61e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.788+0000 7fbb9dc00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb9819efa0 0x7fbb9819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.788+0000 7fbb9dc00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb98108fe0 0x7fbb9819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.788+0000 7fbb9dc00640 1 --2- 172.21.15.62:0/1574626542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb98103f10 0x7fbb9819a6e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:55:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.788+0000 7fbb9dc00640 1 -- 172.21.15.62:0/1574626542 >> 172.21.15.62:0/1574626542 conn(0x7fbb980ff7b0 msgr2=0x7fbb98100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:55:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.788+0000 7fbb9dc00640 1 -- 172.21.15.62:0/1574626542 shutdown_connections 2024-04-02T14:55:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-02T14:55:04.788+0000 7fbb9dc00640 1 -- 172.21.15.62:0/1574626542 wait complete. 2024-04-02T14:55:04.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:55:04.887 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:55:04.923 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:55:04.955 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-02T14:55:04.955 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T14:55:05.016 DEBUG:teuthology.orchestra.run.smithi113:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T14:55:05.038 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T14:55:05.053 INFO:tasks.cephadm:Stopping all daemons... 2024-04-02T14:55:05.054 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-02T14:55:05.054 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.a 2024-04-02T14:55:05.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:04 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/19616150' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T14:55:05.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:04 smithi062 ceph-mon[32116]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T14:55:05.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:04 smithi062 ceph-mon[32116]: from='client.? 172.21.15.62:0/1574626542' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:55:05.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:04 smithi062 ceph-mon[32116]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:55:05.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:04 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/19616150' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T14:55:05.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:04 smithi113 ceph-mon[36135]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T14:55:05.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:04 smithi113 ceph-mon[36135]: from='client.? 172.21.15.62:0/1574626542' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:55:05.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:04 smithi113 ceph-mon[36135]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:55:05.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:04 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/19616150' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T14:55:05.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:04 smithi204 ceph-mon[35948]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T14:55:05.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:04 smithi204 ceph-mon[35948]: from='client.? 172.21.15.62:0/1574626542' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:55:05.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:04 smithi204 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:55:05.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:05 smithi062 systemd[1]: Stopping Ceph mon.a for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:06.018 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:05 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[32112]: 2024-04-02T14:55:05.746+0000 7f774b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:55:06.018 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:05 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a[32112]: 2024-04-02T14:55:05.746+0000 7f774b800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-02T14:55:06.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:06 smithi062 podman[93389]: 2024-04-02 14:55:06.025541036 +0000 UTC m=+0.435647019 container died 6078bd3e7250e50ac00fc26479425fe00738ca02c598d3f54db0d0f07e79d1c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, ceph=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-02T14:55:06.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:06 smithi062 podman[93389]: 2024-04-02 14:55:06.533279557 +0000 UTC m=+0.943385541 container cleanup 6078bd3e7250e50ac00fc26479425fe00738ca02c598d3f54db0d0f07e79d1c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-02T14:55:06.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:06 smithi062 bash[93389]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a 2024-04-02T14:55:07.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Apr 02 14:55:06 smithi062 podman[93399]: 2024-04-02 14:55:06.759228406 +0000 UTC m=+0.717939786 container remove 6078bd3e7250e50ac00fc26479425fe00738ca02c598d3f54db0d0f07e79d1c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-a, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-02T14:55:07.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.a.service' 2024-04-02T14:55:07.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:55:07.513 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-02T14:55:07.514 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-02T14:55:07.514 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.b 2024-04-02T14:55:07.816 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:07 smithi113 systemd[1]: Stopping Ceph mon.b for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:08.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:07 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-b[36131]: 2024-04-02T14:55:07.825+0000 7f7746f23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:55:08.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:07 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-b[36131]: 2024-04-02T14:55:07.825+0000 7f7746f23700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-02T14:55:08.111 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:07 smithi113 podman[72708]: 2024-04-02 14:55:07.907387234 +0000 UTC m=+0.194129560 container died 65380f09a8d9491c2cd01961ce0e46cc31430eed4bed74d61e131762dac47eb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-02T14:55:08.509 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:08 smithi113 podman[72708]: 2024-04-02 14:55:08.275541559 +0000 UTC m=+0.562283886 container cleanup 65380f09a8d9491c2cd01961ce0e46cc31430eed4bed74d61e131762dac47eb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-b, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5) 2024-04-02T14:55:08.509 INFO:journalctl@ceph.mon.b.smithi113.stdout:Apr 02 14:55:08 smithi113 bash[72708]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-b 2024-04-02T14:55:08.619 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.b.service' 2024-04-02T14:55:08.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:55:08.664 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-02T14:55:08.664 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-02T14:55:08.664 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.c 2024-04-02T14:55:08.968 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:08 smithi204 systemd[1]: Stopping Ceph mon.c for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:09.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:08 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-c[35922]: 2024-04-02T14:55:08.976+0000 7f96add96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:55:09.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:08 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-c[35922]: 2024-04-02T14:55:08.976+0000 7f96add96700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-02T14:55:09.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:09 smithi204 podman[73233]: 2024-04-02 14:55:09.017860628 +0000 UTC m=+0.145146509 container died b0510f1b34f1d1fdda4289a50e68bee2308ad78afb64f049b4583daeaa27f678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:55:09.719 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:09 smithi204 podman[73233]: 2024-04-02 14:55:09.40230562 +0000 UTC m=+0.529591489 container cleanup b0510f1b34f1d1fdda4289a50e68bee2308ad78afb64f049b4583daeaa27f678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-02T14:55:09.719 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 02 14:55:09 smithi204 bash[73233]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mon-c 2024-04-02T14:55:09.877 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mon.c.service' 2024-04-02T14:55:09.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:55:09.925 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-02T14:55:09.925 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-02T14:55:09.925 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.x 2024-04-02T14:55:10.334 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:55:10 smithi062 systemd[1]: Stopping Ceph mgr.x for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:11.003 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:55:10 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:55:10.596+0000 7f0d00c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:55:11.003 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:55:10 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x[32346]: 2024-04-02T14:55:10.596+0000 7f0d00c00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-02T14:55:11.003 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:55:10 smithi062 podman[93510]: 2024-04-02 14:55:10.69044372 +0000 UTC m=+0.287182492 container died 4e5db03de101204795db3714573125b818cd25ff55604c51c55fe0fb7d772d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-04-02T14:55:11.333 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:55:11 smithi062 podman[93510]: 2024-04-02 14:55:11.221010639 +0000 UTC m=+0.817749374 container cleanup 4e5db03de101204795db3714573125b818cd25ff55604c51c55fe0fb7d772d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:55:11.334 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:55:11 smithi062 bash[93510]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x 2024-04-02T14:55:12.001 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Apr 02 14:55:11 smithi062 podman[93524]: 2024-04-02 14:55:11.680972684 +0000 UTC m=+0.974075748 container remove 4e5db03de101204795db3714573125b818cd25ff55604c51c55fe0fb7d772d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-x, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:55:12.033 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.x.service' 2024-04-02T14:55:12.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:55:12.153 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-02T14:55:12.153 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-02T14:55:12.154 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.y 2024-04-02T14:55:12.491 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:55:12 smithi113 systemd[1]: Stopping Ceph mgr.y for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:12.804 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:55:12 smithi113 podman[72832]: 2024-04-02 14:55:12.501236942 +0000 UTC m=+0.146518744 container died 995bf8c68dae4406685885865913e207112fcd27a68154aedb0d341676209391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:55:13.111 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:55:12 smithi113 podman[72832]: 2024-04-02 14:55:12.938809294 +0000 UTC m=+0.584091103 container cleanup 995bf8c68dae4406685885865913e207112fcd27a68154aedb0d341676209391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-02T14:55:13.111 INFO:journalctl@ceph.mgr.y.smithi113.stdout:Apr 02 14:55:12 smithi113 bash[72832]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-y 2024-04-02T14:55:13.333 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.y.service' 2024-04-02T14:55:13.370 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:55:13.370 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-02T14:55:13.370 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-02T14:55:13.370 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.z 2024-04-02T14:55:13.709 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:55:13 smithi204 systemd[1]: Stopping Ceph mgr.z for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:14.000 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:55:13 smithi204 podman[73366]: 2024-04-02 14:55:13.719201212 +0000 UTC m=+0.145182081 container died a846a5f3b4b88121e598a1ec70578d1af8ad4d12a60691545ce39c0bcc374ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-04-02T14:55:14.478 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:55:14 smithi204 podman[73366]: 2024-04-02 14:55:14.03992087 +0000 UTC m=+0.465901731 container cleanup a846a5f3b4b88121e598a1ec70578d1af8ad4d12a60691545ce39c0bcc374ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-02T14:55:14.478 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:55:14 smithi204 bash[73366]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z 2024-04-02T14:55:14.478 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:55:14 smithi204 podman[73380]: 2024-04-02 14:55:14.266608672 +0000 UTC m=+0.539613917 container remove a846a5f3b4b88121e598a1ec70578d1af8ad4d12a60691545ce39c0bcc374ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mgr-z, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:55:14.478 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 02 14:55:14 smithi204 systemd[1]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-02T14:55:14.497 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mgr.z.service' 2024-04-02T14:55:14.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:55:14.534 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-02T14:55:14.535 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-02T14:55:14.535 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.0 2024-04-02T14:55:15.084 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:14 smithi062 systemd[1]: Stopping Ceph osd.0 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:15.584 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:15 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0[44630]: 2024-04-02T14:55:15.157+0000 7f11abe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:55:15.584 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:15 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0[44630]: 2024-04-02T14:55:15.157+0000 7f11abe00700 -1 osd.0 102 *** Got signal Terminated *** 2024-04-02T14:55:15.584 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:15 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0[44630]: 2024-04-02T14:55:15.157+0000 7f11abe00700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:55:20.895 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:20 smithi062 podman[93631]: 2024-04-02 14:55:20.637103783 +0000 UTC m=+5.647462788 container died 815503bf6b67b8635b8df02ef24284d4d11439c5c03827d4849a138c443ffed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-02T14:55:21.314 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:21 smithi062 podman[93631]: 2024-04-02 14:55:21.108904084 +0000 UTC m=+6.119263066 container cleanup 815503bf6b67b8635b8df02ef24284d4d11439c5c03827d4849a138c443ffed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:55:21.315 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:21 smithi062 bash[93631]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0 2024-04-02T14:55:21.584 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:21 smithi062 podman[93651]: 2024-04-02 14:55:21.323086112 +0000 UTC m=+0.667100640 container remove 815503bf6b67b8635b8df02ef24284d4d11439c5c03827d4849a138c443ffed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:55:22.539 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:22 smithi062 podman[93740]: 2024-04-02 14:55:22.080943968 +0000 UTC m=+0.037664929 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:55:22.539 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:22 smithi062 podman[93740]: 2024-04-02 14:55:22.252260227 +0000 UTC m=+0.208981197 container create 14b53d7cb39e25632546ca91c85899c94fb30fce4f3f9a546300f5da01a8d38d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.vendor=CentOS) 2024-04-02T14:55:22.539 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:22 smithi062 podman[93740]: 2024-04-02 14:55:22.547747214 +0000 UTC m=+0.504468188 container init 14b53d7cb39e25632546ca91c85899c94fb30fce4f3f9a546300f5da01a8d38d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-02T14:55:22.834 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:22 smithi062 podman[93740]: 2024-04-02 14:55:22.559395628 +0000 UTC m=+0.516116585 container start 14b53d7cb39e25632546ca91c85899c94fb30fce4f3f9a546300f5da01a8d38d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:55:22.834 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:22 smithi062 podman[93740]: 2024-04-02 14:55:22.64653793 +0000 UTC m=+0.603258957 container attach 14b53d7cb39e25632546ca91c85899c94fb30fce4f3f9a546300f5da01a8d38d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-02T14:55:23.334 INFO:journalctl@ceph.osd.0.smithi062.stdout:Apr 02 14:55:23 smithi062 podman[93740]: 2024-04-02 14:55:23.016416771 +0000 UTC m=+0.973137768 container died 14b53d7cb39e25632546ca91c85899c94fb30fce4f3f9a546300f5da01a8d38d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T14:55:23.729 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.0.service' 2024-04-02T14:55:23.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:55:23.862 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-02T14:55:23.862 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-02T14:55:23.862 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.1 2024-04-02T14:55:24.151 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:23 smithi113 systemd[1]: Stopping Ceph osd.1 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:24.611 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:24 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1[40719]: 2024-04-02T14:55:24.159+0000 7fb47ea67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:55:24.611 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:24 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1[40719]: 2024-04-02T14:55:24.159+0000 7fb47ea67700 -1 osd.1 102 *** Got signal Terminated *** 2024-04-02T14:55:24.611 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:24 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1[40719]: 2024-04-02T14:55:24.159+0000 7fb47ea67700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:55:29.861 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:29 smithi113 podman[72967]: 2024-04-02 14:55:29.568000495 +0000 UTC m=+5.512525108 container died daceecafcb8e599d601a13c9bae6a30702990e4abe3fa8945ce358d1be06aa03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:55:30.210 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:30 smithi113 podman[72967]: 2024-04-02 14:55:30.009692057 +0000 UTC m=+5.954216671 container cleanup daceecafcb8e599d601a13c9bae6a30702990e4abe3fa8945ce358d1be06aa03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326) 2024-04-02T14:55:30.210 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:30 smithi113 bash[72967]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1 2024-04-02T14:55:30.601 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:30 smithi113 podman[72984]: 2024-04-02 14:55:30.219463186 +0000 UTC m=+0.649744816 container remove daceecafcb8e599d601a13c9bae6a30702990e4abe3fa8945ce358d1be06aa03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:55:30.861 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:30 smithi113 podman[73075]: 2024-04-02 14:55:30.514792665 +0000 UTC m=+0.017760262 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:55:30.861 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:30 smithi113 podman[73075]: 2024-04-02 14:55:30.653603328 +0000 UTC m=+0.156570914 container create 8cd038dc110a11b08f4137190362c482ad10fce2597f85da2382d0822be2a21f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-02T14:55:31.153 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:30 smithi113 podman[73075]: 2024-04-02 14:55:30.873002337 +0000 UTC m=+0.375969945 container init 8cd038dc110a11b08f4137190362c482ad10fce2597f85da2382d0822be2a21f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-02T14:55:31.153 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:30 smithi113 podman[73075]: 2024-04-02 14:55:30.876063891 +0000 UTC m=+0.379031489 container start 8cd038dc110a11b08f4137190362c482ad10fce2597f85da2382d0822be2a21f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-04-02T14:55:31.153 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:30 smithi113 podman[73075]: 2024-04-02 14:55:30.948038981 +0000 UTC m=+0.451006602 container attach 8cd038dc110a11b08f4137190362c482ad10fce2597f85da2382d0822be2a21f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=) 2024-04-02T14:55:31.441 INFO:journalctl@ceph.osd.1.smithi113.stdout:Apr 02 14:55:31 smithi113 podman[73075]: 2024-04-02 14:55:31.162800711 +0000 UTC m=+0.665768301 container died 8cd038dc110a11b08f4137190362c482ad10fce2597f85da2382d0822be2a21f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-02T14:55:31.783 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.1.service' 2024-04-02T14:55:31.826 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:55:31.826 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-02T14:55:31.826 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-02T14:55:31.827 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.2 2024-04-02T14:55:32.176 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:31 smithi204 systemd[1]: Stopping Ceph osd.2 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:32.500 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:32 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2[41105]: 2024-04-02T14:55:32.185+0000 7fbe60195700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:55:32.500 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:32 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2[41105]: 2024-04-02T14:55:32.185+0000 7fbe60195700 -1 osd.2 102 *** Got signal Terminated *** 2024-04-02T14:55:32.500 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:32 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2[41105]: 2024-04-02T14:55:32.185+0000 7fbe60195700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:55:37.793 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:37 smithi204 podman[73489]: 2024-04-02 14:55:37.52781225 +0000 UTC m=+5.495737534 container died 417c14fb1150650c2d5f72d8b6d18539bec9812af3a4eb50c25f81114f1cf983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:55:38.169 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:37 smithi204 podman[73489]: 2024-04-02 14:55:37.894409848 +0000 UTC m=+5.862335116 container cleanup 417c14fb1150650c2d5f72d8b6d18539bec9812af3a4eb50c25f81114f1cf983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-02T14:55:38.169 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:37 smithi204 bash[73489]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2 2024-04-02T14:55:38.487 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:38 smithi204 podman[73506]: 2024-04-02 14:55:38.179029541 +0000 UTC m=+0.641364804 container remove 417c14fb1150650c2d5f72d8b6d18539bec9812af3a4eb50c25f81114f1cf983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-02T14:55:38.750 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:38 smithi204 podman[73600]: 2024-04-02 14:55:38.400131233 +0000 UTC m=+0.017889003 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:55:38.750 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:38 smithi204 podman[73600]: 2024-04-02 14:55:38.538871997 +0000 UTC m=+0.156629779 container create cb8f2ea9fd9a6c2c6bfeb1cf9917de41eeffa521bb89b6eb3aefc37b11e5e482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-04-02T14:55:38.750 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:38 smithi204 podman[73600]: 2024-04-02 14:55:38.741143521 +0000 UTC m=+0.358901291 container init cb8f2ea9fd9a6c2c6bfeb1cf9917de41eeffa521bb89b6eb3aefc37b11e5e482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux ) 2024-04-02T14:55:38.750 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:38 smithi204 podman[73600]: 2024-04-02 14:55:38.744411212 +0000 UTC m=+0.362168975 container start cb8f2ea9fd9a6c2c6bfeb1cf9917de41eeffa521bb89b6eb3aefc37b11e5e482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T14:55:39.029 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:38 smithi204 podman[73600]: 2024-04-02 14:55:38.858069537 +0000 UTC m=+0.475827320 container attach cb8f2ea9fd9a6c2c6bfeb1cf9917de41eeffa521bb89b6eb3aefc37b11e5e482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326) 2024-04-02T14:55:39.306 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 02 14:55:39 smithi204 podman[73600]: 2024-04-02 14:55:39.039098492 +0000 UTC m=+0.656856269 container died cb8f2ea9fd9a6c2c6bfeb1cf9917de41eeffa521bb89b6eb3aefc37b11e5e482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326) 2024-04-02T14:55:39.683 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.2.service' 2024-04-02T14:55:39.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:55:39.725 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-02T14:55:39.725 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-02T14:55:39.725 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.3 2024-04-02T14:55:40.084 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:39 smithi062 systemd[1]: Stopping Ceph osd.3 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:40.783 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:40 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3[51591]: 2024-04-02T14:55:40.357+0000 7fc2ace00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:55:40.783 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:40 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3[51591]: 2024-04-02T14:55:40.357+0000 7fc2ace00700 -1 osd.3 102 *** Got signal Terminated *** 2024-04-02T14:55:40.784 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:40 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3[51591]: 2024-04-02T14:55:40.357+0000 7fc2ace00700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:55:41.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:41 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:41.473+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:42.334 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:41 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:41.987+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:42.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:42 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:42.114+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:43.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:42.515+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:43.333 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:43 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:43.032+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:43.366 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:43 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:43.071+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:43.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:43.469+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:44.333 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:44 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:44.026+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:44.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:44.042+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:44.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:44 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:44.478+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:44.750 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:44 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:44.393+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:45.333 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:45 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:45.043+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:45.360 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:45 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:45.030+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:45.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:45.473+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:45.750 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:45.353+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:46.189 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:46 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:46.043+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:46.190 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:45 smithi062 podman[93843]: 2024-04-02 14:55:45.860247623 +0000 UTC m=+5.642152216 container died 520201df43105a392c177de29f2d2f4950ed94f046f6453fbabfed4b3708b5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-02T14:55:46.190 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:45 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:45.865+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:46.360 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:46 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:46.078+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:46.584 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:46 smithi062 podman[93843]: 2024-04-02 14:55:46.350276044 +0000 UTC m=+6.132180674 container cleanup 520201df43105a392c177de29f2d2f4950ed94f046f6453fbabfed4b3708b5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:55:46.584 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:46 smithi062 bash[93843]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3 2024-04-02T14:55:46.750 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:46 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:46.314+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:46.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:46 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:46.518+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:46.853 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:46 smithi062 podman[93874]: 2024-04-02 14:55:46.719677049 +0000 UTC m=+0.844046218 container remove 520201df43105a392c177de29f2d2f4950ed94f046f6453fbabfed4b3708b5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-02T14:55:47.301 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:47 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:47.065+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:47.301 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:46 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:46.860+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:47.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:47 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:47.041+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:47.551 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:47.285+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:47.584 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:47 smithi062 podman[93960]: 2024-04-02 14:55:47.216798909 +0000 UTC m=+0.034675393 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:55:47.584 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:47 smithi062 podman[93960]: 2024-04-02 14:55:47.343856017 +0000 UTC m=+0.161732530 container create 8bf2c4bba8f784fc7d1905f51cceb5a8e2fbff8ca7bfbfde90f19ba803895300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:55:47.999 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:47.561+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:48.012 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:47 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:47.827+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:48.013 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:47 smithi062 podman[93960]: 2024-04-02 14:55:47.65624159 +0000 UTC m=+0.474118123 container init 8bf2c4bba8f784fc7d1905f51cceb5a8e2fbff8ca7bfbfde90f19ba803895300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:55:48.013 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:47 smithi062 podman[93960]: 2024-04-02 14:55:47.668232113 +0000 UTC m=+0.486108632 container start 8bf2c4bba8f784fc7d1905f51cceb5a8e2fbff8ca7bfbfde90f19ba803895300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:55:48.013 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:47 smithi062 podman[93960]: 2024-04-02 14:55:47.746258771 +0000 UTC m=+0.564135327 container attach 8bf2c4bba8f784fc7d1905f51cceb5a8e2fbff8ca7bfbfde90f19ba803895300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326) 2024-04-02T14:55:48.334 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:48 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:48.021+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:48.334 INFO:journalctl@ceph.osd.3.smithi062.stdout:Apr 02 14:55:48 smithi062 podman[93960]: 2024-04-02 14:55:48.121283485 +0000 UTC m=+0.939160005 container died 8bf2c4bba8f784fc7d1905f51cceb5a8e2fbff8ca7bfbfde90f19ba803895300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:55:48.360 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:48 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:48.088+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:48.500 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:48 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:48.239+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:48.895 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.3.service' 2024-04-02T14:55:49.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:48 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:48.546+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:49.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:55:49.024 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-02T14:55:49.024 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-02T14:55:49.024 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.4 2024-04-02T14:55:49.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:48 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:48.993+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:49.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:48 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:48.789+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:49.343 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:49 smithi113 systemd[1]: Stopping Ceph osd.4 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:49.344 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:49 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:49.122+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:49.524 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:49 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:49.271+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:49.611 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:49 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4[47700]: 2024-04-02T14:55:49.352+0000 7f72c8a5c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:55:49.611 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:49 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4[47700]: 2024-04-02T14:55:49.352+0000 7f72c8a5c700 -1 osd.4 102 *** Got signal Terminated *** 2024-04-02T14:55:49.611 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:49 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4[47700]: 2024-04-02T14:55:49.352+0000 7f72c8a5c700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:55:50.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:49 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:49.533+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:50.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:49.978+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:50.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:49.834+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:50.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:49 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:49.834+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:55:50.360 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:50 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:50.106+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:50.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:55:50 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:55:50.279+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:55:50.750 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:50 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:50.301+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:50.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:50 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:50.489+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:51.086 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:50 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:50.970+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:51.086 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:50 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:50.786+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:51.086 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:50 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:50.786+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:55:51.360 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:51 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:51.110+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:51.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:55:51 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:55:51.235+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:55:51.545 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:51.304+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:51.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:51.493+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:51.861 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:51 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4[47700]: 2024-04-02T14:55:51.427+0000 7f72c1048700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.682439+0000 front 2024-04-02T14:55:25.682418+0000 (oldest deadline 2024-04-02T14:55:50.982584+0000) 2024-04-02T14:55:52.000 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:55:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:55:51.555+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:55:52.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:51 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:51.938+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:52.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:51 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:51.938+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:55:52.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:51 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:51.786+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:52.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:51 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:51.786+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:55:52.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:55:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:55:52.226+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:55:52.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:52.097+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:52.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:52.097+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:55:52.750 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:52.280+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:52.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:55:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:55:52.525+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:55:52.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:52.478+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:52.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:52 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:52.478+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:55:52.861 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:52 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4[47700]: 2024-04-02T14:55:52.410+0000 7f72c1048700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.682439+0000 front 2024-04-02T14:55:25.682418+0000 (oldest deadline 2024-04-02T14:55:50.982584+0000) 2024-04-02T14:55:53.083 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:52 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:52.810+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:53.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:52 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:52.810+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:55:53.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:52 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:52.931+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:53.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:52 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:52.931+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:55:53.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:55:53 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:55:53.215+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:55:53.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:53 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:53.099+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:53.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:53 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:53.099+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:55:53.750 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:53 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:53.305+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:53.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:55:53 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:55:53.476+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:55:53.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:53 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:53.464+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:53.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:53 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:53.464+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:55:53.861 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:53 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4[47700]: 2024-04-02T14:55:53.442+0000 7f72c1048700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.682439+0000 front 2024-04-02T14:55:25.682418+0000 (oldest deadline 2024-04-02T14:55:50.982584+0000) 2024-04-02T14:55:54.333 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:53 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:53.848+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:54.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:53 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:53.848+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:55:54.334 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:53 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:53.950+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:54.334 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:53 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:53.950+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:55:54.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:55:54 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:55:54.242+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:55:54.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:54 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:54.112+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:54.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:54 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:54.112+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:55:54.500 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:54 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:54.258+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:54.500 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:55:54 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:55:54.506+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:55:54.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:54 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:54.503+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:54.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:54 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:54.503+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:55:54.992 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:54 smithi113 podman[73178]: 2024-04-02 14:55:54.725741797 +0000 UTC m=+5.501140983 container died edb50252c1567bb61f9558d4c8b50186873ff7260f4b4668de2121ca421bf33a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-02T14:55:55.333 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:54 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:54.936+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:55.334 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:54 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:54.936+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:55:55.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:54 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:54.874+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:55.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:54 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:54.874+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:55:55.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:55:55 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:55:55.202+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:55:55.361 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:55 smithi113 podman[73178]: 2024-04-02 14:55:55.002744538 +0000 UTC m=+5.778143693 container cleanup edb50252c1567bb61f9558d4c8b50186873ff7260f4b4668de2121ca421bf33a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-02T14:55:55.361 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:55 smithi113 bash[73178]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4 2024-04-02T14:55:55.361 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:55 smithi113 podman[73199]: 2024-04-02 14:55:55.237415529 +0000 UTC m=+0.510057705 container remove edb50252c1567bb61f9558d4c8b50186873ff7260f4b4668de2121ca421bf33a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True) 2024-04-02T14:55:55.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:55 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:55.133+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:55.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:55 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:55.133+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:55:55.500 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:55 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:55.242+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:55.500 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:55 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:55.242+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:28.755793+0000 front 2024-04-02T14:55:28.755842+0000 (oldest deadline 2024-04-02T14:55:54.655798+0000) 2024-04-02T14:55:55.861 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:55 smithi113 podman[73292]: 2024-04-02 14:55:55.461413788 +0000 UTC m=+0.017474382 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:55:55.862 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:55 smithi113 podman[73292]: 2024-04-02 14:55:55.563374033 +0000 UTC m=+0.119434631 container create e65953820650a0bf4e4ef1a0d48e3a5e9df2a60e5201a633038146efb1115968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-02T14:55:55.862 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:55 smithi113 podman[73292]: 2024-04-02 14:55:55.799339082 +0000 UTC m=+0.355399662 container init e65953820650a0bf4e4ef1a0d48e3a5e9df2a60e5201a633038146efb1115968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:55:55.862 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:55 smithi113 podman[73292]: 2024-04-02 14:55:55.802549494 +0000 UTC m=+0.358610068 container start e65953820650a0bf4e4ef1a0d48e3a5e9df2a60e5201a633038146efb1115968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:55:56.000 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:55:55 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:55:55.545+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:55:56.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:55 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:55.549+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:56.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:55 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:55.549+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:55:56.213 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:56 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:56.120+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:56.213 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:56 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:56.120+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:55:56.214 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:55 smithi113 podman[73292]: 2024-04-02 14:55:55.874108443 +0000 UTC m=+0.430169042 container attach e65953820650a0bf4e4ef1a0d48e3a5e9df2a60e5201a633038146efb1115968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5) 2024-04-02T14:55:56.214 INFO:journalctl@ceph.osd.4.smithi113.stdout:Apr 02 14:55:56 smithi113 podman[73292]: 2024-04-02 14:55:56.091294256 +0000 UTC m=+0.647354835 container died e65953820650a0bf4e4ef1a0d48e3a5e9df2a60e5201a633038146efb1115968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-04-02T14:55:56.333 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:55 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:55.922+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:56.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:55 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:55.922+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:55:56.334 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:55 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:55.888+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:56.334 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:55 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:55.888+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:55:56.496 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:56 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:56.209+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:56.496 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:56 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:56.209+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:28.755793+0000 front 2024-04-02T14:55:28.755842+0000 (oldest deadline 2024-04-02T14:55:54.655798+0000) 2024-04-02T14:55:56.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:55:56 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:55:56.223+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:55:56.641 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.4.service' 2024-04-02T14:55:56.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:55:56.682 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-02T14:55:56.682 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-02T14:55:56.682 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.5 2024-04-02T14:55:56.750 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:56 smithi204 systemd[1]: Stopping Ceph osd.5 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:55:56.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:55:56 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:55:56.559+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:55:56.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:56 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:56.505+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:56.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:56 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:56.505+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:55:57.250 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:56 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:56.980+0000 7fde388f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:55:57.250 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:56 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:56.980+0000 7fde388f4700 -1 osd.5 102 *** Got signal Terminated *** 2024-04-02T14:55:57.250 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:56 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:56.980+0000 7fde388f4700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:55:57.250 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:57 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:57.206+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:57.250 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:57 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:57.206+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:28.755793+0000 front 2024-04-02T14:55:28.755842+0000 (oldest deadline 2024-04-02T14:55:54.655798+0000) 2024-04-02T14:55:57.333 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:56 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:56.854+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:57.334 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:56 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:56.854+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:55:57.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:56 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:56.915+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:57.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:56 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:56.915+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:55:57.360 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:55:57 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:55:57.210+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:55:57.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:57 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:57.102+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:57.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:57 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:57.102+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:55:57.749 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:55:57 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:55:57.567+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:55:57.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:57 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:57.456+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:57.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:57 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:57.456+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:55:58.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:57 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:57.832+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:58.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:57 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:57.832+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:55:58.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:57 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:57.905+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:58.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:57 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:57.905+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:55:58.500 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:58 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:58.218+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:58.500 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:58 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:58.218+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:28.755793+0000 front 2024-04-02T14:55:28.755842+0000 (oldest deadline 2024-04-02T14:55:54.655798+0000) 2024-04-02T14:55:58.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:58 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:58.417+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:58.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:58 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:58.417+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:55:58.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:55:58 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:55:58.197+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:55:58.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:58 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:58.142+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:58.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:58 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:58.142+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:55:59.000 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:55:58 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:55:58.618+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:55:59.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:58 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:58.796+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:55:59.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:58 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:58.796+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:55:59.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:58 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:58.870+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:55:59.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:58 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:58.870+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:55:59.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:58 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:58.870+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:55:59.581 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:59 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:59.420+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:55:59.581 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:55:59 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:55:59.420+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:55:59.581 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:59 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:59.263+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:55:59.581 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:59 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:59.263+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:28.755793+0000 front 2024-04-02T14:55:28.755842+0000 (oldest deadline 2024-04-02T14:55:54.655798+0000) 2024-04-02T14:55:59.581 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:55:59 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:55:59.263+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.656155+0000 front 2024-04-02T14:55:34.656172+0000 (oldest deadline 2024-04-02T14:55:58.756321+0000) 2024-04-02T14:55:59.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:55:59 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:55:59.198+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:55:59.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:59 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:59.129+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:55:59.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:59 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:59.129+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:55:59.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:55:59 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:55:59.129+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:55:59.999 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:55:59 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:55:59.590+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:00.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:59 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:59.789+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:00.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:55:59 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:55:59.789+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:00.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:59 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:59.908+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:56:00.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:59 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:59.908+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:56:00.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:55:59 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:55:59.908+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:56:00.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:00 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:00.459+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:00.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:00 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:00.459+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:00.501 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:00 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:56:00.214+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:56:00.501 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:00 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:56:00.214+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:28.755793+0000 front 2024-04-02T14:55:28.755842+0000 (oldest deadline 2024-04-02T14:55:54.655798+0000) 2024-04-02T14:56:00.501 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:00 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:56:00.214+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.656155+0000 front 2024-04-02T14:55:34.656172+0000 (oldest deadline 2024-04-02T14:55:58.756321+0000) 2024-04-02T14:56:00.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:00 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:00.188+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:00.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:00 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:00.188+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:00.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:00 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:00.168+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:00.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:00 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:00.168+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:00.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:00 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:00.168+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:00.999 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:00 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:00.599+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:01.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:00.815+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:01.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:00.815+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:01.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:00.868+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:56:01.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:00.868+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:56:01.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:00 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:00.868+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:56:01.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:01.198+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:01.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:01.198+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:01.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:01.119+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:01.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:01.119+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:01.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:01 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:01.119+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:01.499 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:56:01.247+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:20.054293+0000 front 2024-04-02T14:55:20.054387+0000 (oldest deadline 2024-04-02T14:55:44.154173+0000) 2024-04-02T14:56:01.500 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:56:01.247+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:28.755793+0000 front 2024-04-02T14:55:28.755842+0000 (oldest deadline 2024-04-02T14:55:54.655798+0000) 2024-04-02T14:56:01.500 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5[47952]: 2024-04-02T14:56:01.247+0000 7fde34ee8700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.656155+0000 front 2024-04-02T14:55:34.656172+0000 (oldest deadline 2024-04-02T14:55:58.756321+0000) 2024-04-02T14:56:01.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:01.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:01.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:01.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:01.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:01.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:02.000 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:01.645+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:02.000 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:01 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:01.645+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:02.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:01.889+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:56:02.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:01.889+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:56:02.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:01.889+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:56:02.335 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:01.858+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:02.346 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:01.858+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:02.346 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:01 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:01.858+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:02.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:02 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:02.191+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:02.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:02 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:02.191+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:02.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:02 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:02.105+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:02.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:02 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:02.105+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:02.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:02 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:02.105+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:02.750 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:02 smithi204 podman[73716]: 2024-04-02 14:56:02.517244492 +0000 UTC m=+5.626652336 container died a899c2bd035a0921eb56a25f6b7dc60b5af4b44a29461ab9b6c3f03b9bd47d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:56:02.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:02 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:02.597+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:02.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:02 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:02.597+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:02.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:02 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:02.408+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:02.751 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:02 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:02.408+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:02.751 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:02 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:02.408+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:03.139 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:02 smithi204 podman[73716]: 2024-04-02 14:56:02.881125884 +0000 UTC m=+5.990533701 container cleanup a899c2bd035a0921eb56a25f6b7dc60b5af4b44a29461ab9b6c3f03b9bd47d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-02T14:56:03.139 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:02 smithi204 bash[73716]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5 2024-04-02T14:56:03.333 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:02 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:02.929+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:56:03.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:02 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:02.929+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:56:03.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:02 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:02.929+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:56:03.334 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:02 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:02.849+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:03.334 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:02 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:02.849+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:03.334 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:02 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:02.849+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:03.360 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:03.232+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:03.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:03.232+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:03.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:03.113+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:03.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:03.113+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:03.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:03 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:03.113+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:03.487 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:03 smithi204 podman[73733]: 2024-04-02 14:56:03.148833181 +0000 UTC m=+0.623990739 container remove a899c2bd035a0921eb56a25f6b7dc60b5af4b44a29461ab9b6c3f03b9bd47d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:56:03.487 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:03 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:03.384+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:03.487 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:03 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:03.384+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:03.487 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:03 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:03.384+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:03.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:03 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:03.640+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:03.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:03 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:03.640+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:03.750 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:03 smithi204 podman[73828]: 2024-04-02 14:56:03.399881309 +0000 UTC m=+0.017815509 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:56:03.750 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:03 smithi204 podman[73828]: 2024-04-02 14:56:03.499650053 +0000 UTC m=+0.117584243 container create 625257fd1eeac4aa46405e27160c935d5162864c1a0d2b121059b3bda0dd02cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-02T14:56:03.750 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:03 smithi204 podman[73828]: 2024-04-02 14:56:03.743610549 +0000 UTC m=+0.361544744 container init 625257fd1eeac4aa46405e27160c935d5162864c1a0d2b121059b3bda0dd02cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-02T14:56:03.751 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:03 smithi204 podman[73828]: 2024-04-02 14:56:03.746762705 +0000 UTC m=+0.364696897 container start 625257fd1eeac4aa46405e27160c935d5162864c1a0d2b121059b3bda0dd02cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:56:04.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:03.802+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:04.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:03.802+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:04.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:03.802+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:04.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:03.959+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:56:04.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:03.959+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:56:04.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:03.959+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:56:04.250 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:03 smithi204 podman[73828]: 2024-04-02 14:56:03.852501186 +0000 UTC m=+0.470435383 container attach 625257fd1eeac4aa46405e27160c935d5162864c1a0d2b121059b3bda0dd02cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-02T14:56:04.250 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 02 14:56:04 smithi204 podman[73828]: 2024-04-02 14:56:04.038497823 +0000 UTC m=+0.656432021 container died 625257fd1eeac4aa46405e27160c935d5162864c1a0d2b121059b3bda0dd02cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-02T14:56:04.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:04 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:04.278+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:04.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:04 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:04.278+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:04.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:04 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:04.152+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:04.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:04 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:04.152+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:04.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:04 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:04.152+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:04.630 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:04 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:04.591+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:04.630 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:04 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:04.591+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:04.630 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:04 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:04.403+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:04.630 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:04 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:04.403+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:04.630 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:04 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:04.403+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:04.644 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.5.service' 2024-04-02T14:56:04.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:56:04.685 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-02T14:56:04.685 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-02T14:56:04.685 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.6 2024-04-02T14:56:05.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:04 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:04.762+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:05.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:04 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:04.762+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:05.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:04 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:04.762+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:05.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:04 smithi062 systemd[1]: Stopping Ceph osd.6 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:56:05.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:04 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:04.950+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:56:05.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:04 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:04.950+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:56:05.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:04 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:04.950+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:56:05.583 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:05 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:05.272+0000 7fca5c400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:56:05.584 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:05 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:05.272+0000 7fca5c400700 -1 osd.6 102 *** Got signal Terminated *** 2024-04-02T14:56:05.584 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:05 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:05.272+0000 7fca5c400700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:56:05.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:05 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:05.273+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:05.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:05 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:05.273+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:05.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:05 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:05.127+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:05.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:05 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:05.127+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:05.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:05 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:05.127+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:05.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:05 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:05.553+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:05.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:05 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:05.553+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:05.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:05 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:05.429+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:05.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:05 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:05.429+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:05.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:05 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:05.429+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:06.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:05 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:05.941+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:56:06.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:05 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:05.941+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:56:06.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:05 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:05.941+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:56:06.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:05 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:05.783+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:06.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:05 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:05.783+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:06.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:05 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:05.783+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:06.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:06 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:06.251+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:06.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:06 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:06.251+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:06.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:06 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:06.251+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:06.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:06 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:06.124+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:06.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:06 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:06.124+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:06.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:06 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:06.124+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:06.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:06 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:06.583+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:06.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:06 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:06.583+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:06.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:06 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:06.406+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:06.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:06 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:06.406+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:06.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:06 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:06.406+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:07.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:06 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:06.896+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:56:07.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:06 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:06.896+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:56:07.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:06 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:06.896+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:56:07.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:06 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:06.896+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:42.619363+0000 front 2024-04-02T14:55:42.619427+0000 (oldest deadline 2024-04-02T14:56:06.119660+0000) 2024-04-02T14:56:07.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:06 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:06.790+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:07.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:06 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:06.790+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:07.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:06 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:06.790+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:07.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:06 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:06.790+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:07.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:07 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:07.233+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:07.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:07 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:07.233+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:07.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:07 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:07.233+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:07.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:07 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:07.174+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:07.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:07 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:07.174+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:07.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:07 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:07.174+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:07.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:07 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:07.174+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:07.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:07 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:07.613+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:07.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:07 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:07.613+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:07.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:07 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:07.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:07.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:07 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:07.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:07.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:07 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:07.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:08.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:07.934+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:56:08.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:07.934+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:56:08.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:07.934+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:56:08.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:07.934+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:42.619363+0000 front 2024-04-02T14:55:42.619427+0000 (oldest deadline 2024-04-02T14:56:06.119660+0000) 2024-04-02T14:56:08.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:07.749+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:08.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:07.749+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:08.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:07.749+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:08.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:07.749+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:08.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:08 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:08.198+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:08.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:08 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:08.198+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:08.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:08 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:08.198+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:08.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:08 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:08.148+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:08.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:08 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:08.148+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:08.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:08 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:08.148+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:08.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:08 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:08.148+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:08.749 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:08 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:08.620+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:08.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:08 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:08.620+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:08.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:08 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:08.457+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:08.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:08 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:08.457+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:08.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:08 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:08.457+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:09.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:08 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:08.909+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:56:09.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:08 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:08.909+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:56:09.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:08 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:08.909+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:56:09.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:08 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:08.909+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:42.619363+0000 front 2024-04-02T14:55:42.619427+0000 (oldest deadline 2024-04-02T14:56:06.119660+0000) 2024-04-02T14:56:09.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:08 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:08.711+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:09.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:08 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:08.711+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:09.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:08 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:08.711+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:09.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:08 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:08.711+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:09.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:09 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:09.226+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:09.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:09 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:09.226+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:09.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:09 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:09.226+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:09.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:09 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:09.141+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:09.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:09 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:09.141+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:09.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:09 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:09.141+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:09.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:09 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:09.141+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:09.749 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:09 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:09.602+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:09.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:09 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:09.602+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:09.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:09 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:09.463+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:09.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:09 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:09.463+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:09.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:09 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:09.463+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:10.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:09 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:09.866+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.713839+0000 front 2024-04-02T14:55:19.714121+0000 (oldest deadline 2024-04-02T14:55:45.013759+0000) 2024-04-02T14:56:10.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:09 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:09.866+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.015015+0000 front 2024-04-02T14:55:25.015081+0000 (oldest deadline 2024-04-02T14:55:49.715262+0000) 2024-04-02T14:56:10.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:09 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:09.866+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.617500+0000 front 2024-04-02T14:55:35.617417+0000 (oldest deadline 2024-04-02T14:55:57.917353+0000) 2024-04-02T14:56:10.084 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:09 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6[58306]: 2024-04-02T14:56:09.866+0000 7fca52c00700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:42.619363+0000 front 2024-04-02T14:55:42.619427+0000 (oldest deadline 2024-04-02T14:56:06.119660+0000) 2024-04-02T14:56:10.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:09 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:09.723+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:10.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:09 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:09.723+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:10.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:09 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:09.723+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:10.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:09 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:09.723+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:10.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:10 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:10.223+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:10.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:10 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:10.223+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:10.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:10 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:10.223+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:10.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:10 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:10.092+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:10.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:10 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:10.092+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:10.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:10 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:10.092+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:10.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:10 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:10.092+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:10.749 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:10 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:10.564+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:10.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:10 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:10.564+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:10.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:10 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:10.564+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:10.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:10 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:10.455+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:10.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:10 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:10.455+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:10.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:10 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:10.455+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:10.937 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:10 smithi062 podman[94066]: 2024-04-02 14:56:10.647698871 +0000 UTC m=+5.514846203 container died 0b5b6f838dbd92adb8a57e8db0c418554a2a59aa29b26a98ac1168f4d797e9c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-02T14:56:10.938 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:10 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:10.701+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:10.938 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:10 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:10.701+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:10.938 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:10 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:10.701+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:10.938 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:10 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:10.702+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:11.333 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:11 smithi062 podman[94066]: 2024-04-02 14:56:11.106916251 +0000 UTC m=+5.974063550 container cleanup 0b5b6f838dbd92adb8a57e8db0c418554a2a59aa29b26a98ac1168f4d797e9c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:56:11.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:11 smithi062 bash[94066]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6 2024-04-02T14:56:11.360 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:11 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:11.219+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:11.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:11 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:11.219+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:11.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:11 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:11.219+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:11.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:11 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:11.118+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:11.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:11 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:11.118+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:11.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:11 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:11.118+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:11.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:11 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:11.118+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:11.735 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:11 smithi062 podman[94084]: 2024-04-02 14:56:11.408296975 +0000 UTC m=+0.746275330 container remove 0b5b6f838dbd92adb8a57e8db0c418554a2a59aa29b26a98ac1168f4d797e9c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:56:11.749 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:11 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:11.548+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:11.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:11 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:11.548+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:11.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:11 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:11.548+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:11.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:11 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:11.459+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:11.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:11 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:11.459+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:11.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:11 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:11.459+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:12.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:11 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:11.744+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:12.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:11 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:11.744+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:12.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:11 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:11.744+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:12.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:11 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:11.744+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:12.360 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:12 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:12.186+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:12.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:12 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:12.186+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:12.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:12 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:12.186+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:12.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:12 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:12.102+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:12.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:12 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:12.102+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:12.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:12 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:12.102+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:12.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:12 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:12.102+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:12.560 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:12 smithi062 podman[94173]: 2024-04-02 14:56:12.141901896 +0000 UTC m=+0.035266391 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:56:12.560 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:12 smithi062 podman[94173]: 2024-04-02 14:56:12.283681559 +0000 UTC m=+0.177046087 container create 281cf9e3d1cfc3a294b5d39d93be64e3116f81800c02e27c3dd7c249cd14b1e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-02T14:56:12.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:12 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:12.564+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:12.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:12 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:12.564+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:12.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:12 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:12.564+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:12.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:12 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:12.443+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:12.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:12 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:12.443+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:12.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:12 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:12.443+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:12.834 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:12 smithi062 podman[94173]: 2024-04-02 14:56:12.569831092 +0000 UTC m=+0.463195573 container init 281cf9e3d1cfc3a294b5d39d93be64e3116f81800c02e27c3dd7c249cd14b1e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-02T14:56:12.834 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:12 smithi062 podman[94173]: 2024-04-02 14:56:12.580859902 +0000 UTC m=+0.474224400 container start 281cf9e3d1cfc3a294b5d39d93be64e3116f81800c02e27c3dd7c249cd14b1e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:56:12.834 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:12 smithi062 podman[94173]: 2024-04-02 14:56:12.702620016 +0000 UTC m=+0.595984509 container attach 281cf9e3d1cfc3a294b5d39d93be64e3116f81800c02e27c3dd7c249cd14b1e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T14:56:12.834 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:12 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:12.723+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:12.834 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:12 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:12.723+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:12.834 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:12 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:12.723+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:12.834 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:12 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:12.723+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:13.334 INFO:journalctl@ceph.osd.6.smithi062.stdout:Apr 02 14:56:13 smithi062 podman[94173]: 2024-04-02 14:56:13.032016699 +0000 UTC m=+0.925381205 container died 281cf9e3d1cfc3a294b5d39d93be64e3116f81800c02e27c3dd7c249cd14b1e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-6-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-02T14:56:13.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:13 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:13.210+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:13.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:13 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:13.210+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:13.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:13 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:13.210+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:13.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:13 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:13.145+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:13.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:13 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:13.145+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:13.612 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:13 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:13.145+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:13.612 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:13 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:13.145+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:13.749 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:13 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:13.582+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:13.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:13 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:13.582+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:13.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:13 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:13.582+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:13.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:13 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:13.423+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:13.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:13 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:13.423+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:13.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:13 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:13.423+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:13.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.6.service' 2024-04-02T14:56:13.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:56:13.938 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-02T14:56:13.938 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-02T14:56:13.938 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.7 2024-04-02T14:56:14.074 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:13 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:13.704+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:14.075 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:13 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:13.705+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:14.075 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:13 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:13.705+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:14.075 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:13 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:13.705+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:14.226 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:13 smithi113 systemd[1]: Stopping Ceph osd.7 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:56:14.226 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:14.113+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:14.226 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:14.113+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:14.226 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:14.113+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:14.226 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:14.113+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:14.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:14.235+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:14.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:14.235+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:14.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:14.235+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:14.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:14.269+0000 7fe913dca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:56:14.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:14.269+0000 7fe913dca700 -1 osd.7 102 *** Got signal Terminated *** 2024-04-02T14:56:14.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:14.269+0000 7fe913dca700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:56:14.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:14 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:14.595+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:14.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:14 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:14.595+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:14.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:14 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:14.595+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:14.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:14 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:14.381+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:14.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:14 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:14.381+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:14.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:14 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:14.381+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:15.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:14 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:14.734+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:15.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:14 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:14.734+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:15.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:14 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:14.734+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:15.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:14 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:14.734+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:15.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:15 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:15.213+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:15.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:15 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:15.213+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:15.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:15 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:15.213+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:15.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:15 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:15.149+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:15.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:15 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:15.149+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:15.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:15 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:15.149+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:15.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:15 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:15.149+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:15.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:15 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:15.362+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:15.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:15 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:15.362+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:15.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:15 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:15.362+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:15.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:15 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:15.610+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:15.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:15 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:15.610+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:15.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:15 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:15.610+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:15.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:15 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:15.610+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:16.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:15 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:15.776+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:16.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:15 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:15.776+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:16.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:15 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:15.776+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:16.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:15 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:15.776+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:16.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:16 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:16.209+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:16.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:16 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:16.209+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:16.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:16 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:16.209+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:16.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:16 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:16.117+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:16.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:16 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:16.117+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:16.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:16 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:16.117+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:16.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:16 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:16.117+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:16.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:16 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:16.348+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:16.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:16 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:16.348+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:16.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:16 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:16.348+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:16.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:16 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:16.579+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:16.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:16 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:16.579+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:16.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:16 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:16.579+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:16.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:16 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:16.579+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:17.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:16 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:16.791+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:17.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:16 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:16.791+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:17.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:16 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:16.791+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:17.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:16 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:16.791+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:17.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:17 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:17.212+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:17.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:17 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:17.212+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:17.611 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:17 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:17.212+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:17.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:17 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:17.125+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:17.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:17 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:17.125+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:17.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:17 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:17.125+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:17.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:17 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:17.125+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:17.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:17 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:17.385+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:17.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:17 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:17.385+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:17.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:17 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:17.385+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:17.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:17 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:17.566+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:17.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:17 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:17.566+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:17.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:17 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:17.566+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:17.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:17 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:17.566+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:18.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:17 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:17.839+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:18.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:17 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:17.839+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:18.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:17 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:17.839+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:18.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:17 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:17.839+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:18.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:18 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:18.226+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:18.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:18 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:18.226+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:18.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:18 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:18.226+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:18.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:18 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:18.226+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.005118+0000 front 2024-04-02T14:55:52.005294+0000 (oldest deadline 2024-04-02T14:56:17.905219+0000) 2024-04-02T14:56:18.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:18 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:18.119+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:18.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:18 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:18.119+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:18.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:18 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:18.119+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:18.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:18 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:18.119+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:18.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:18 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:18.597+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:18.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:18 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:18.597+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:18.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:18 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:18.597+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:18.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:18 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:18.597+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:18.751 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:18 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:18.427+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:18.751 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:18 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:18.427+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:18.751 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:18 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:18.427+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:18.751 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:18 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:18.427+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:19.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:18 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:18.794+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:19.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:18 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:18.794+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:19.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:18 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:18.795+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:19.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:18 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:18.795+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:19.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:19 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:19.229+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.301214+0000 front 2024-04-02T14:55:26.301351+0000 (oldest deadline 2024-04-02T14:55:49.801434+0000) 2024-04-02T14:56:19.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:19 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:19.229+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:33.902669+0000 front 2024-04-02T14:55:33.902655+0000 (oldest deadline 2024-04-02T14:55:59.802508+0000) 2024-04-02T14:56:19.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:19 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:19.229+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:43.804457+0000 front 2024-04-02T14:55:43.804339+0000 (oldest deadline 2024-04-02T14:56:06.104177+0000) 2024-04-02T14:56:19.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:19 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7[54727]: 2024-04-02T14:56:19.229+0000 7fe9103be700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.005118+0000 front 2024-04-02T14:55:52.005294+0000 (oldest deadline 2024-04-02T14:56:17.905219+0000) 2024-04-02T14:56:19.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:19 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:19.114+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:19.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:19 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:19.114+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:19.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:19 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:19.114+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:19.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:19 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:19.114+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:19.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:19 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:19.445+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:19.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:19 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:19.445+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:19.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:19 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:19.445+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:19.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:19 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:19.445+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:19.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:19 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:19.646+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:19.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:19 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:19.646+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:19.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:19 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:19.646+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:19.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:19 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:19.646+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:19.861 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:19 smithi113 podman[73399]: 2024-04-02 14:56:19.589010158 +0000 UTC m=+5.451268882 container died 2947fee64d72ce4fe53f21b59e82c3b0a2510f20b9ba161958891f162474d78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-02T14:56:20.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:19 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:19.754+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:20.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:19 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:19.754+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:20.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:19 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:19.754+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:20.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:19 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:19.754+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:20.204 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:20 smithi113 podman[73399]: 2024-04-02 14:56:20.028513448 +0000 UTC m=+5.890772158 container cleanup 2947fee64d72ce4fe53f21b59e82c3b0a2510f20b9ba161958891f162474d78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-02T14:56:20.204 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:20 smithi113 bash[73399]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7 2024-04-02T14:56:20.204 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:20 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:20.081+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:20.204 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:20 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:20.081+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:20.204 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:20 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:20.081+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:20.204 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:20 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:20.081+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:20.204 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:20 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:20.081+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:20.610 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:20 smithi113 podman[73416]: 2024-04-02 14:56:20.213150258 +0000 UTC m=+0.623039925 container remove 2947fee64d72ce4fe53f21b59e82c3b0a2510f20b9ba161958891f162474d78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-02T14:56:20.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:20 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:20.601+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:20.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:20 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:20.601+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:20.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:20 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:20.601+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:20.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:20 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:20.601+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:20.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:20 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:20.455+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:20.751 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:20 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:20.455+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:20.751 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:20 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:20.455+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:20.751 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:20 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:20.455+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:20.907 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:20 smithi113 podman[73504]: 2024-04-02 14:56:20.622434013 +0000 UTC m=+0.109572506 container create 8ec248923f66e055be62f53cb3d26198e23f76b14f3854adf39a40852f88f27d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7-deactivate, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-02T14:56:20.907 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:20 smithi113 podman[73504]: 2024-04-02 14:56:20.530364871 +0000 UTC m=+0.017503370 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:56:20.907 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:20 smithi113 podman[73504]: 2024-04-02 14:56:20.833453726 +0000 UTC m=+0.320592207 container init 8ec248923f66e055be62f53cb3d26198e23f76b14f3854adf39a40852f88f27d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-02T14:56:20.907 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:20 smithi113 podman[73504]: 2024-04-02 14:56:20.836480483 +0000 UTC m=+0.323618975 container start 8ec248923f66e055be62f53cb3d26198e23f76b14f3854adf39a40852f88f27d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7-deactivate, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD) 2024-04-02T14:56:20.907 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:20 smithi113 podman[73504]: 2024-04-02 14:56:20.916624847 +0000 UTC m=+0.403763347 container attach 8ec248923f66e055be62f53cb3d26198e23f76b14f3854adf39a40852f88f27d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-02T14:56:21.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:20 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:20.729+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:21.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:20 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:20.729+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:21.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:20 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:20.729+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:21.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:20 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:20.729+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:21.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:20 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:20.729+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:21.361 INFO:journalctl@ceph.osd.7.smithi113.stdout:Apr 02 14:56:21 smithi113 podman[73504]: 2024-04-02 14:56:21.137130259 +0000 UTC m=+0.624268744 container died 8ec248923f66e055be62f53cb3d26198e23f76b14f3854adf39a40852f88f27d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:56:21.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:21 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:21.056+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:21.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:21 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:21.056+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:21.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:21 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:21.056+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:21.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:21 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:21.056+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:21.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:21 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:21.056+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:21.730 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.7.service' 2024-04-02T14:56:21.749 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:21 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:21.566+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:21.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:21 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:21.566+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:21.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:21 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:21.566+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:21.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:21 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:21.566+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:21.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:21 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:21.456+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:21.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:21 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:21.456+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:21.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:21 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:21.456+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:21.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:21 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:21.456+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:21.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:56:21.771 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-02T14:56:21.771 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-02T14:56:21.771 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.8 2024-04-02T14:56:22.082 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:21 smithi204 systemd[1]: Stopping Ceph osd.8 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:56:22.082 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:22 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:22.091+0000 7fdf5d823700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:56:22.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:21 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:21.693+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:22.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:21 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:21.693+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:22.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:21 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:21.693+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:22.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:21 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:21.693+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:22.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:21 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:21.693+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:22.330 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:22 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:22.083+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:22.330 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:22 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:22.083+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:22.330 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:22 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:22.083+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:22.331 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:22 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:22.083+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:22.331 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:22 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:22.083+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:22.424 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:22 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:22.091+0000 7fdf5d823700 -1 osd.8 102 *** Got signal Terminated *** 2024-04-02T14:56:22.424 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:22 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:22.091+0000 7fdf5d823700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:56:22.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:22 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:22.434+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:22.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:22 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:22.434+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:22.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:22 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:22.434+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:22.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:22 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:22.434+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:22.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:22 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:22.520+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:22.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:22 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:22.520+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:22.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:22 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:22.521+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:22.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:22 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:22.521+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:23.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:22 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:22.720+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:23.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:22 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:22.720+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:23.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:22 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:22.720+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:23.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:22 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:22.720+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:23.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:22 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:22.720+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:23.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:23 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:23.105+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:23.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:23 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:23.105+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:23.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:23 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:23.105+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:23.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:23 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:23.105+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:23.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:23 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:23.105+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:23.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:23 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:23.445+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:23.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:23 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:23.445+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:23.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:23 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:23.445+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:23.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:23 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:23.445+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:23.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:23 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:23.526+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:23.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:23 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:23.526+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:23.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:23 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:23.526+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:23.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:23 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:23.526+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:23.834 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:23 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:23.727+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:23.834 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:23 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:23.727+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:23.834 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:23 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:23.727+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:23.834 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:23 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:23.727+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:23.834 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:23 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:23.727+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:24.360 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:24 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:24.065+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:24.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:24 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:24.065+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:24.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:24 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:24.065+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:24.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:24 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:24.065+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:24.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:24 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:24.065+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:24.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:24 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:24.444+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:24.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:24 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:24.444+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:24.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:24 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:24.444+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:24.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:24 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:24.444+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:24.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:24 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:24.554+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:24.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:24 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:24.554+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:24.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:24 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:24.554+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:24.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:24 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:24.554+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:24.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:24 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:24.554+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.302614+0000 front 2024-04-02T14:56:00.302850+0000 (oldest deadline 2024-04-02T14:56:23.802643+0000) 2024-04-02T14:56:25.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:24 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:24.712+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:25.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:24 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:24.712+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:25.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:24 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:24.712+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:25.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:24 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:24.712+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:25.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:24 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:24.712+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:25.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:25 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:25.111+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:25.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:25 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:25.111+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:25.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:25 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:25.111+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:25.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:25 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:25.111+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:25.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:25 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:25.111+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:25.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:25 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:25.111+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:25.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:25 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:25.453+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:25.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:25 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:25.453+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:25.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:25 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:25.453+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:25.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:25 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:25.453+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:25.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:25 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:25.453+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:25.750 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:25 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:25.598+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:25.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:25 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:25.598+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:25.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:25 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:25.598+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:25.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:25 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:25.598+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:25.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:25 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:25.598+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.302614+0000 front 2024-04-02T14:56:00.302850+0000 (oldest deadline 2024-04-02T14:56:23.802643+0000) 2024-04-02T14:56:26.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:25 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:25.681+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:26.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:25 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:25.681+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:26.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:25 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:25.681+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:26.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:25 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:25.681+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:26.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:25 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:25.681+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:26.360 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:26 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:26.110+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:26.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:26 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:26.110+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:26.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:26 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:26.110+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:26.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:26 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:26.110+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:26.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:26 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:26.110+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:26.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:26 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:26.110+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:26.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:26.501+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:26.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:26.501+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:26.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:26.501+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:26.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:26.501+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:26.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:26.501+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:26.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:26.596+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.097551+0000 front 2024-04-02T14:55:27.097629+0000 (oldest deadline 2024-04-02T14:55:51.197550+0000) 2024-04-02T14:56:26.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:26.596+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.298520+0000 front 2024-04-02T14:55:35.298561+0000 (oldest deadline 2024-04-02T14:56:01.198734+0000) 2024-04-02T14:56:26.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:26.596+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:45.101018+0000 front 2024-04-02T14:55:45.101462+0000 (oldest deadline 2024-04-02T14:56:09.800812+0000) 2024-04-02T14:56:26.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:26.596+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:49.801568+0000 front 2024-04-02T14:55:49.801529+0000 (oldest deadline 2024-04-02T14:56:15.101355+0000) 2024-04-02T14:56:26.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8[54956]: 2024-04-02T14:56:26.596+0000 7fdf55e0f700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.302614+0000 front 2024-04-02T14:56:00.302850+0000 (oldest deadline 2024-04-02T14:56:23.802643+0000) 2024-04-02T14:56:27.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:26 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:26.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:27.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:26 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:26.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:27.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:26 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:26.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:27.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:26 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:26.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:27.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:26 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:26.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:27.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:26 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:26.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.573553+0000 front 2024-04-02T14:56:00.573439+0000 (oldest deadline 2024-04-02T14:56:25.873752+0000) 2024-04-02T14:56:27.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:27 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:27.085+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:27.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:27 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:27.085+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:27.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:27 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:27.085+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:27.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:27 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:27.085+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:27.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:27 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:27.085+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:27.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:27 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:27.085+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:27.749 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:27 smithi204 podman[73947]: 2024-04-02 14:56:27.602249108 +0000 UTC m=+5.621013339 container died d021ed3484333d3e9276bfbfa9a9abc14fb5bdd2f6b640dee6222abed1e201cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-02T14:56:27.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:27 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:27.470+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:27.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:27 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:27.470+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:27.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:27 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:27.470+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:27.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:27 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:27.470+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:27.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:27 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:27.470+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:28.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:27 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:27.683+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:28.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:27 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:27.683+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:28.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:27 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:27.683+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:28.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:27 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:27.683+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:28.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:27 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:27.683+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:28.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:27 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:27.683+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.573553+0000 front 2024-04-02T14:56:00.573439+0000 (oldest deadline 2024-04-02T14:56:25.873752+0000) 2024-04-02T14:56:28.199 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:27 smithi204 podman[73947]: 2024-04-02 14:56:27.958339595 +0000 UTC m=+5.977103816 container cleanup d021ed3484333d3e9276bfbfa9a9abc14fb5bdd2f6b640dee6222abed1e201cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:56:28.199 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:27 smithi204 bash[73947]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8 2024-04-02T14:56:28.360 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:28 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:28.092+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:28.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:28 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:28.092+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:28.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:28 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:28.092+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:28.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:28 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:28.092+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:28.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:28 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:28.092+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:28.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:28 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:28.092+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:28.500 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:28 smithi204 podman[73964]: 2024-04-02 14:56:28.209305696 +0000 UTC m=+0.601130807 container remove d021ed3484333d3e9276bfbfa9a9abc14fb5bdd2f6b640dee6222abed1e201cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-02T14:56:28.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:28 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:28.423+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:28.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:28 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:28.423+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:28.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:28 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:28.423+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:28.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:28 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:28.423+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:28.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:28 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:28.423+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:28.828 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:28 smithi204 podman[74058]: 2024-04-02 14:56:28.423841869 +0000 UTC m=+0.017565699 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:56:28.828 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:28 smithi204 podman[74058]: 2024-04-02 14:56:28.635394723 +0000 UTC m=+0.229118572 container create 809e86c018a79b39d33f939c549e3d9f18c6808e11a96b242cf97266789090e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux ) 2024-04-02T14:56:28.828 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:28 smithi204 podman[74058]: 2024-04-02 14:56:28.838250139 +0000 UTC m=+0.431973973 container init 809e86c018a79b39d33f939c549e3d9f18c6808e11a96b242cf97266789090e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux ) 2024-04-02T14:56:29.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:28.712+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:29.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:28.712+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:29.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:28.712+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:29.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:28.712+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:29.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:28.712+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:29.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:28 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:28.712+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.573553+0000 front 2024-04-02T14:56:00.573439+0000 (oldest deadline 2024-04-02T14:56:25.873752+0000) 2024-04-02T14:56:29.121 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:28 smithi204 podman[74058]: 2024-04-02 14:56:28.841469363 +0000 UTC m=+0.435193193 container start 809e86c018a79b39d33f939c549e3d9f18c6808e11a96b242cf97266789090e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8-deactivate, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-02T14:56:29.121 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:28 smithi204 podman[74058]: 2024-04-02 14:56:28.912588486 +0000 UTC m=+0.506312332 container attach 809e86c018a79b39d33f939c549e3d9f18c6808e11a96b242cf97266789090e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-02T14:56:29.360 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:29 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:29.070+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:29.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:29 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:29.070+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:29.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:29 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:29.070+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:29.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:29 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:29.070+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:29.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:29 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:29.070+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:29.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:29 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:29.070+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:29.386 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 02 14:56:29 smithi204 podman[74058]: 2024-04-02 14:56:29.131519266 +0000 UTC m=+0.725243096 container died 809e86c018a79b39d33f939c549e3d9f18c6808e11a96b242cf97266789090e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-8-deactivate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-04-02T14:56:29.697 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:29 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:29.420+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:29.698 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:29 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:29.420+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:29.698 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:29 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:29.420+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:29.698 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:29 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:29.420+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:29.698 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:29 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:29.420+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:29.715 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.8.service' 2024-04-02T14:56:29.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:56:29.757 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-02T14:56:29.757 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-02T14:56:29.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.9 2024-04-02T14:56:30.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:29 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:29.666+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:30.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:29 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:29.666+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:30.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:29 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:29.666+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:30.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:29 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:29.666+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:30.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:29 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:29.666+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:30.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:29 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:29.666+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.573553+0000 front 2024-04-02T14:56:00.573439+0000 (oldest deadline 2024-04-02T14:56:25.873752+0000) 2024-04-02T14:56:30.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:29 smithi062 systemd[1]: Stopping Ceph osd.9 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:56:30.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:30 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:30.107+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:30.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:30 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:30.107+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:30.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:30 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:30.107+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:30.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:30 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:30.107+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:30.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:30 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:30.107+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:30.362 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:30 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:30.107+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:30.584 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:30 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:30.314+0000 7ffbe4e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:56:30.584 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:30 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:30.314+0000 7ffbe4e00700 -1 osd.9 102 *** Got signal Terminated *** 2024-04-02T14:56:30.584 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:30 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:30.314+0000 7ffbe4e00700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:56:30.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:30 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:30.430+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:30.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:30 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:30.430+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:30.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:30 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:30.430+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:30.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:30 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:30.430+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:30.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:30 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:30.430+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:31.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:30 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:30.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:31.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:30 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:30.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:31.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:30 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:30.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:31.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:30 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:30.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:31.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:30 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:30.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:31.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:30 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:30.695+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.573553+0000 front 2024-04-02T14:56:00.573439+0000 (oldest deadline 2024-04-02T14:56:25.873752+0000) 2024-04-02T14:56:31.610 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:31 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:31.149+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:31.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:31 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:31.149+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:31.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:31 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:31.149+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:31.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:31 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:31.149+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:31.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:31 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:31.149+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:31.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:31 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:31.149+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:31.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:31 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:31.429+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:31.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:31 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:31.429+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:31.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:31 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:31.429+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:31.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:31 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:31.430+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:31.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:31 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:31.430+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:31.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:31 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:31.430+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:32.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:31 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:31.670+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:32.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:31 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:31.670+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:32.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:31 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:31.670+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:32.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:31 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:31.670+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:32.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:31 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:31.670+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:32.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:31 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:31.670+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.573553+0000 front 2024-04-02T14:56:00.573439+0000 (oldest deadline 2024-04-02T14:56:25.873752+0000) 2024-04-02T14:56:32.610 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:32 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:32.172+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:32.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:32 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:32.172+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:32.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:32 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:32.172+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:32.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:32 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:32.172+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:32.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:32 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:32.172+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:32.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:32 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:32.172+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:32.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:32 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:32.404+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:32.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:32 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:32.404+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:32.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:32 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:32.404+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:32.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:32 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:32.404+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:32.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:32 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:32.404+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:32.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:32 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:32.404+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:33.083 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:32 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:32.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:33.102 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:32 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:32.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:33.102 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:32 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:32.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:33.103 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:32 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:32.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:33.103 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:32 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:32.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:33.103 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:32 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:32.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.573553+0000 front 2024-04-02T14:56:00.573439+0000 (oldest deadline 2024-04-02T14:56:25.873752+0000) 2024-04-02T14:56:33.610 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:33 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:33.197+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:33.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:33 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:33.197+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:33.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:33 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:33.197+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:33.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:33 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:33.197+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:33.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:33 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:33.197+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:33.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:33 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:33.197+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:33.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:33 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:33.381+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:33.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:33 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:33.381+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:33.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:33 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:33.381+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:33.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:33 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:33.381+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:33.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:33 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:33.381+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:33.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:33 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:33.381+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:34.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:33 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:33.748+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:34.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:33 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:33.748+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:34.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:33 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:33.748+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:34.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:33 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:33.748+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:34.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:33 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:33.748+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:34.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:33 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:33.748+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.573553+0000 front 2024-04-02T14:56:00.573439+0000 (oldest deadline 2024-04-02T14:56:25.873752+0000) 2024-04-02T14:56:34.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:34 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:34.205+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:34.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:34 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:34.205+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:34.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:34 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:34.205+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:34.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:34 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:34.205+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:34.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:34 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:34.205+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:34.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:34 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:34.205+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:34.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:34 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:34.405+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:34.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:34 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:34.405+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:34.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:34 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:34.405+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:34.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:34 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:34.405+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:34.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:34 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:34.405+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:34.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:34 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:34.405+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:35.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:34 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:34.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:16.061812+0000 front 2024-04-02T14:55:16.062704+0000 (oldest deadline 2024-04-02T14:55:41.361702+0000) 2024-04-02T14:56:35.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:34 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:34.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:25.964029+0000 front 2024-04-02T14:55:25.964135+0000 (oldest deadline 2024-04-02T14:55:51.264259+0000) 2024-04-02T14:56:35.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:34 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:34.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:35.867411+0000 front 2024-04-02T14:55:35.867292+0000 (oldest deadline 2024-04-02T14:56:01.767561+0000) 2024-04-02T14:56:35.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:34 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:34.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.768796+0000 front 2024-04-02T14:55:41.768878+0000 (oldest deadline 2024-04-02T14:56:06.468783+0000) 2024-04-02T14:56:35.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:34 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:34.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:48.770787+0000 front 2024-04-02T14:55:48.771032+0000 (oldest deadline 2024-04-02T14:56:20.572239+0000) 2024-04-02T14:56:35.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:34 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9[65370]: 2024-04-02T14:56:34.700+0000 7ffbdb600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:00.573553+0000 front 2024-04-02T14:56:00.573439+0000 (oldest deadline 2024-04-02T14:56:25.873752+0000) 2024-04-02T14:56:35.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:35 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:35.179+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:35.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:35 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:35.179+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:35.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:35 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:35.179+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:35.612 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:35 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:35.179+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:35.612 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:35 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:35.179+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:35.612 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:35 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:35.179+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:35.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:35 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:35.380+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:35.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:35 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:35.380+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:35.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:35 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:35.380+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:35.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:35 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:35.380+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:35.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:35 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:35.380+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:35.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:35 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:35.380+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:36.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:35 smithi062 podman[94289]: 2024-04-02 14:56:35.708039621 +0000 UTC m=+5.516126757 container died 8eccc4555a8b4661773e3cb85729fe172ce013664be314cc31bfeaa93d3a5056 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:56:36.500 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:36 smithi062 podman[94289]: 2024-04-02 14:56:36.258270294 +0000 UTC m=+6.066357497 container cleanup 8eccc4555a8b4661773e3cb85729fe172ce013664be314cc31bfeaa93d3a5056 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-02T14:56:36.500 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:36 smithi062 bash[94289]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9 2024-04-02T14:56:36.610 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:36 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:36.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:36.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:36 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:36.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:36.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:36 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:36.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:36.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:36 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:36.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:36.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:36 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:36.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:36.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:36 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:36.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:36.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:36 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:36.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:09.469643+0000 front 2024-04-02T14:56:09.469792+0000 (oldest deadline 2024-04-02T14:56:35.369318+0000) 2024-04-02T14:56:36.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:36 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:36.377+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:36.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:36 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:36.377+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:36.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:36 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:36.377+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:36.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:36 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:36.377+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:36.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:36 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:36.377+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:36.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:36 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:36.377+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:36.833 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:36 smithi062 podman[94307]: 2024-04-02 14:56:36.509386876 +0000 UTC m=+0.792937782 container remove 8eccc4555a8b4661773e3cb85729fe172ce013664be314cc31bfeaa93d3a5056 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-02T14:56:37.610 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:37 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:37.183+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:37.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:37 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:37.183+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:37.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:37 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:37.183+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:37.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:37 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:37.183+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:37.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:37 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:37.183+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:37.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:37 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:37.183+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:37.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:37 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:37.183+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:09.469643+0000 front 2024-04-02T14:56:09.469792+0000 (oldest deadline 2024-04-02T14:56:35.369318+0000) 2024-04-02T14:56:37.705 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:37 smithi062 podman[94396]: 2024-04-02 14:56:37.306439196 +0000 UTC m=+0.040387234 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:56:37.705 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:37 smithi062 podman[94396]: 2024-04-02 14:56:37.401819405 +0000 UTC m=+0.135767463 container create 2ea9f48b563ff6b5dbf7c4afb5d16fdde7b600f4611aee202ce1d3a44ebe434d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD) 2024-04-02T14:56:37.705 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:37 smithi062 podman[94396]: 2024-04-02 14:56:37.714759421 +0000 UTC m=+0.448707486 container init 2ea9f48b563ff6b5dbf7c4afb5d16fdde7b600f4611aee202ce1d3a44ebe434d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:56:37.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:37 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:37.410+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:37.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:37 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:37.410+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:37.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:37 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:37.410+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:37.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:37 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:37.410+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:37.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:37 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:37.410+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:37.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:37 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:37.410+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:38.084 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:37 smithi062 podman[94396]: 2024-04-02 14:56:37.726581727 +0000 UTC m=+0.460529791 container start 2ea9f48b563ff6b5dbf7c4afb5d16fdde7b600f4611aee202ce1d3a44ebe434d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326) 2024-04-02T14:56:38.085 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:37 smithi062 podman[94396]: 2024-04-02 14:56:37.821065406 +0000 UTC m=+0.555013531 container attach 2ea9f48b563ff6b5dbf7c4afb5d16fdde7b600f4611aee202ce1d3a44ebe434d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-02T14:56:38.568 INFO:journalctl@ceph.osd.9.smithi062.stdout:Apr 02 14:56:38 smithi062 podman[94396]: 2024-04-02 14:56:38.18399993 +0000 UTC m=+0.917947951 container died 2ea9f48b563ff6b5dbf7c4afb5d16fdde7b600f4611aee202ce1d3a44ebe434d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-9-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:56:38.610 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:38 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:38.184+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:38.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:38 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:38.184+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:38.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:38 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:38.184+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:38.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:38 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:38.184+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:38.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:38 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:38.184+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:38.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:38 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:38.184+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:38.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:38 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:38.184+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:09.469643+0000 front 2024-04-02T14:56:09.469792+0000 (oldest deadline 2024-04-02T14:56:35.369318+0000) 2024-04-02T14:56:38.651 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:38 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:38.393+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:38.651 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:38 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:38.393+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:38.651 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:38 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:38.393+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:38.651 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:38 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:38.393+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:38.651 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:38 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:38.393+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:38.651 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:38 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:38.393+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:38.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.9.service' 2024-04-02T14:56:39.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:56:39.045 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-02T14:56:39.046 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-02T14:56:39.046 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.10 2024-04-02T14:56:39.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:39 smithi113 systemd[1]: Stopping Ceph osd.10 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:56:39.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:39 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:39.215+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:39.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:39 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:39.215+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:39.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:39 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:39.215+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:39.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:39 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:39.215+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:39.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:39 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:39.215+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:39.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:39 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:39.215+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:39.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:39 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:39.215+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:09.469643+0000 front 2024-04-02T14:56:09.469792+0000 (oldest deadline 2024-04-02T14:56:35.369318+0000) 2024-04-02T14:56:39.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:39 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:39.382+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:39.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:39 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:39.382+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:39.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:39 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:39.382+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:39.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:39 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:39.382+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:39.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:39 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:39.382+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:39.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:39 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:39.382+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:39.861 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:39 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:39.430+0000 7f937cae0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:56:39.861 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:39 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:39.430+0000 7f937cae0700 -1 osd.10 102 *** Got signal Terminated *** 2024-04-02T14:56:39.861 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:39 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:39.430+0000 7f937cae0700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:56:40.610 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:40 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:40.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:40.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:40 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:40.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:40.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:40 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:40.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:40.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:40 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:40.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:40.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:40 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:40.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:40.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:40 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:40.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:40.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:40 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:40.213+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:09.469643+0000 front 2024-04-02T14:56:09.469792+0000 (oldest deadline 2024-04-02T14:56:35.369318+0000) 2024-04-02T14:56:40.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:40 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:40.406+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:40.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:40 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:40.406+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:40.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:40 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:40.406+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:40.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:40 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:40.406+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:40.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:40 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:40.406+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:40.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:40 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:40.406+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:41.610 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:41 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:41.260+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:41.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:41 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:41.260+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:41.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:41 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:41.260+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:41.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:41 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:41.260+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:41.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:41 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:41.260+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:41.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:41 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:41.260+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:41.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:41 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:41.260+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:09.469643+0000 front 2024-04-02T14:56:09.469792+0000 (oldest deadline 2024-04-02T14:56:35.369318+0000) 2024-04-02T14:56:41.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:41 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:41.433+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:41.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:41 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:41.433+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:41.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:41 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:41.433+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:41.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:41 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:41.433+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:41.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:41 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:41.433+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:41.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:41 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:41.433+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:42.610 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:42 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:42.255+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:42.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:42 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:42.256+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:42.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:42 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:42.256+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:42.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:42 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:42.256+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:42.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:42 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:42.256+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:42.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:42 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:42.256+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:42.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:42 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:42.256+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:09.469643+0000 front 2024-04-02T14:56:09.469792+0000 (oldest deadline 2024-04-02T14:56:35.369318+0000) 2024-04-02T14:56:42.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:42.425+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:42.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:42.425+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:42.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:42.425+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:42.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:42.425+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:42.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:42.425+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:42.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:42.425+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:42.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:42 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:42.425+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6820 osd.7 since back 2024-04-02T14:56:19.156910+0000 front 2024-04-02T14:56:19.156878+0000 (oldest deadline 2024-04-02T14:56:41.456839+0000) 2024-04-02T14:56:43.610 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:43 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:43.275+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:43.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:43 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:43.275+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:43.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:43 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:43.275+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:43.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:43 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:43.275+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:43.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:43 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:43.275+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:43.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:43 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:43.275+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:43.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:43 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:43.275+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:09.469643+0000 front 2024-04-02T14:56:09.469792+0000 (oldest deadline 2024-04-02T14:56:35.369318+0000) 2024-04-02T14:56:43.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:43.452+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:43.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:43.452+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:43.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:43.452+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:43.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:43.452+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:43.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:43.452+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:43.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:43.452+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:43.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:43 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:43.452+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6820 osd.7 since back 2024-04-02T14:56:19.156910+0000 front 2024-04-02T14:56:19.156878+0000 (oldest deadline 2024-04-02T14:56:41.456839+0000) 2024-04-02T14:56:44.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:44.255+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.663016+0000 front 2024-04-02T14:55:19.662913+0000 (oldest deadline 2024-04-02T14:55:41.362793+0000) 2024-04-02T14:56:44.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:44.255+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:26.663583+0000 front 2024-04-02T14:55:26.663705+0000 (oldest deadline 2024-04-02T14:55:51.963662+0000) 2024-04-02T14:56:44.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:44.255+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.264717+0000 front 2024-04-02T14:55:34.264636+0000 (oldest deadline 2024-04-02T14:55:58.364642+0000) 2024-04-02T14:56:44.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:44.255+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-04-02T14:55:41.865915+0000 front 2024-04-02T14:55:41.866044+0000 (oldest deadline 2024-04-02T14:56:06.565871+0000) 2024-04-02T14:56:44.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:44.255+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:53.567377+0000 front 2024-04-02T14:55:53.567413+0000 (oldest deadline 2024-04-02T14:56:19.467559+0000) 2024-04-02T14:56:44.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:44.255+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:55:59.468256+0000 front 2024-04-02T14:55:59.468306+0000 (oldest deadline 2024-04-02T14:56:24.768100+0000) 2024-04-02T14:56:44.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:44 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10[61631]: 2024-04-02T14:56:44.255+0000 7f93790d4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:09.469643+0000 front 2024-04-02T14:56:09.469792+0000 (oldest deadline 2024-04-02T14:56:35.369318+0000) 2024-04-02T14:56:44.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:44 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:44.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:44.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:44 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:44.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:44.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:44 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:44.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:44.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:44 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:44.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:44.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:44 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:44.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:44.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:44 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:44.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:44.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:44 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:44.446+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6820 osd.7 since back 2024-04-02T14:56:19.156910+0000 front 2024-04-02T14:56:19.156878+0000 (oldest deadline 2024-04-02T14:56:41.456839+0000) 2024-04-02T14:56:45.111 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:44 smithi113 podman[73621]: 2024-04-02 14:56:44.849355798 +0000 UTC m=+5.597119802 container died d1a2a974f592d5c4f981a0670f2069637040595121bd4b844b5711ec5a88f52d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-02T14:56:45.610 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:45 smithi113 podman[73621]: 2024-04-02 14:56:45.204184485 +0000 UTC m=+5.951948476 container cleanup d1a2a974f592d5c4f981a0670f2069637040595121bd4b844b5711ec5a88f52d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:56:45.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:45 smithi113 bash[73621]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10 2024-04-02T14:56:45.611 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:45 smithi113 podman[73640]: 2024-04-02 14:56:45.36372501 +0000 UTC m=+0.513254006 container remove d1a2a974f592d5c4f981a0670f2069637040595121bd4b844b5711ec5a88f52d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:56:45.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:45.407+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:45.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:45.407+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:45.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:45.407+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:45.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:45.407+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:45.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:45.407+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:45.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:45.407+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:45.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:45 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:45.407+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6820 osd.7 since back 2024-04-02T14:56:19.156910+0000 front 2024-04-02T14:56:19.156878+0000 (oldest deadline 2024-04-02T14:56:41.456839+0000) 2024-04-02T14:56:46.057 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:45 smithi113 podman[73733]: 2024-04-02 14:56:45.756502801 +0000 UTC m=+0.108851546 container create acaf018dabaac8ea573263655d501b5805a8333dd028d3b2e40b2bbb909155e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:56:46.057 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:45 smithi113 podman[73733]: 2024-04-02 14:56:45.664922683 +0000 UTC m=+0.017271450 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:56:46.057 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:45 smithi113 podman[73733]: 2024-04-02 14:56:45.984276548 +0000 UTC m=+0.336625313 container init acaf018dabaac8ea573263655d501b5805a8333dd028d3b2e40b2bbb909155e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-04-02T14:56:46.057 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:45 smithi113 podman[73733]: 2024-04-02 14:56:45.987520646 +0000 UTC m=+0.339869393 container start acaf018dabaac8ea573263655d501b5805a8333dd028d3b2e40b2bbb909155e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-02T14:56:46.057 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:46 smithi113 podman[73733]: 2024-04-02 14:56:46.067147444 +0000 UTC m=+0.419496190 container attach acaf018dabaac8ea573263655d501b5805a8333dd028d3b2e40b2bbb909155e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.license=GPLv2) 2024-04-02T14:56:46.361 INFO:journalctl@ceph.osd.10.smithi113.stdout:Apr 02 14:56:46 smithi113 podman[73733]: 2024-04-02 14:56:46.277079608 +0000 UTC m=+0.629428359 container died acaf018dabaac8ea573263655d501b5805a8333dd028d3b2e40b2bbb909155e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-10-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:56:46.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:46 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:46.447+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:46.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:46 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:46.447+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:46.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:46 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:46.447+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:46.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:46 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:46.447+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:46.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:46 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:46.447+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:46.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:46 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:46.447+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:46.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:46 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:46.447+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6820 osd.7 since back 2024-04-02T14:56:19.156910+0000 front 2024-04-02T14:56:19.156878+0000 (oldest deadline 2024-04-02T14:56:41.456839+0000) 2024-04-02T14:56:46.879 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.10.service' 2024-04-02T14:56:46.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:56:46.921 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-02T14:56:46.921 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-02T14:56:46.921 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.11 2024-04-02T14:56:47.225 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:46 smithi204 systemd[1]: Stopping Ceph osd.11 for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:56:47.499 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:47.234+0000 7f41192e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:56:47.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:47.234+0000 7f41192e0700 -1 osd.11 102 *** Got signal Terminated *** 2024-04-02T14:56:47.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:47.234+0000 7f41192e0700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:56:47.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:47.444+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:47.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:47.444+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:47.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:47.444+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:47.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:47.444+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:47.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:47.444+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:47.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:47.444+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:47.500 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:47 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:47.444+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6820 osd.7 since back 2024-04-02T14:56:19.156910+0000 front 2024-04-02T14:56:19.156878+0000 (oldest deadline 2024-04-02T14:56:41.456839+0000) 2024-04-02T14:56:48.713 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:48 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:48.461+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:48.713 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:48 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:48.461+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:48.713 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:48 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:48.462+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:48.713 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:48 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:48.462+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:48.713 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:48 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:48.462+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:48.713 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:48 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:48.462+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:48.713 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:48 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:48.462+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6820 osd.7 since back 2024-04-02T14:56:19.156910+0000 front 2024-04-02T14:56:19.156878+0000 (oldest deadline 2024-04-02T14:56:41.456839+0000) 2024-04-02T14:56:49.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:49 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:49.485+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:49.749 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:49 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:49.485+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:49.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:49 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:49.485+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:49.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:49 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:49.485+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:49.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:49 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:49.485+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:49.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:49 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:49.485+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:49.750 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:49 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:49.485+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6820 osd.7 since back 2024-04-02T14:56:19.156910+0000 front 2024-04-02T14:56:19.156878+0000 (oldest deadline 2024-04-02T14:56:41.456839+0000) 2024-04-02T14:56:51.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:50 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:50.530+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:51.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:50 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:50.530+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:51.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:50 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:50.530+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:51.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:50 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:50.530+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:51.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:50 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:50.530+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:51.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:50 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:50.530+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:51.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:50 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:50.530+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6820 osd.7 since back 2024-04-02T14:56:19.156910+0000 front 2024-04-02T14:56:19.156878+0000 (oldest deadline 2024-04-02T14:56:41.456839+0000) 2024-04-02T14:56:52.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:51.566+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-04-02T14:55:19.747444+0000 front 2024-04-02T14:55:19.747433+0000 (oldest deadline 2024-04-02T14:55:41.447244+0000) 2024-04-02T14:56:52.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:51.566+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6804 osd.1 since back 2024-04-02T14:55:27.248549+0000 front 2024-04-02T14:55:27.248645+0000 (oldest deadline 2024-04-02T14:55:52.449336+0000) 2024-04-02T14:56:52.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:51.566+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-02T14:55:34.750222+0000 front 2024-04-02T14:55:34.750255+0000 (oldest deadline 2024-04-02T14:56:00.650384+0000) 2024-04-02T14:56:52.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:51.566+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6812 osd.4 since back 2024-04-02T14:55:52.853304+0000 front 2024-04-02T14:55:52.853401+0000 (oldest deadline 2024-04-02T14:56:17.553120+0000) 2024-04-02T14:56:52.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:51.566+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-02T14:56:01.653961+0000 front 2024-04-02T14:56:01.653925+0000 (oldest deadline 2024-04-02T14:56:25.153999+0000) 2024-04-02T14:56:52.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:51.566+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-04-02T14:56:05.154471+0000 front 2024-04-02T14:56:05.154504+0000 (oldest deadline 2024-04-02T14:56:31.054456+0000) 2024-04-02T14:56:52.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:51 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11[62013]: 2024-04-02T14:56:51.566+0000 7f41158d4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.113:6820 osd.7 since back 2024-04-02T14:56:19.156910+0000 front 2024-04-02T14:56:19.156878+0000 (oldest deadline 2024-04-02T14:56:41.456839+0000) 2024-04-02T14:56:52.990 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:52 smithi204 podman[74178]: 2024-04-02 14:56:52.672794923 +0000 UTC m=+5.548301341 container died c1892443fdb3c3ca2c3d1c205070851d9d05d3654bec795714d18d6af84be522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-02T14:56:53.249 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:53 smithi204 podman[74178]: 2024-04-02 14:56:53.001470446 +0000 UTC m=+5.876976853 container cleanup c1892443fdb3c3ca2c3d1c205070851d9d05d3654bec795714d18d6af84be522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-02T14:56:53.250 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:53 smithi204 bash[74178]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11 2024-04-02T14:56:53.250 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:53 smithi204 podman[74195]: 2024-04-02 14:56:53.211086407 +0000 UTC m=+0.534583821 container remove c1892443fdb3c3ca2c3d1c205070851d9d05d3654bec795714d18d6af84be522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-02T14:56:53.787 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:53 smithi204 podman[74288]: 2024-04-02 14:56:53.419360329 +0000 UTC m=+0.017926111 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:56:53.787 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:53 smithi204 podman[74288]: 2024-04-02 14:56:53.528418109 +0000 UTC m=+0.126983867 container create c520352942d542a02275b52ec7f5752ef658f774f34c09d2699b7d38f1738f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:56:54.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:53 smithi204 podman[74288]: 2024-04-02 14:56:53.797619854 +0000 UTC m=+0.396185648 container init c520352942d542a02275b52ec7f5752ef658f774f34c09d2699b7d38f1738f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326) 2024-04-02T14:56:54.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:53 smithi204 podman[74288]: 2024-04-02 14:56:53.80108489 +0000 UTC m=+0.399650659 container start c520352942d542a02275b52ec7f5752ef658f774f34c09d2699b7d38f1738f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5) 2024-04-02T14:56:54.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:53 smithi204 podman[74288]: 2024-04-02 14:56:53.906101071 +0000 UTC m=+0.504666857 container attach c520352942d542a02275b52ec7f5752ef658f774f34c09d2699b7d38f1738f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:56:54.366 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 02 14:56:54 smithi204 podman[74288]: 2024-04-02 14:56:54.09655409 +0000 UTC m=+0.695119866 container died c520352942d542a02275b52ec7f5752ef658f774f34c09d2699b7d38f1738f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-02T14:56:54.630 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@osd.11.service' 2024-04-02T14:56:54.671 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:56:54.671 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-02T14:56:54.672 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-02T14:56:54.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.a 2024-04-02T14:56:55.083 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:56:54 smithi062 systemd[1]: Stopping Ceph mds.a for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:56:55.583 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:56:55 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: 2024-04-02T14:56:55.327+0000 7f1442000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:56:55.583 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:56:55 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a[72296]: 2024-04-02T14:56:55.327+0000 7f1442000700 -1 mds.a *** got signal Terminated *** 2024-04-02T14:56:58.164 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:56:57 smithi062 podman[94513]: 2024-04-02 14:56:57.90674795 +0000 UTC m=+2.797970189 container died e5aee3247ea0b3a05590c9d5b6147383f343464fe8ba7e531779bcbe427fd953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux ) 2024-04-02T14:56:58.520 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:56:58 smithi062 podman[94513]: 2024-04-02 14:56:58.300981643 +0000 UTC m=+3.192203872 container cleanup e5aee3247ea0b3a05590c9d5b6147383f343464fe8ba7e531779bcbe427fd953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5) 2024-04-02T14:56:58.520 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:56:58 smithi062 bash[94513]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a 2024-04-02T14:56:58.833 INFO:journalctl@ceph.mds.a.smithi062.stdout:Apr 02 14:56:58 smithi062 podman[94527]: 2024-04-02 14:56:58.530036486 +0000 UTC m=+0.606644365 container remove e5aee3247ea0b3a05590c9d5b6147383f343464fe8ba7e531779bcbe427fd953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-a, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T14:56:59.000 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.a.service' 2024-04-02T14:56:59.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:56:59.129 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-02T14:56:59.129 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-02T14:56:59.129 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.d 2024-04-02T14:56:59.583 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:56:59 smithi062 systemd[1]: Stopping Ceph mds.d for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:57:00.083 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:56:59 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d[71341]: 2024-04-02T14:56:59.820+0000 7f1f36400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:57:00.083 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:56:59 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d[71341]: 2024-04-02T14:56:59.820+0000 7f1f36400700 -1 mds.d *** got signal Terminated *** 2024-04-02T14:57:02.191 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:57:01 smithi062 podman[94633]: 2024-04-02 14:57:01.88034447 +0000 UTC m=+2.253390690 container died dab721609ae75c70b9310df6d2ca345e3033ba603aa169d7054387a1b1b7b84b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:57:02.571 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:57:02 smithi062 podman[94633]: 2024-04-02 14:57:02.312555178 +0000 UTC m=+2.685601396 container cleanup dab721609ae75c70b9310df6d2ca345e3033ba603aa169d7054387a1b1b7b84b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d, ceph=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:57:02.571 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:57:02 smithi062 bash[94633]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d 2024-04-02T14:57:02.833 INFO:journalctl@ceph.mds.d.smithi062.stdout:Apr 02 14:57:02 smithi062 podman[94647]: 2024-04-02 14:57:02.580018496 +0000 UTC m=+0.678077455 container remove dab721609ae75c70b9310df6d2ca345e3033ba603aa169d7054387a1b1b7b84b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-d, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:57:03.146 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.d.service' 2024-04-02T14:57:03.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:57:03.266 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-02T14:57:03.267 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-02T14:57:03.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.g 2024-04-02T14:57:03.833 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:57:03 smithi062 systemd[1]: Stopping Ceph mds.g for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:57:04.333 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:57:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-g[71943]: 2024-04-02T14:57:03.914+0000 7fc386800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:57:04.333 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:57:03 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-g[71943]: 2024-04-02T14:57:03.914+0000 7fc386800700 -1 mds.g *** got signal Terminated *** 2024-04-02T14:57:06.195 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:57:05 smithi062 podman[94754]: 2024-04-02 14:57:05.874577858 +0000 UTC m=+2.128513902 container died 3fc8af3cdc2c3251a894dff132927130df6b8eb630b6d21602761f0419fda253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-g, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:57:06.499 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:57:06 smithi062 podman[94754]: 2024-04-02 14:57:06.324552781 +0000 UTC m=+2.578488835 container cleanup 3fc8af3cdc2c3251a894dff132927130df6b8eb630b6d21602761f0419fda253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-g, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:57:06.499 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:57:06 smithi062 bash[94754]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-g 2024-04-02T14:57:06.833 INFO:journalctl@ceph.mds.g.smithi062.stdout:Apr 02 14:57:06 smithi062 podman[94768]: 2024-04-02 14:57:06.508981626 +0000 UTC m=+0.613531782 container remove 3fc8af3cdc2c3251a894dff132927130df6b8eb630b6d21602761f0419fda253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-g, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:57:07.011 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.g.service' 2024-04-02T14:57:07.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:57:07.134 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-02T14:57:07.135 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-02T14:57:07.135 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.j 2024-04-02T14:57:07.583 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:57:07 smithi062 systemd[1]: Stopping Ceph mds.j for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:57:08.083 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:57:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-j[70962]: 2024-04-02T14:57:07.763+0000 7f5b47c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:57:08.083 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:57:07 smithi062 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-j[70962]: 2024-04-02T14:57:07.763+0000 7f5b47c00700 -1 mds.j *** got signal Terminated *** 2024-04-02T14:57:10.506 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:57:10 smithi062 podman[94873]: 2024-04-02 14:57:10.168542052 +0000 UTC m=+2.586112193 container stop e1c39c3c50789e51f8aaf29f430d6baa7aacdbcfc32e67ef3b81d2d91eb6347c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-j, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-04-02T14:57:10.506 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:57:10 smithi062 podman[94873]: 2024-04-02 14:57:10.196999683 +0000 UTC m=+2.614569784 container died e1c39c3c50789e51f8aaf29f430d6baa7aacdbcfc32e67ef3b81d2d91eb6347c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-j, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:57:10.833 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:57:10 smithi062 podman[94873]: 2024-04-02 14:57:10.628032646 +0000 UTC m=+3.045602767 container cleanup e1c39c3c50789e51f8aaf29f430d6baa7aacdbcfc32e67ef3b81d2d91eb6347c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-j, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:57:10.833 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:57:10 smithi062 bash[94873]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-j 2024-04-02T14:57:11.333 INFO:journalctl@ceph.mds.j.smithi062.stdout:Apr 02 14:57:10 smithi062 podman[94900]: 2024-04-02 14:57:10.853765397 +0000 UTC m=+0.666652967 container remove e1c39c3c50789e51f8aaf29f430d6baa7aacdbcfc32e67ef3b81d2d91eb6347c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-j, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0) 2024-04-02T14:57:11.384 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.j.service' 2024-04-02T14:57:11.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:57:11.506 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-02T14:57:11.506 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-02T14:57:11.506 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.b 2024-04-02T14:57:11.792 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:57:11 smithi113 systemd[1]: Stopping Ceph mds.b for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:57:12.059 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:57:11 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b[67157]: 2024-04-02T14:57:11.801+0000 7ff8ffca5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:57:12.059 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:57:11 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b[67157]: 2024-04-02T14:57:11.801+0000 7ff8ffca5700 -1 mds.b *** got signal Terminated *** 2024-04-02T14:57:14.110 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:57:13 smithi113 podman[73839]: 2024-04-02 14:57:13.830089385 +0000 UTC m=+2.126077048 container died 0e462871d7d40d864d82ba4eae0626199351c862efd1ba2162a988c08024ed08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-04-02T14:57:14.439 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:57:14 smithi113 podman[73839]: 2024-04-02 14:57:14.144069313 +0000 UTC m=+2.440056964 container cleanup 0e462871d7d40d864d82ba4eae0626199351c862efd1ba2162a988c08024ed08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD) 2024-04-02T14:57:14.440 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:57:14 smithi113 bash[73839]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b 2024-04-02T14:57:14.440 INFO:journalctl@ceph.mds.b.smithi113.stdout:Apr 02 14:57:14 smithi113 podman[73865]: 2024-04-02 14:57:14.345084989 +0000 UTC m=+0.508763098 container remove 0e462871d7d40d864d82ba4eae0626199351c862efd1ba2162a988c08024ed08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-b, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T14:57:14.462 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.b.service' 2024-04-02T14:57:14.500 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:57:14.500 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-02T14:57:14.500 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-02T14:57:14.501 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.e 2024-04-02T14:57:14.821 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:57:14 smithi113 systemd[1]: Stopping Ceph mds.e for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:57:15.110 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:57:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e[66537]: 2024-04-02T14:57:14.830+0000 7fab30fa0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:57:15.110 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:57:14 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e[66537]: 2024-04-02T14:57:14.830+0000 7fab30fa0700 -1 mds.e *** got signal Terminated *** 2024-04-02T14:57:18.096 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:57:17 smithi113 podman[73973]: 2024-04-02 14:57:17.814117289 +0000 UTC m=+3.090219529 container died e9fd3363bc176939f37739638eaafb45abba031c9ac0645239ecb3fad0fb0d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-02T14:57:18.360 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:57:18 smithi113 podman[73973]: 2024-04-02 14:57:18.189103642 +0000 UTC m=+3.465205909 container cleanup e9fd3363bc176939f37739638eaafb45abba031c9ac0645239ecb3fad0fb0d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:57:18.360 INFO:journalctl@ceph.mds.e.smithi113.stdout:Apr 02 14:57:18 smithi113 bash[73973]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-e 2024-04-02T14:57:18.590 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.e.service' 2024-04-02T14:57:18.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:57:18.626 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-02T14:57:18.626 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-02T14:57:18.626 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.h 2024-04-02T14:57:19.110 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:57:18 smithi113 systemd[1]: Stopping Ceph mds.h for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:57:19.111 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:57:18 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: 2024-04-02T14:57:18.942+0000 7efe7d886700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:57:19.111 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:57:18 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h[66846]: 2024-04-02T14:57:18.942+0000 7efe7d886700 -1 mds.h *** got signal Terminated *** 2024-04-02T14:57:22.111 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:57:21 smithi113 podman[74094]: 2024-04-02 14:57:21.831670119 +0000 UTC m=+2.976819633 container died 16a6adee0d7a7b21f3018c04cbb5339923c7d4a5d31e4d2cf9bb08fbf97c1db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-02T14:57:22.480 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:57:22 smithi113 podman[74094]: 2024-04-02 14:57:22.125844635 +0000 UTC m=+3.270994139 container cleanup 16a6adee0d7a7b21f3018c04cbb5339923c7d4a5d31e4d2cf9bb08fbf97c1db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5) 2024-04-02T14:57:22.480 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:57:22 smithi113 bash[74094]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h 2024-04-02T14:57:22.480 INFO:journalctl@ceph.mds.h.smithi113.stdout:Apr 02 14:57:22 smithi113 podman[74109]: 2024-04-02 14:57:22.310751852 +0000 UTC m=+0.480586477 container remove 16a6adee0d7a7b21f3018c04cbb5339923c7d4a5d31e4d2cf9bb08fbf97c1db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-h, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:57:22.532 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.h.service' 2024-04-02T14:57:22.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:57:22.567 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-02T14:57:22.567 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-02T14:57:22.568 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.k 2024-04-02T14:57:22.860 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:57:22 smithi113 systemd[1]: Stopping Ceph mds.k for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:57:23.360 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:57:22 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k[66212]: 2024-04-02T14:57:22.870+0000 7f550dd40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:57:23.361 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:57:22 smithi113 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k[66212]: 2024-04-02T14:57:22.870+0000 7f550dd40700 -1 mds.k *** got signal Terminated *** 2024-04-02T14:57:26.094 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:57:25 smithi113 podman[74217]: 2024-04-02 14:57:25.820645708 +0000 UTC m=+3.028623749 container died 5be66e02d69448a83d68ed8fd9a18c5d22b3bfeaf296e6e9ac63340a4154e894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD) 2024-04-02T14:57:26.360 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:57:26 smithi113 podman[74217]: 2024-04-02 14:57:26.187898907 +0000 UTC m=+3.395876883 container cleanup 5be66e02d69448a83d68ed8fd9a18c5d22b3bfeaf296e6e9ac63340a4154e894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:57:26.361 INFO:journalctl@ceph.mds.k.smithi113.stdout:Apr 02 14:57:26 smithi113 bash[74217]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-k 2024-04-02T14:57:26.593 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.k.service' 2024-04-02T14:57:26.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:57:26.626 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-02T14:57:26.626 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-02T14:57:26.627 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.c 2024-04-02T14:57:26.924 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:57:26 smithi204 systemd[1]: Stopping Ceph mds.c for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:57:27.187 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:57:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c[67310]: 2024-04-02T14:57:26.933+0000 7fa038acd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:57:27.187 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:57:26 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c[67310]: 2024-04-02T14:57:26.934+0000 7fa038acd700 -1 mds.c *** got signal Terminated *** 2024-04-02T14:57:30.099 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:57:29 smithi204 podman[74393]: 2024-04-02 14:57:29.831327597 +0000 UTC m=+3.002309062 container died b1021335a446eea8ab19d6ff3a7e7a2a89ccfc634225a5973e4d209090e2eb71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-02T14:57:30.458 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:57:30 smithi204 podman[74393]: 2024-04-02 14:57:30.209390177 +0000 UTC m=+3.380371633 container cleanup b1021335a446eea8ab19d6ff3a7e7a2a89ccfc634225a5973e4d209090e2eb71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True) 2024-04-02T14:57:30.458 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 02 14:57:30 smithi204 bash[74393]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-c 2024-04-02T14:57:30.629 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.c.service' 2024-04-02T14:57:30.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:57:30.669 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-02T14:57:30.669 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-02T14:57:30.669 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.f 2024-04-02T14:57:30.986 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:57:30 smithi204 systemd[1]: Stopping Ceph mds.f for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:57:31.249 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:57:30 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f[67002]: 2024-04-02T14:57:30.995+0000 7fdb11d84700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:57:31.249 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:57:30 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f[67002]: 2024-04-02T14:57:30.996+0000 7fdb11d84700 -1 mds.f *** got signal Terminated *** 2024-04-02T14:57:34.110 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:57:33 smithi204 podman[74528]: 2024-04-02 14:57:33.833490076 +0000 UTC m=+2.930991286 container died 0719b437b19e38afba04c13a63cac3b4fc0dfd3cfc3a3c0e25199ba230f5239f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True) 2024-04-02T14:57:34.111 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:57:34 smithi204 podman[74528]: 2024-04-02 14:57:34.121126863 +0000 UTC m=+3.218628074 container cleanup 0719b437b19e38afba04c13a63cac3b4fc0dfd3cfc3a3c0e25199ba230f5239f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-02T14:57:34.467 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:57:34 smithi204 bash[74528]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f 2024-04-02T14:57:34.467 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 02 14:57:34 smithi204 podman[74542]: 2024-04-02 14:57:34.363887579 +0000 UTC m=+0.530932421 container remove 0719b437b19e38afba04c13a63cac3b4fc0dfd3cfc3a3c0e25199ba230f5239f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-f, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-02T14:57:34.478 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.f.service' 2024-04-02T14:57:34.514 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:57:34.514 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-02T14:57:34.514 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-02T14:57:34.515 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.i 2024-04-02T14:57:34.749 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:57:34 smithi204 systemd[1]: Stopping Ceph mds.i for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:57:35.249 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:57:34 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i[67624]: 2024-04-02T14:57:34.865+0000 7f64e46cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:57:35.249 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:57:34 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i[67624]: 2024-04-02T14:57:34.865+0000 7f64e46cc700 -1 mds.i *** got signal Terminated *** 2024-04-02T14:57:37.999 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:57:37 smithi204 podman[74647]: 2024-04-02 14:57:37.816495357 +0000 UTC m=+3.078362795 container died 781f554886b89a4f3aadfae974e0221857f362f4e63050476e18e895421228ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:57:38.424 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:57:38 smithi204 podman[74647]: 2024-04-02 14:57:38.175755335 +0000 UTC m=+3.437622770 container cleanup 781f554886b89a4f3aadfae974e0221857f362f4e63050476e18e895421228ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-02T14:57:38.424 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 02 14:57:38 smithi204 bash[74647]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-i 2024-04-02T14:57:38.545 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.i.service' 2024-04-02T14:57:38.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:57:38.580 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-02T14:57:38.580 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-02T14:57:38.580 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.l 2024-04-02T14:57:38.749 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:57:38 smithi204 systemd[1]: Stopping Ceph mds.l for 9e86b65e-f0fd-11ee-b647-cb9ed24678a4... 2024-04-02T14:57:39.249 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:57:38 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-l[66696]: 2024-04-02T14:57:38.903+0000 7fe06b2b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:57:39.249 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:57:38 smithi204 ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-l[66696]: 2024-04-02T14:57:38.903+0000 7fe06b2b6700 -1 mds.l *** got signal Terminated *** 2024-04-02T14:57:43.194 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:57:42 smithi204 podman[74768]: 2024-04-02 14:57:42.910777061 +0000 UTC m=+4.106616572 container died c1ddf547fb051b9324dcfcb5068d989dfead323ff0ae50acf3efc4af40e69ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:57:43.499 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:57:43 smithi204 podman[74768]: 2024-04-02 14:57:43.304226367 +0000 UTC m=+4.500065869 container cleanup c1ddf547fb051b9324dcfcb5068d989dfead323ff0ae50acf3efc4af40e69ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-l, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326) 2024-04-02T14:57:43.499 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 02 14:57:43 smithi204 bash[74768]: ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4-mds-l 2024-04-02T14:57:43.639 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e86b65e-f0fd-11ee-b647-cb9ed24678a4@mds.l.service' 2024-04-02T14:57:43.672 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:57:43.672 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-02T14:57:43.673 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T14:57:44.018 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:57:49.095 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T14:57:49.286 INFO:teuthology.orchestra.run.smithi113.stdout:Deleting cluster with fsid: 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:57:51.813 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T14:57:52.002 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:57:54.477 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:57:54.585 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:57:54.620 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:57:54.649 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-02T14:57:54.651 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645/remote/smithi062/crash 2024-04-02T14:57:54.651 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T14:57:54.756 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T14:57:54.756 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2024-04-02T14:57:54.762 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645/remote/smithi113/crash 2024-04-02T14:57:54.763 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T14:57:54.806 INFO:teuthology.orchestra.run.smithi113.stderr:tar: /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T14:57:54.806 INFO:teuthology.orchestra.run.smithi113.stderr:tar: Error is not recoverable: exiting now 2024-04-02T14:57:54.808 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645/remote/smithi204/crash 2024-04-02T14:57:54.809 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T14:57:54.844 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T14:57:54.844 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-04-02T14:57:54.845 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-02T14:57:54.845 DEBUG:teuthology.orchestra.run.smithi062:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-02T14:57:54.960 INFO:teuthology.orchestra.run.smithi062.stdout:2024-04-02T14:47:03.258267+0000 mds.g (mds.2) 15 : cluster [WRN] Scrub error on inode 0x10000002ba7 (/volumes/qa/sv_1/578900f3-6c0c-478c-90ae-75c0f7bd10b8/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mn10300/compartmentalisation.txt) see mds.g log and `damage ls` output for details 2024-04-02T14:57:54.961 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-02T14:57:54.961 INFO:tasks.cephadm:Compressing logs... 2024-04-02T14:57:54.961 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T14:57:54.966 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T14:57:54.968 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T14:57:54.996 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T14:57:54.996 INFO:teuthology.orchestra.run.smithi204.stderr:: No such file or directory 2024-04-02T14:57:54.996 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.l.log 2024-04-02T14:57:54.997 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T14:57:54.997 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.i.log 2024-04-02T14:57:54.997 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.11.log 2024-04-02T14:57:54.997 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mgr.z.log 2024-04-02T14:57:54.997 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.8.log 2024-04-02T14:57:54.997 INFO:teuthology.orchestra.run.smithi204.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T14:57:54.997 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.5.log 2024-04-02T14:57:54.997 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.f.log 2024-04-02T14:57:54.997 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T14:57:55.005 INFO:teuthology.orchestra.run.smithi113.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T14:57:55.005 INFO:teuthology.orchestra.run.smithi113.stderr:: No such file or directory 2024-04-02T14:57:55.005 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.10.log 2024-04-02T14:57:55.005 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T14:57:55.006 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.b.log 2024-04-02T14:57:55.006 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.10.log: /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.4.log 2024-04-02T14:57:55.006 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.1.log 2024-04-02T14:57:55.006 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.b.log: /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.k.log 2024-04-02T14:57:55.006 INFO:teuthology.orchestra.run.smithi113.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T14:57:55.006 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.h.log 2024-04-02T14:57:55.006 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.7.log 2024-04-02T14:57:55.006 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.e.log 2024-04-02T14:57:55.006 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.7.log: 90.4% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T14:57:55.006 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mgr.y.log 2024-04-02T14:57:55.007 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.f.log: 90.4% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T14:57:55.007 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T14:57:55.007 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T14:57:55.007 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.2.log 2024-04-02T14:57:55.007 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.log: 94.6% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.l.log.gz 2024-04-02T14:57:55.007 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.c.log 2024-04-02T14:57:55.007 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.2.log: 87.5% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T14:57:55.007 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T14:57:55.008 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.c.log: 91.9% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mgr.z.log.gz 2024-04-02T14:57:55.008 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mon.c.log 2024-04-02T14:57:55.013 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T14:57:55.019 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-volume.log: /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T14:57:55.032 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T14:57:55.033 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.cephadm.log: 91.9% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mgr.y.log.gz 2024-04-02T14:57:55.041 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-02T14:57:55.041 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T14:57:55.042 INFO:teuthology.orchestra.run.smithi113.stderr: 83.5%gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T14:57:55.042 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.6.log 2024-04-02T14:57:55.043 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mon.a.log 2024-04-02T14:57:55.044 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T14:57:55.046 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.log: -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T14:57:55.046 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.g.log 2024-04-02T14:57:55.047 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.j.log 2024-04-02T14:57:55.049 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.3.log 2024-04-02T14:57:55.051 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mgr.x.log 2024-04-02T14:57:55.052 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.3.log: 90.2% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T14:57:55.052 INFO:teuthology.orchestra.run.smithi062.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T14:57:55.053 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.0.log 2024-04-02T14:57:55.055 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T14:57:55.057 INFO:teuthology.orchestra.run.smithi113.stderr: 87.5% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T14:57:55.058 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.0.log: /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.cephadm.log: 94.5% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.j.log.gz 2024-04-02T14:57:55.058 INFO:teuthology.orchestra.run.smithi062.stderr: 83.9% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T14:57:55.058 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T14:57:55.060 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T14:57:55.061 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mon.b.log 2024-04-02T14:57:55.063 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.log: /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.9.log 2024-04-02T14:57:55.065 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T14:57:55.067 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.d.log 2024-04-02T14:57:55.069 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.a.log 2024-04-02T14:57:55.088 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-volume.log: /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T14:57:55.100 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.d.log: /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.a.log: 90.8% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T14:57:55.135 INFO:teuthology.orchestra.run.smithi113.stderr: 94.0% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.e.log.gz 2024-04-02T14:57:55.167 INFO:teuthology.orchestra.run.smithi204.stderr: 94.5% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.f.log.gz 2024-04-02T14:57:55.239 INFO:teuthology.orchestra.run.smithi062.stderr: 94.2% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.d.log.gz 2024-04-02T14:57:55.265 INFO:teuthology.orchestra.run.smithi113.stderr: 94.6% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.k.log.gz 2024-04-02T14:57:55.466 INFO:teuthology.orchestra.run.smithi204.stderr: 91.7% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mon.c.log.gz 2024-04-02T14:57:55.469 INFO:teuthology.orchestra.run.smithi062.stderr: 90.1% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mgr.x.log.gz 2024-04-02T14:57:55.628 INFO:teuthology.orchestra.run.smithi113.stderr: 91.8% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mon.b.log.gz 2024-04-02T14:57:55.677 INFO:teuthology.orchestra.run.smithi062.stderr: 93.4% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.a.log.gz 2024-04-02T14:57:55.779 INFO:teuthology.orchestra.run.smithi204.stderr: 93.8% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.i.log.gz 2024-04-02T14:57:56.083 INFO:teuthology.orchestra.run.smithi062.stderr: 93.7% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.g.log.gz 2024-04-02T14:57:56.091 INFO:teuthology.orchestra.run.smithi062.stderr: 90.5% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mon.a.log.gz 2024-04-02T14:57:57.360 INFO:teuthology.orchestra.run.smithi113.stderr: 93.8% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.h.log.gz 2024-04-02T14:57:57.612 INFO:teuthology.orchestra.run.smithi113.stderr: 93.6% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.10.log.gz 2024-04-02T14:57:57.647 INFO:teuthology.orchestra.run.smithi204.stderr: 93.5% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.2.log.gz 2024-04-02T14:57:57.720 INFO:teuthology.orchestra.run.smithi113.stderr: 93.7% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.4.log.gz 2024-04-02T14:57:57.736 INFO:teuthology.orchestra.run.smithi062.stderr: 93.5% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.9.log.gz 2024-04-02T14:57:58.052 INFO:teuthology.orchestra.run.smithi062.stderr: 93.5% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.6.log.gz 2024-04-02T14:57:58.069 INFO:teuthology.orchestra.run.smithi204.stderr: 93.5% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.5.log.gz 2024-04-02T14:57:58.232 INFO:teuthology.orchestra.run.smithi204.stderr: 95.2% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.c.log.gz 2024-04-02T14:57:58.303 INFO:teuthology.orchestra.run.smithi204.stderr: 93.6% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.8.log.gz 2024-04-02T14:57:58.505 INFO:teuthology.orchestra.run.smithi204.stderr: 93.8% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.11.log.gz 2024-04-02T14:57:58.507 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-04-02T14:57:58.507 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m3.525s 2024-04-02T14:57:58.507 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m16.629s 2024-04-02T14:57:58.507 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.573s 2024-04-02T14:57:58.518 INFO:teuthology.orchestra.run.smithi062.stderr: 93.5% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.0.log.gz 2024-04-02T14:57:58.541 INFO:teuthology.orchestra.run.smithi113.stderr: 93.6% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.1.log.gz 2024-04-02T14:57:58.659 INFO:teuthology.orchestra.run.smithi113.stderr: 93.6% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.7.log.gz 2024-04-02T14:57:58.773 INFO:teuthology.orchestra.run.smithi062.stderr: 93.4% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-osd.3.log.gz 2024-04-02T14:57:58.777 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-02T14:57:58.777 INFO:teuthology.orchestra.run.smithi062.stderr:real 0m3.777s 2024-04-02T14:57:58.777 INFO:teuthology.orchestra.run.smithi062.stderr:user 0m14.931s 2024-04-02T14:57:58.778 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m1.174s 2024-04-02T14:58:32.447 INFO:teuthology.orchestra.run.smithi113.stderr: 93.0% -- replaced with /var/log/ceph/9e86b65e-f0fd-11ee-b647-cb9ed24678a4/ceph-mds.b.log.gz 2024-04-02T14:58:32.448 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-02T14:58:32.449 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m37.467s 2024-04-02T14:58:32.449 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m51.529s 2024-04-02T14:58:32.449 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m1.602s 2024-04-02T14:58:32.449 INFO:tasks.cephadm:Archiving logs... 2024-04-02T14:58:32.449 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645/remote/smithi062/log 2024-04-02T14:58:32.451 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T14:58:34.488 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645/remote/smithi113/log 2024-04-02T14:58:34.489 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T14:58:42.072 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645/remote/smithi204/log 2024-04-02T14:58:42.073 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T14:58:44.223 INFO:tasks.cephadm:Removing cluster... 2024-04-02T14:58:44.223 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 --force 2024-04-02T14:58:44.564 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:58:45.259 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 --force 2024-04-02T14:58:45.450 INFO:teuthology.orchestra.run.smithi113.stdout:Deleting cluster with fsid: 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:58:45.799 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 --force 2024-04-02T14:58:45.991 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 9e86b65e-f0fd-11ee-b647-cb9ed24678a4 2024-04-02T14:58:46.268 INFO:tasks.cephadm:Removing cephadm ... 2024-04-02T14:58:46.268 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T14:58:46.329 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T14:58:46.346 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T14:58:46.361 INFO:tasks.cephadm:Teardown complete 2024-04-02T14:58:46.361 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-02T14:58:46.373 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-02T14:58:46.373 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T14:58:46.381 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T14:58:46.388 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T14:58:46.546 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-02T14:58:46.557 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-02T14:58:46.557 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:58:46.565 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:58:46.568 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:58:46.579 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:58:46.581 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:58:46.584 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:58:46.584 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-04-02T14:58:46.584 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 65 -576us[ -585us] +/- 93ms 2024-04-02T14:58:46.584 INFO:teuthology.orchestra.run.smithi113.stdout:^* hv02.front.sepia.ceph.com 3 7 377 0 -1187us[-1194us] +/- 82ms 2024-04-02T14:58:46.585 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +1621us[+1614us] +/- 72ms 2024-04-02T14:58:46.585 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:58:46.586 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:58:46.586 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-04-02T14:58:46.586 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 63 -580us[ -583us] +/- 93ms 2024-04-02T14:58:46.586 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv02.front.sepia.ceph.com 3 6 377 0 -1252us[-1254us] +/- 82ms 2024-04-02T14:58:46.586 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 0 +1584us[+1584us] +/- 72ms 2024-04-02T14:58:46.586 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:58:46.617 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:58:46.628 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:58:46.628 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-04-02T14:58:46.628 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 62 -613us[ -613us] +/- 93ms 2024-04-02T14:58:46.628 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 0 -1261us[-1261us] +/- 82ms 2024-04-02T14:58:46.628 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv03.front.sepia.ceph.com 3 6 377 64 +1602us[+1597us] +/- 71ms 2024-04-02T14:58:46.628 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:58:46.630 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-02T14:58:46.641 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-02T14:58:46.642 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-02T14:58:46.653 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T14:58:46.905 DEBUG:teuthology.orchestra.run.smithi113:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T14:58:46.978 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T14:58:47.051 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:58:47.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:58:47.161 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:58:47.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:58:47.192 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:58:47.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:58:47.222 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-02T14:58:47.233 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-02T14:58:47.244 INFO:teuthology.task.internal:Duration was 2190.444101 seconds 2024-04-02T14:58:47.245 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-02T14:58:47.256 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-02T14:58:47.256 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T14:58:47.264 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T14:58:47.267 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T14:58:47.307 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:58:47.309 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:58:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:58:47.927 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-02T14:58:47.927 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-04-02T14:58:47.927 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T14:58:47.987 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-04-02T14:58:47.987 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T14:58:48.011 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T14:58:48.012 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T14:58:48.034 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-02T14:58:48.035 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T14:58:48.039 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T14:58:48.054 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T14:58:48.105 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-02T14:58:48.105 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T14:58:48.109 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T14:58:48.123 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T14:58:48.597 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-02T14:58:48.610 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-02T14:58:48.610 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T14:58:48.698 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T14:58:48.726 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T14:58:48.754 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-02T14:58:48.768 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:58:48.776 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:58:48.778 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:58:48.801 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-04-02T14:58:48.820 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-04-02T14:58:48.881 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-04-02T14:58:48.938 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:58:48.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:58:48.991 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:58:49.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:58:49.005 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:58:49.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:58:49.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-02T14:58:49.032 INFO:teuthology.task.internal:Transferring archived files... 2024-04-02T14:58:49.033 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645/remote/smithi062 2024-04-02T14:58:49.033 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T14:58:49.156 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645/remote/smithi113 2024-04-02T14:58:49.156 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T14:58:49.195 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645/remote/smithi204 2024-04-02T14:58:49.195 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T14:58:49.235 INFO:teuthology.task.internal:Removing archive directory... 2024-04-02T14:58:49.236 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T14:58:49.244 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T14:58:49.246 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T14:58:49.293 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-02T14:58:49.306 INFO:teuthology.task.internal:Not uploading archives. 2024-04-02T14:58:49.306 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-02T14:58:49.318 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-02T14:58:49.318 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T14:58:49.324 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T14:58:49.326 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T14:58:49.338 INFO:teuthology.orchestra.run.smithi113.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 14:58 /home/ubuntu/cephtest 2024-04-02T14:58:49.340 INFO:teuthology.orchestra.run.smithi204.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 14:58 /home/ubuntu/cephtest 2024-04-02T14:58:49.377 INFO:teuthology.orchestra.run.smithi062.stdout: 393692 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 14:58 /home/ubuntu/cephtest 2024-04-02T14:58:49.381 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-02T14:58:49.394 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-02T14:58:49.429 INFO:teuthology.nuke:Checking targets against current locks 2024-04-02T14:58:49.452 DEBUG:teuthology.nuke:shortname: smithi062 2024-04-02T14:58:49.452 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:58:49.475 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 14:12:55.520513', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHX+vuiFpHaCuctNjO+X9/cOuGDm1+16BZqDUdWfrbPaHTF0FCJlawslQwUZPL26zMPphDbRjn7DyK623T3crlM='} 2024-04-02T14:58:49.477 DEBUG:teuthology.nuke:shortname: smithi113 2024-04-02T14:58:49.477 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:58:49.499 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 14:12:55.524050', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBHea/tAzbc/det7dLVqiIpOXfZaZrD6vzQc4SU8xJUfIIuFR/TZrB3p+rxVCqfHWxUwYCmfNx2mBHS2W4ykynM='} 2024-04-02T14:58:49.503 DEBUG:teuthology.nuke:shortname: smithi204 2024-04-02T14:58:49.504 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:58:49.524 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635645', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 14:12:55.522505', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAHQ9umABXGvlkXBtgq3+xzBXvq2nBLey/ECKPtEu6tdho/a9bqhetZfH4kePNVWWrnZP8yK9awtNzk5q95fAm0='} 2024-04-02T14:58:49.535 INFO:teuthology.orchestra.console.smithi062:Power off 2024-04-02T14:58:49.535 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:58:49.559 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2024-04-02T14:58:49.559 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:58:49.576 INFO:teuthology.orchestra.console.smithi113:Power off 2024-04-02T14:58:49.576 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:58:49.588 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-02T14:58:49.599 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2024-04-02T14:58:49.599 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:58:49.612 INFO:teuthology.orchestra.console.smithi204:Power off 2024-04-02T14:58:49.613 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:58:49.621 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-04-02T14:58:49.634 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2024-04-02T14:58:49.635 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:58:49.660 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-04-02T14:58:53.589 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:58:53.622 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:58:53.660 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:58:53.700 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-02T14:58:53.731 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-04-02T14:58:53.771 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-04-02T14:58:57.702 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:58:57.732 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:58:57.773 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:58:57.812 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2024-04-02T14:58:57.841 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2024-04-02T14:58:57.881 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2024-04-02T14:58:57.913 INFO:teuthology.orchestra.console.smithi062:Power off completed 2024-04-02T14:58:57.941 INFO:teuthology.orchestra.console.smithi113:Power off completed 2024-04-02T14:58:57.983 INFO:teuthology.orchestra.console.smithi204:Power off completed 2024-04-02T14:58:58.089 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} duration: 2190.4441010951996 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-04-02T14:58:58.089 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:58:58.170 INFO:teuthology.run:FAIL